摘要
针对目前流数据存在数量巨大、生成迅速和概念漂移的特点,提出了一种基于长短期记忆(LSTM)网络和滑动窗口的流数据异常检测方法。首先采用LSTM网络进行数据预测,之后计算预测值与实际值的差值。对于每个数据,选择合适的滑动窗口,将滑动窗口区间内的所有差值进行分布建模,再根据每个差值在当前分布的概率密度来计算数据异常可能性。LSTM网络不仅可以进行数据预测,还可以边预测边学习,实时更新调整网络,保证模型的有效性;而利用滑动窗口可以使得异常分数的分配更为合理。最后使用在真实数据基础上制造的模拟数据进行了实验。实验结果验证了所提方法在低噪声环境下比直接利用差值进行检测和异常数据分布建模法(ADM)方法的平均曲线下面积(AUC)值分别提高了0.187和0.05。
Aiming at the characteristics of large volume,rapid generation and concept drift of current stream data,a stream data anomaly detection method based on Long Short-Term Memory(LSTM)network and sliding window was proposed.Firstly,the LSTM network was used for data prediction,and the difference between the predicted value and the actual value was calculated.For each datum,the appropriate sliding window was selected,and the distribution modeling was performed to all the differences in the sliding window interval,then the probability of data anomaly was calculated according to the probability density of each difference in the current distribution.The LSTM network was not only able to predict data,but also able to predict and learn at the same time,as well as update and adjust the network in real time to ensure the validity of the model.The use of sliding windows was able to make the allocation of abnormal scores more reasonable.Finally,the simulation data made on the basis of real data were used for experiment.The experimental results verify that the average Area Under Curve(AUC)value of the proposed method in low-noise environment is 0.187 and 0.05 higher than that of direct difference detection and Abnormal data Distribution Modeling(ADM)method,respectively.
作者
仇媛
常相茂
仇倩
彭程
苏善婷
QIU Yuan;Chang Xiangmao;QIU Qian;PENG Cheng;SU Shanting(College of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing Jiangsu 211106,China;College of Artificial Intelligence and Data Science,Hebei University of Technology,Tianjin 300401,China)
出处
《计算机应用》
CSCD
北大核心
2020年第5期1335-1339,共5页
journal of Computer Applications
关键词
流数据
异常检测
滑动窗口
长短期记忆网络
神经网络
stream data
anomaly detection
sliding window
Long Short-Term Memory(LSTM)network
neural network
