SA2E-AD: A Stacked Attention Autoencoder for Anomaly Detection in Multivariate Time Series
Authors: 
Mengyao Li, Zhiyong Li, Zhibang Yang, Xu Zhou, Yifan Li, Ziyan Wu, Lingzhao Kong, Ke NaiAuthors Info & Claims
Article No.: 174, Pages 1 - 15
Abstract
Anomaly detection for multivariate time series is an essential task in the modern industrial field. Although several methods have been developed for anomaly detection, they usually fail to effectively exploit the metrical-temporal correlation and the other dependencies among multiple variables. To address this problem, we propose a stacked attention autoencoder for anomaly detection in multivariate time series (SA2E-AD); it focuses on fully utilizing the metrical and temporal relationships among multivariate time series. We design a multiattention block, alternately containing the temporal attention and metrical attention components in a hierarchical structure to better reconstruct normal time series, which is helpful in distinguishing the anomalies from the normal time series. Meanwhile, a two-stage training strategy is designed to further separate the anomalies from the normal data. Experiments on three publicly available datasets show that SA2E-AD outperforms the advanced baseline methods in detection performance and demonstrate the effectiveness of each part of the process in our method.
References
[1]
Subutai Ahmad, Alexander Lavin, Scott Purdy, and Zuha Agha. 2017. Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262 (June 2017), 134–147.
[2]
Chuadhry Mujeeb Ahmed, Venkata Reddy Palleti, and Aditya P. Mathur. 2017. WADI: A water distribution testbed for research in the design of secure cyber physical systems. In Proceedings of the 3rd International Workshop on Cyber-Physical Systems for Smart Water Networks. New York, NY, USA, 25–28.
[3]
M. Shyu, S. Chen, K. Sarinnapakorn, and L. Chang. 2003. A novel anomaly detection scheme based on principal component classifier. In Proc Icdm Foundation & New Direction of Data Mining Workshop.
[4]
Julien Audibert, Pietro Michiardi, Frédéric Guyard, Sébastien Marti, and Maria A. Zuluaga. 2020. USAD: Unsupervised anomaly detection on multivariate time series. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3395–3404.
[5]
Md. Abul Bashar and Richi Nayak. 2020. TAnoGAN: Time series anomaly detection with generative adversarial networks. In 2020 IEEE Symposium Series on Computational Intelligence (SSCI’20). Canberra, ACT, Australia, 1778–1785.
[6]
Paul Boniol, Michele Linardi, Federico Roncallo, and Themis Palpanas. 2020. Automated anomaly detection in large sequences. In 2020 IEEE 36th International Conference on Data Engineering (ICDE’20). 1834–1837.
[7]
M. M. Breunig, H. P. Kriegel, R. T. Ng, and J. Sander. 2000. LOF: Identifying density-based local outliers. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data 29, 2 (2000), 93–104.
[8]
Ailin Deng and Bryan Hooi. 2021. Graph neural network-based anomaly detection in multivariate time series. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4027–4035.
[9]
Alexander Geiger, Dongyu Liu, Sarah Alnegheimish, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. 2020. TadGAN: Time series anomaly detection using generative adversarial networks. In 2020 IEEE International Conference on Big Data (Big Data’20). Atlanta, GA, USA, 33–43.
[10]
Manish Gupta, Jing Gao, Charu C. Aggarwal, and Jiawei Han. 2013. Outlier detection for temporal data: A survey. IEEE Transactions on Knowledge and Data Engineering 26, 9 (2013), 2250–2267.
[11]
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. 2018. Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, NY, 387–395.
[12]
I. Kiss, B. Genge, P. Haller, and G. Sebestyen. 2014. Data clustering-based anomaly detection in industrial control systems. In 2014 IEEE 10th International Conference on Intelligent Computer Communication and Processing (ICCP’14). 275–281.
[13]
Alexander Lavin and Subutai Ahmad. 2015. Evaluating real-time anomaly detection algorithms–the Numenta anomaly benchmark. In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA’15). 38–44.
[14]
C. K. Lee, Y. J. Cheon, and W. Y. Hwang. 2021. Studies on the GAN-based anomaly detection methods for the time series data. IEEE Access 9, 99 (2021), 73201–73215.
[15]
Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See-Kiong Ng. 2019. MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks. In International Conference on Artificial Neural Networks. Cham, 703–716.
[16]
Yifan Li, Xiaoyan Peng, Jia Zhang, Zhiyong Li, and Ming Wen. 2023. DCT-GAN: Dilated convolutional transformer-based GAN for time series anomaly detection. IEEE Transactions on Knowledge and Data Engineering 35, 4 (2023), 3632–3644.
[17]
J. Ma and S. Perkins. 2003. Time-series novelty detection using one-class support vector machines. In Proceedings of the International Joint Conference on Neural Networks, 2003, Vol. 3. 1741–1745.
[18]
Aditya P. Mathur and Nils Ole Tippenhauer. 2016. SWaT: A water treatment testbed for research and training on ICS security. In 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater’16). Vienna, Austria, 31–36.
[19]
Mohsin Munir, Shoaib Ahmed Siddiqui, Andreas Dengel, and Sheraz Ahmed. 2018. DeepAnT: A deep learning approach for unsupervised anomaly detection in time series. IEEE Access 7 (December 2018), 1991–2005.
[20]
Sasho Nedelkoski, Jasmin Bogatinovski, Ajay Kumar Mandapati, Soeren Becker, Jorge Cardoso, and Odej Kao. 2020. Multi-source distributed system data for AI-powered analytics. In European Conference on Service-Oriented and Cloud Computing. Cham, 161–176.
[21]
Bernhard Schölkopf, John C. Platt, John Shawe-Taylor, Alex J. Smola, and Robert C. Williamson. 2001. Estimating the support of a high-dimensional distribution. Neural Computation 13, 7 (2001), 1443–1471.
[22]
Lifeng Shen, Zhuocong Li, and James T. Kwok. 2020. Timeseries anomaly detection using temporal hierarchical one-class network. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6–12, 2020, Virtual.
[23]
A. Siffer, P. A. Fouque, A. Termier, and C. Largouet. 2017. Anomaly detection in streams with extreme value theory. In ACM SIGKDD International Conference. 1067–1075.
[24]
Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. 2019. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Anchorage, AK, USA, 2828–2837.
[25]
M. Thill, W. Konen, H. Wang, and T. Bck. 2021. Temporal convolutional autoencoder for unsupervised anomaly detection in time series. Applied Soft Computing 112, 3 (2021), 107751–107751.
[26]
S. Tuli, G. Casale, and N. R. Jennings. 2022. TranAD: Deep transformer networks for anomaly detection in multivariate time series data. Proc. VLDB Endow. 15, 6 (2022), 1201–1214.
[27]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA, 6000–6010.
[28]
D. J. Weller-Fahy, B. J. Borghetti, and A. A. Sodemann. 2015. A survey of distance and similarity measures used within network intrusion anomaly detection. IEEE Communications Surveys & Tutorials 17, 1 (2015), 70–91.
[29]
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. 2022. Anomaly transformer: Time series anomaly detection with association discrepancy. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25–29, 2022. Virtual Event.
[30]
Liyan Xu, Kang Xu, Yinchuan Qin, Yixuan Li, Xingting Huang, Zhicheng Lin, Ning Ye, and Xuechun Ji. 2022. TGAN-AD: Transformer-based GAN for anomaly detection of time series data. Applied Sciences 12, 16 (2022), 8085.
[31]
Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. 2019. Time-series generative adversarial networks. Advances in Neural Information Processing Systems 32 (December 2019), 5508–5518.
[32]
Chuxu Zhang, Dongjin Song, Yuncong Chen, Xinyang Feng, Cristian Lumezanu, Wei Cheng, Jingchao Ni, Bo Zong, Haifeng Chen, and Nitesh V. Chawla. 2019. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 1409–1416.
[33]
Hang Zhao, Yujing Wang, Juanyong Duan, Congrui Huang, Defu Cao, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. 2020. Multivariate time-series anomaly detection via graph attention network. In 2020 IEEE International Conference on Data Mining (ICDM’20). Sorrento, Italy, 841–850.
[34]
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. 2018. Deep autoencoding Gaussian mixture model for unsupervised anomaly detection. In International Conference on Learning Representations. Toulon, France, 1–19.
Index Terms
- SA2E-AD: A Stacked Attention Autoencoder for Anomaly Detection in Multivariate Time Series
Recommendations
Deep learning for anomaly detection in multivariate time series: Approaches, applications, and challenges
AbstractAnomaly detection has recently been applied to various areas, and several techniques based on deep learning have been proposed for the analysis of multivariate time series. In this study, we classify the anomalies into three types, ...
Highlights- The methods for anomaly detection on multivariate time series are reviewed.
- The ...
Exact variable-length anomaly detection algorithm for univariate and multivariate time series
The problem of anomaly detection in time series has received a lot of attention in the past two decades. However, existing techniques cannot locate where the anomalies are within anomalous time series, or they require users to provide the length of ...
Adversarial Autoencoder for Unsupervised Time Series Anomaly Detection and Interpretation
WSDM '23: Proceedings of the Sixteenth ACM International Conference on Web Search and Data MiningIn many complex systems, devices are typically monitored and generating massive multivariate time series. However, due to the complex patterns and little useful labeled data, it is a great challenge to detect anomalies from these time series data. ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 19 June 2024
Online AM: 26 March 2024
Accepted: 14 March 2024
Revised: 07 September 2023
Received: 16 October 2022
Published in TKDD Volume 18, Issue 7
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- State Grid Science and Technology Project
- Hunan Provincial Natural Science Foundation of China
- Special Project of the Foshan Science and Technology Innovation Team
- Ant Group through CCF-Ant Research Fund
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 403Total Downloads
- Downloads (Last 12 months)403
- Downloads (Last 6 weeks)55
Reflects downloads up to 19 Nov 2024
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in