Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Dynamic deep graph convolution with enhanced transformer networks for time series anomaly detection in IoT

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Anomaly detection of multi-time series data during the working process of Internet of Things systems that utilize sensors is one of the key aspects to prevent accidents in industrial information systems. The key challenge is to discover generalized normal patterns by capturing spatio-temporal correlations in multi-sensor data. However, most of the existing studies face the following challenges: (1) Complex topologies and nonlinear connectivity among sensors lack effective characterization methods. (2) Sophisticated correlations among time series need to be mined deeply. Therefore, we propose a novel dynamic deep graph convolution with enhanced transformer networks (DDGCT) for time series anomaly detection. We first construct a dynamic deep graph convolutional network to automatically learn the complex spatial dependencies of sensor data, which introduces \(l_0\) norm with Hard Concrete distribution to further guide the optimization of graph structure in graph learning. Meanwhile, we devise a new transformer model to deeply mine temporal dependencies from time-series data by designing a new positional encoding coupled with patch design as well as channel independence constraint. Then, DDGCT fuses and optimizes the captured temporal and deep spatial features using attention networks. Finally, anomaly scores are efficiently computed by prediction methods with threshold-based approaches to detect anomalies. Extensive experiments on real datasets show that DDGCT outperforms several state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

SMD is publicly published on github at https://github.com/NetManAIOps/OmniAnomaly (Reference 11). SMAP and MSL are two public datasets from NASA at https://github.com/khundman/telemanom (Reference 35).

References

  1. Vu, L., Cao, V.L., Nguyen, Q.U., Nguyen, D.N., Hoang, D.T., Dutkiewicz, E.: Learning latent representation for IoT anomaly detection. IEEE Trans. Cybern. 52(5), 3769–3782 (2022)

    Article  Google Scholar 

  2. Deng, A., Hooi, B.: Graph neural network-based anomaly detection in multivariate time series. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 4027–4035 (2021)

  3. Mohammadi, M., Al-Fuqaha, A., Sorour, S., Guizani, M.: Deep learning for IoT big data and streaming analytics: a survey. IEEE Commun. Surv. Tutor. 20(4), 2923–2960 (2018)

    Article  Google Scholar 

  4. Zhang, Y., Chen, Y., Wang, J., Pan, Z.: Unsupervised deep anomaly detection for multi-sensor time-series signals. IEEE Trans. Knowl. Data Eng. 35(1), 2118–2132 (2023)

    Google Scholar 

  5. Gao, R., He, W., Yan, L., Liu, D., Yu, Y., Ye, Z.: Hybrid graph transformer networks for multivariate time series anomaly detection. J. Supercomput. 80(1), 642–669 (2024)

    Article  Google Scholar 

  6. Paffenroth, R.C., du Toit, P., Nong, R., Scharf, L., Jayasumana, A.P., Bandara, V.W.: Space-time signal processing for distributed pattern detection in sensor networks. IEEE J. Select. Topics Signal Process. 7(1), 38–49 (2013)

    Article  Google Scholar 

  7. Sakurada, M., Yairi, T.: Anomaly detection using autoencoders with nonlinear dimensionality reduction. In: Proceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis, p. 4 (2014)

  8. Khan, W., Haroon, M.: A pilot study and survey on methods for anomaly detection in online social networks. In: Human-Centric Smart Computing: Proceedings of ICHCSC 2022, pp. 119–128 (2022)

  9. Khan, W., Haroon, M.: An efficient framework for anomaly detection in attributed social networks. Int. J. Inf. Technol. 14(6), 3069–3076 (2022)

    Google Scholar 

  10. Malhotra, P., Vig, L., Shroff, G., Agarwal, P.: Long short term memory networks for anomaly detection in time series. In: Proceedings of the 23rd European Symposium on Artificial Neural Networks (2015)

  11. Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., Pei, D.: Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, vol. 35, pp. 2828–2837 (2019)

  12. Li, D., Chen, D., Jin, B., Shi, L., Goh, J., Ng, S.-K.: Mad-gan: Multivariate anomaly detection for time series data with generative adversarial networks. In: Proceedings of the 28th International Conference on Artificial Neural Networks, pp. 703–716 (2019)

  13. Khan, W., Haroon, M.: An unsupervised deep learning ensemble model for anomaly detection in static attributed social networks. Int. J. Cognit. Comput. Eng. 3, 153–160 (2022)

    Google Scholar 

  14. Khan, W., Haroon, M., Khan, A.N., Hasan, M.K., Khan, A., Mokhtar, U.A., Islam, S.: Dvaegmm: dual variational autoencoder with gaussian mixture model for anomaly detection on attributed networks. IEEE Access 10, 91160–91176 (2022)

    Article  Google Scholar 

  15. Khan, W., Abidin, S., Arif, M., Ishrat, M., Haleem, M., Shaikh, A.A., Farooqui, N.A., Faisal, S.M.: Anomalous node detection in attributed social networks using dual variational autoencoder with generative adversarial networks. Data Sci. Manag. 2(7), 89–98 (2024)

    Article  Google Scholar 

  16. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 4–24 (2021)

    Article  MathSciNet  Google Scholar 

  17. Zhao, H., Wang, Y., Duan, J., Huang, C., Cao, D., Tong, Y., Xu, B., Bai, J., Tong, J., Zhang, Q.: Multivariate time-series anomaly detection via graph attention network. In: Proceedings of the 20th IEEE International Conference on Data Mining, pp. 841–850 (2020)

  18. Wang, Y., Xu, Y., Yang, J., Wu, M., Li, X., Xie, L., Chen, Z.: Fully-connected spatial-temporal graph for multivariate time-series data. In: Proceedings of the 38th AAAI Conference on Artificial Intelligence, pp. 15715–15724 (2024)

  19. Wu, Z., Pan, S., Long, G., Jiang, J., Chang, X., Zhang, C.: Connecting the dots: Multivariate time series forecasting with graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 753–763 (2020)

  20. Zhang, W., Zhang, C., Tsung, F.: Grelen: Multivariate time series anomaly detection from the perspective of graph relational learning. In: Proceedings of the 31th International Joint Conference on Artificial Intelligence, pp. 2390–2397 (2022)

  21. Chen, Z., Chen, D., Zhang, X., Yuan, Z., Cheng, X.: Learning graph structures with transformer for multivariate time-series anomaly detection in IoT. IEEE Internet Things J. 9(12), 9179–9189 (2021)

    Article  Google Scholar 

  22. Tuli, S., Casale, G., Jennings, N.R.: Tranad: Deep transformer networks for anomaly detection in multivariate time series data. VLDB Endowment 15(6), 1201–1214 (2022)

    Article  Google Scholar 

  23. Nie, Y., Nguyen, N.H., Sinthong, P., Kalagnanam, J.: A time series is worth 64 words: Long-term forecasting with transformers. In: Proceedings of the 11th International Conference on Learning Representations (2023)

  24. Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., Long, M.: itransformer: Inverted transformers are effective for time series forecasting. In: Proceedings of The 12th International Conference on Learning Representations (2024)

  25. Zeng, A., Chen, M., Zhang, L., Xu, Q.: Are transformers effective for time series forecasting? In: Proceedings of the 37th AAAI Conference on Artificial Intelligence, pp. 11121–11128 (2023)

  26. Schlichtkrull, M.S., Nicola, C.D., Titov, I.: Interpreting graph neural networks for nlp with differentiable edge masking. In: Proceedings of the 9th International Conference on Learning Representation (2020)

  27. Louizos, C., Welling, M., Kingma, D.: Learning sparse neural networks through \({L_0}\) regularization. In: Proceedings of the 6th International Conference on Learning Representations (2018)

  28. Liu, M., Gao, H., Ji, S.: Towards deeper graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 338–348 (2020)

  29. Yang, T., Deng, J., Quan, X., Wang, Q.: Orders are unwanted: dynamic deep graph convolutional network for personality detection. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence, pp. 13896–13904 (2023)

  30. Andrew, A.C., Misirli, G., Fan, Z.: Anomaly detection for IoT time-series data: a survey. IEEE Internet Things J. 7(7), 6481–6494 (2020)

    Article  Google Scholar 

  31. Yaacob, A.H., Tan, I.K., Chien, S.F., Tan, H.K.: Arima based network anomaly detection. In: Procedding of the 2nd International Conference on Communication Software and Networks, pp. 205–209 (2010)

  32. Williamson, B.S.: Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)

    Article  Google Scholar 

  33. Tax, D.J., Duin, R.W.: Support vector data description. Mach. Learn. 54(1), 45–66 (2004)

    Article  Google Scholar 

  34. Ren, H., Xu, B., Wang, Y., Yi, C., Huang, C., Kou, X., Xing, T., Yang, M., Tong, J., Zhang, Q.: Time-series anomaly detection service at microsoft. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3009–3017 (2019)

  35. Hundman, K., Constantinou, V., Laporte, C., Colwell, I., Soderstrom, T.: Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 387–395 (2018)

  36. Audibert, J., Michiardi, P., Guyard, F., Marti, S., Zuluaga, M.A.: Usad: Unsupervised anomaly detection on multivariate time series. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3395–3404 (2020)

  37. Peng, W., Hong, X., Chen, H., Zhao, G.: Learning graph convolutional network for skeleton-based human action recognition by neural searching. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence, pp. 2669–2676 (2020)

  38. Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T.: Session-based recommendation with graph neural networks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence, pp. 346–353 (2019)

  39. Renhe, J., Zhaonan, W., Jiawei, Y., Puneet, J., Quanjun, C., Yasumasa, K., Xuan, S., Shintaro, F., Toyotaro, S.: Spatio-temporal meta-graph learning for traffic forecasting. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence, pp. 8078–8086 (2023)

  40. Shang, C., Chen, J., Bi, J.: Discrete graph structure learning for forecasting multiple time series. In: Proceedings of the 9th International Conference on Learning Representations (2020)

  41. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of 5th International Conference on Learning Representations (2017)

  42. Ding, C., Sun, S., Zhao, J.: Mst-gat: a multimodal spatial-temporal graph attention network for time series anomaly detection. Inf. Fusion 2023, 527–536 (2023)

    Article  Google Scholar 

  43. Shan, N., Xu, X., Bao, X., Xu, C., Zhu, G., Wu, E.Q.: Multisensor anomaly detection and interpretable analysis for linear induction motors. IEEE Trans. Intell. Transp. Syst. 24(9), 9861–9870 (2023)

    Article  Google Scholar 

  44. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Proceedings of the Neural Information Processing Systems, pp. 5998–6008 (2017)

  45. Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y.X., Yan, X.: Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in neural information processing systems, 5244–5254 (2019)

  46. Liu, S., Yu, H., Liao, C., Li, J., Lin, W., Liu, A.X., Dustdar, S.: Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In: Proceedings of the 10th International Conference on Learning Representations (2022)

  47. Cirstea, R.-G., Guo, C., Yang, B., Kieu, T., Dong, X., Pan, S.: Triformer: Triangular, variable-specific attentions for long sequence multivariate time series forecasting. In: Proceedings of the 31st International Joint Conference on Artificial Intelligence, pp. 1994–2001 (2022)

  48. Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., Zhang, W.: Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 11106–11115 (2021)

  49. Xu, J., Wu, H., Wang, J., Long, M.: Anomaly transformer: Time series anomaly detection with association discrepancy. In: Proceedings of the 10th International Conference on Learning Representations (2022)

  50. Yang, Y., Zhang, C., Zhou, T., Wen, Q., Sun, L.: Dual attention contrastive representation learning for time series anomaly detection. In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3033–3045 (2023)

  51. Cheng, M., Liu, Q., Liu, Z., Li, Z., Luo, Y., Chen, E.: Formertime: Hierarchical multi-scale representations for multivariate time series classification. In: Proceedings of the ACM Web Conference, pp. 1437–1445 (2023)

  52. Zhang, J., Dai, Q.: Multi-relations aware convolutional attention network for multivariate time series forecasting. Inf. Sci. 643,(2023)

    Article  Google Scholar 

  53. Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., Eickhoff, C.: A transformer-based framework for multivariate time series representation learning. In: Proceedings of the the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2114–2124 (2021)

  54. Lee, J., Park, B., Chae, D.-K.: Duogat: Dual time-oriented graph attention networks for accurate, efficient and explainable anomaly detection on time-series. In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pp. 1188–1197 (2023)

  55. Mathur, A.P., Tippenhauer, N.O.: Swat: A water treatment testbed for research and training on ics security. In: 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater), pp. 31–36 (2016)

  56. Ahmed, C.M., Palleti, V.R., Mathur, A.P.: Wadi: a water distribution testbed for research in the design of secure cyber physical systems. In: Proceedings of the 3rd International Workshop on Cyber-physical Systems for Smart Water Networks, pp. 25–28 (2017)

  57. Li, W., Hu, W., Chen, T., Chen, N., Feng, C.: Stackvae-g: an efficient and interpretable model for time series anomaly detection. AI Open 3, 101–110 (2022)

    Article  Google Scholar 

Download references

Funding

This work is supported by the Ministry of Education Chunhui Plan Cooperation Project (HZKY20220350).

Author information

Authors and Affiliations

  1. School of Computer Science, Hubei University of Technology, Wuhan, 430068, China

    Rong Gao, Zhiwei Chen & Xinyun Wu

  2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Rong Gao

  3. College of Tongda, Nanjing University of Posts and Telecommunications, Nanjing, 210003, China

    Yonghong Yu

  4. Department of Computer Science, Royal Holloway, University of London, London, Surrey, TW20 0EX, UK

    Li Zhang

Authors
  1. Rong Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiwei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinyun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yonghong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Li Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, Gao R, and Zhang L; methodology, Gao R and Chen Z; software, Chen Z; validation, Gao R, Zhang L and Wu X; formal analysis, Gao R; investigation, Chen Z; resources, Chen Z; data curation, Chen Z and Gao R; writing—original draft preparation, Gao R and Chen Z and Wu X; writing—review and editing, Gao R, Chen Z and Wu X; visualization, Zhang L and Wu X; supervision, Zhang L, Yu Y and Wu X; project administration, Zhang L; funding acquisition, Zhang L and Yu Y and Wu X; All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Xinyun Wu.

Ethics declarations

Conflict of interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, R., Chen, Z., Wu, X. et al. Dynamic deep graph convolution with enhanced transformer networks for time series anomaly detection in IoT. Cluster Comput 28, 15 (2025). https://doi.org/10.1007/s10586-024-04707-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10586-024-04707-w

Keywords

Search

Navigation