Abstract
The cyber threats are often found to threaten individuals, organizations and countries at different levels and evolve continuously over time. Cyber Threat Intelligence (CTI) is an effective approach to solve cyber security problems. However, existing processes are considered inherent responses to known threats. CTI experts recommend proactively checking for emerging threats in existing knowledge. In addition, most researches focus on static snapshots of the CTI knowledge graph, while ignoring the temporal dynamics. To this end, we create a novel framework TSA-TNTM (Time Series Attention based Transformer Neural Turing Machines) for diachronic graph embedding framework, which uses time series self-attention mechanism to capture the non-linearly evolving entity representations over time. We demonstrate significantly improved performance over various approaches. A series of benchmark experiments illustrate that TSA-TNTM could generate higher quality than the state-of-the-art word embedding models in tasks pertaining to semantic analogy, clustering, threat classification and proactively identify emerging threats in CTI fields.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, vol. 26 (2013)
Cao, Z., Xu, Q., Yang, Z., Cao, X., Huang, Q.: Dual quaternion knowledge graph embeddings. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 6894–6902 (2021)
Chen, J., Wang, X., Xu, X.: GC-LSTM: graph convolution embedded LSTM for dynamic link prediction. arXiv preprint arXiv:1812.04206 (2018)
Dasgupta, S.S., Ray, S.N., Talukdar, P.: HyTE: hyperplane-based temporally aware knowledge graph embedding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2001–2011 (2018)
García-Durán, A., Dumančić, S., Niepert, M.: Learning sequence encoders for temporal knowledge graph completion. arXiv preprint arXiv:1809.03202 (2018)
Goel, R., Kazemi, S.M., Brubaker, M., Poupart, P.: Diachronic embedding for temporal knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 3988–3995 (2020)
Graves, A., Wayne, G., Danihelka, I.: Neural turing machines. arXiv preprint arXiv:1410.5401 (2014)
Han, Z., Ma, Y., Wang, Y., Günnemann, S., Tresp, V.: Graph Hawkes neural network for forecasting on temporal knowledge graphs. arXiv preprint arXiv:2003.13432 (2020)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (volume 1: Long papers), pp. 687–696 (2015)
Jiang, T., et al.: Towards time-aware knowledge graph completion. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 1715–1724 (2016)
Jin, W., Qu, M., Jin, X., Ren, X.: Recurrent event network: autoregressive structure inference over temporal knowledge graphs. arXiv preprint arXiv:1904.05530 (2019)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Kumar, S., Zhang, X., Leskovec, J.: Predicting dynamic embedding trajectory in temporal interaction networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1269–1278 (2019)
Leblay, J., Chekol, M.W., Liu, X.: Towards temporal knowledge graph embeddings with arbitrary time precision. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 685–694 (2020)
Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Maheshwari, A., Goyal, A., Hanawal, M.K., Ramakrishnan, G.: DynGAN: generative adversarial networks for dynamic network embedding. In: Graph Representation Learning Workshop at NeurIPS (2019)
Manessi, F., Rozza, A., Manzo, M.: Dynamic graph convolutional networks. Pattern Recogn. 97, 107000 (2020)
Nestor, M.: GitHub has been under a continuous DDoS attack in the last 72 hours (2015)
Niepert, M., Ahmed, M., Kutzkov, K.: Learning convolutional neural networks for graphs. In: International Conference on Machine Learning, pp. 2014–2023. PMLR (2016)
NIST: National vulnerability database (2018). https://nvd.nist.gov/
openTSDB: OpenTSDB. http://opentsdb.net/
Pingle, A., Piplai, A., Mittal, S., Joshi, A., Holt, J., Zak, R.: Relext: relation extraction using deep learning approaches for cybersecurity knowledge graph improvement. In: Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pp. 879–886 (2019)
Rastogi, N., Dutta, S., Zaki, M.J., Gittens, A., Aggarwal, C.: MALOnt: an ontology for malware threat intelligence. In: Wang, G., Ciptadi, A., Ahmadzadeh, A. (eds.) MLHat 2020. CCIS, vol. 1271, pp. 28–44. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59621-7_2
Samtani, S., Zhu, H., Chen, H.: Proactively identifying emerging hacker threats from the dark web: a diachronic graph embedding framework (D-GEF). ACM Trans. Priv. Secur. (TOPS) 23(4), 1–33 (2020)
Sarhan, I., Spruit, M.: Open-CYKG: an open cyber threat intelligence knowledge graph. Knowl.-Based Syst. 233, 107524 (2021)
Shu, X., et al.: Threat intelligence computing. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 1883–1898 (2018)
Trivedi, R., Farajtabar, M., Biswal, P., Zha, H.: DyRep: learning representations over dynamic graphs. In: International Conference on Learning Representations (2019)
Trivedi, R., Farajtabar, M., Wang, Y., Dai, H., Zha, H., Song, L.: Know-evolve: deep reasoning in temporal knowledge graphs. arXiv preprint arXiv:1705.05742 (2017)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, J., Song, G., Wu, Y., Wang, L.: Streaming graph neural networks via continual learning. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 1515–1524 (2020)
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28 (2014)
Xu, C., Nayyeri, M., Alkhoury, F., Yazdi, H.S., Lehmann, J.: Temporal knowledge graph embedding model based on additive time series decomposition. arXiv preprint arXiv:1911.07893 (2019)
Zaremba, W., Sutskever, I., Vinyals, O.: Recurrent neural network regularization. arXiv preprint arXiv:1409.2329 (2014)
Acknowledgements
This research is supported by Key Laboratory of Network Assessment Technology, Chinese Academy of Sciences and Beijing Key Laboratory of Network Security and Protection Technology. We thank the anonymous reviewers for their insightful comments on the paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Song, B., Chen, R., Liu, B., Jiang, Z., Wang, X. (2022). Time Series Attention Based Transformer Neural Turing Machines for Diachronic Graph Embedding in Cyber Threat Intelligence. In: Groen, D., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2022. ICCS 2022. Lecture Notes in Computer Science, vol 13353. Springer, Cham. https://doi.org/10.1007/978-3-031-08760-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-08760-8_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08759-2
Online ISBN: 978-3-031-08760-8
eBook Packages: Computer ScienceComputer Science (R0)