research-article

DACHA: A Dual Graph Convolution Based Temporal Knowledge Graph Representation Learning Method Using Historical Relation

Authors:

Yongjun ZhangAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data (TKDD), Volume 16, Issue 3

Article No.: 46, Pages 1 - 18

https://doi.org/10.1145/3477051

Published: 22 October 2021 Publication History

Abstract

Temporal knowledge graph (TKG) representation learning embeds relations and entities into a continuous low-dimensional vector space by incorporating temporal information. Latest studies mainly aim at learning entity representations by modeling entity interactions from the neighbor structure of the graph. However, the interactions of relations from the neighbor structure of the graph are neglected, which are also of significance for learning informative representations. In addition, there still lacks an effective historical relation encoder to model the multi-range temporal dependencies. In this article, we propose a dual graph convolution network based TKG representation learning method using historical relations (DACHA). Specifically, we first construct the primal graph according to historical relations, as well as the edge graph by regarding historical relations as nodes. Then, we employ the dual graph convolution network to capture the interactions of both entities and historical relations from the neighbor structure of the graph. In addition, the temporal self-attentive historical relation encoder is proposed to explicitly model both local and global temporal dependencies. Extensive experiments on two event based TKG datasets demonstrate that DACHA achieves the state-of-the-art results.

References

[1]

Martin Atzmueller. 2019. Onto model-based anomalous link pattern mining on feature-rich social interaction networks. In Companion Proceedings of the 27th World Wide Web Conference. 1047–1050.

Digital Library

[2]

Trapit Bansal, Da-Cheng Juan, Sujith Ravi, and Andrew McCallum. 2019. A2N: Attending to neighbors for knowledge graph inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 4387–4392.

[3]

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 27th International Conference on Neural Information Processing Systems. 2787–2795.

Digital Library

[4]

Elizabeth Boschee, Jennifer Lautenschlager, Sean O'Brien, Steve Shellman, James Starz, and Michael Ward. 2015. ICEWS coded event data. Harvard Dataverse.

[5]

Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2014. Spectral networks and locally connected networks on graphs. In Proceedings of the 2nd International Conference on Learning Representations.

[6]

Zhengdao Chen, Xiang Li, and Joan Bruna. 2019. Supervised community detection with line graph neural networks. In Proceedings of the 7th International Conference on Learning Representations.

[7]

Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 19th Conference on Empirical Methods in Natural Language Processing. 1724–1734.

[8]

Shib Sankar Dasgupta, Swayambhu Nath Ray, and Partha Talukdar. 2018. HyTE: Hyperplane-based temporally aware knowledge graph embedding. In Proceedings of the 23th Conference on Empirical Methods in Natural Language Processing. 2001–2011.

[9]

Denis Degterev, Kristina Badrutdinova, and Anna Stepanova. 2017. Interconnections among the United States, Russia and China: Does Kissinger's American leadership formula apply? International Organizations Research Journal 12, 1 (2017), 81–109.

[10]

Daniel M. Dunlavy, Tamara G. Kolda, and Evrim Acar. 2011. Temporal link prediction using matrix and tensor factorizations. ACM Transactions on Knowledge Discovery from Data 5, 2 (2011), 1–27.

Digital Library

[11]

Cristóbal Esteban, Volker Tresp, Yinchong Yang, Stephan Baier, and Denis Krompaß. 2016. Predicting the co-evolution of event and knowledge graphs. In Proceedings of the 19th International Conference on Information Fusion. 98–105.

[12]

Chang Gao, Junkun Yan, Shenghua Zhou, Pramod K. Varshney, and Hongwei Liu. 2019. Long short-term memory-based deep recurrent neural networks for target tracking. Information Sciences 502, 2 (2019), 279–296.

[13]

Alberto García-Durán, Sebastijan Dumančić, and Mathias Niepert. 2018. Learning sequence encoders for temporal knowledge graph completion. In Proceedings of the 23rd Conference on Empirical Methods in Natural Language Processing. 4816–4821.

[14]

Xiaobo Guo, Wenfang Lin, Youru Li, Zhongyi Liu, Lin Yang, Shuliang Zhao, and Zhenfeng Zhu. 2020. DKEN: Deep knowledge-enhanced network for recommender systems. Information Sciences 540, 5786 (2020), 263–277.

[15]

Tingsong Jiang, Tianyu Liu, Tao Ge, Lei Sha, Baobao Chang, Sujian Li, and Zhifang Sui. 2016. Towards time-aware knowledge graph completion. In Proceedings of 26th International Conference on Computational Linguistics. 1715–1724.

[16]

Woojeong Jin, Meng Qu, Xisen Jin, and Xiang Ren. 2020. Recurrent event network: Autoregressive structure inference over temporal knowledge graphs. In Proceedings of the 25th Conference on Empirical Methods in Natural Language Processing. 6669–6683.

[17]

Diederik P. Kingma, and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations.

[18]

Thomas N. Kipf, and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations.

[19]

Stefan Kombrink, Tomáš Mikolov, Martin Karafiát, and Lukáš Burget. 2011. Recurrent neural network based language modeling in meeting recognition. In Proceedings of the 12th Annual Conference of the International Speech Communication Association. 1045–1048.

[20]

Julien Leblay, and Melisachew Wudage Chekol. 2018. Deriving validity time in knowledge graph. In Companion Proceedings of the 26th World Wide Web Conference. 1771–1776.

Digital Library

[21]

John Boaz Lee, Ryan A. Rossi, Sungchul Kim, Nesreen K. Ahmed, and Eunyee Koh. 2019. Attention models in graphs: A survey. ACM Transactions on Knowledge Discovery from Data 13, 6 (2019), 1–25.

Digital Library

[22]

Kalev Leetaru, and Philip A. Schrodt. 2013. GDELT: Global data on events, location, and tone, 1979-2012. ISA Annual Convention 2, 4 (2013), 1–49.

[23]

Peipei Li, Haixun Wang, Hongsong Li, and Xindong Wu. 2018. Employing semantic context for sparse information extraction assessment. ACM Transactions on Knowledge Discovery from Data 12, 5 (2018), 1–36.

Digital Library

[24]

Yankai Lin, Zhiyuan Liu, Huanbo Luan, Maosong Sun, Siwei Rao, and Song Liu. 2015. Modeling relation paths for representation learning of knowledge bases. In Proceedings of the 20th Conference on Empirical Methods in Natural Language Processing. 705–714.

[25]

Deepak Nathani, Jatin Chauhan, Charu Sharma, and Manohar Kaul. 2019. Learning attention-based embeddings for relation prediction in knowledge graphs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 4710–4723.

[26]

Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on Machine Learning. 809–816.

Digital Library

[27]

Chen Qiao and Xiao Hu. 2020. A neural knowledge graph evaluator: Combining structural and semantic evidence of knowledge graphs for predicting supportive knowledge in scientific QA. Information Processing and Management 57, 6 (2020), 102309.

[28]

Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In Proceedings of the 15th European Semantic Web Conference. 593–607.

Digital Library

[29]

Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, and Chengqi Zhang. 2018. Bi-directional block self-attention for fast and memory-efficient sequence modeling. In Proceedings of the 6th International Conference on Learning Representations.

[30]

Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. In Proceedings of the 7th International Conference on Learning Representations.

[31]

Xing Tang, Ling Chen, Jun Cui, and Baogang Wei. 2019. Knowledge representation learning with entity descriptions, hierarchical types, and textual relations. Information Processing and Management 56, 3 (2019), 809–822.

[32]

Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. 2017. Know-Evolve: Deep temporal reasoning for dynamic knowledge graphs. In Proceedings of the 34th International Conference on Machine Learning. 3462–3471.

Digital Library

[33]

Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of the 33rd International Conference on Machine Learning. 2071–2080.

Digital Library

[34]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 5998–6008.

Digital Library

[35]

Chenguang Wang, Yangqiu Song, Dan Roth, Ming Zhang, and Jiawei Han. 2016. World knowledge as indirect supervision for document clustering. ACM Transactions on Knowledge Discovery from Data 11, 2 (2016), 1–36.

Digital Library

[36]

Zhen Wang, Jianwen Zhang, and Jianlin Feng. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1112–1119.

Digital Library

[37]

Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the 3rd International Conference on Learning Representations.

[38]

Zhanqiu Zhang, Jianyu Cai, Yongdong Zhang, and Jie Wang. 2020. Learning hierarchy-aware knowledge graph embeddings for link prediction. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. 3065–3072.

[39]

Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019. Heterogeneous graph attention network. In Companion Proceedings of the 27th World Wide Web Conference. 2022–2032.

Digital Library

[40]

Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020. MAGNN: Metapath aggregated graph neural network for heterogeneous graph embedding. In Companion Proceedings of the 28th World Wide Web Conference. 2331–2341.

Digital Library

[41]

Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V. Chawla. 2019. Heterogeneous graph neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 793–803.

Digital Library

[42]

Dasol Hwang, Jinyoung Park, Sunyoung Kwon, Kyung-Min Kim, Jung-Woo Ha, and Hyunwoo J. Kim. 2020. Self-supervised auxiliary learning with meta-paths for heterogeneous graphs. In Proceedings of the 34th Conference on Neural Information Processing Systems.

Digital Library

Cited By

Tang XChen LShi HLyu D(2024)DHyper: A Recurrent Dual Hypergraph Neural Network for Event Prediction in Temporal Knowledge GraphsACM Transactions on Information Systems10.1145/365301542:5(1-23)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3653015
Sun KJiang HHu YYin B(2024)Incorporating Multi-Level Sampling with Adaptive Aggregation for Inductive Knowledge Graph CompletionACM Transactions on Knowledge Discovery from Data10.1145/364482218:5(1-16)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3644822
Han DHuang YLiu JLiao KLin K(2024)LSAB: User Behavioral Pattern Modeling in Sequential Recommendation by Learning Self-Attention BiasACM Transactions on Knowledge Discovery from Data10.1145/363262518:3(1-20)Online publication date: 13-Jan-2024
https://dl.acm.org/doi/10.1145/3632625
Show More Cited By

Index Terms

DACHA: A Dual Graph Convolution Based Temporal Knowledge Graph Representation Learning Method Using Historical Relation
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
      1. Reasoning about belief and knowledge
  2. Machine learning
    1. Machine learning approaches
      1. Learning latent representations

Recommendations

Enhance Temporal Knowledge Graph Completion via Time-Aware Attention Graph Convolutional Network
Machine Learning and Knowledge Discovery in Databases
Abstract
Previous works on knowledge graph representation learning focus on static knowledge graph and get fully developed. However, task on temporal knowledge graph is far from consummation because of its late start. Recent researches have shifted to the ...
Learning Temporal and Spatial Embedding for Temporal Knowledge Graph Reasoning
PRICAI 2022: Trends in Artificial Intelligence
Abstract
Temporal knowledge graphs store a large number of temporal facts that simulate the dynamic interactions of entities along the timeline. Since existing temporal knowledge graphs often suffer from incompleteness, it is crucial to build time-aware ...
TKGAT: Temporal Knowledge Graph Representation Learning Using Attention Network
Advanced Data Mining and Applications
Abstract
Temporal knowledge graph representation learning models can capture more comprehensive semantic information, which has higher practical application value and gradually attracts wide attention. However, the existing temporal knowledge graph ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 16, Issue 3

June 2022

494 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3485152

Editor:
Charu Aggarwal
IBM T. J. Watson Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2021

Accepted: 01 July 2021

Revised: 01 May 2021

Received: 01 December 2020

Published in TKDD Volume 16, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Key Research and Development Program of China
Fundamental Research Funds for the Central Universities

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
1,374
Total Downloads

Downloads (Last 12 months)130
Downloads (Last 6 weeks)12

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tang XChen LShi HLyu D(2024)DHyper: A Recurrent Dual Hypergraph Neural Network for Event Prediction in Temporal Knowledge GraphsACM Transactions on Information Systems10.1145/365301542:5(1-23)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3653015
Sun KJiang HHu YYin B(2024)Incorporating Multi-Level Sampling with Adaptive Aggregation for Inductive Knowledge Graph CompletionACM Transactions on Knowledge Discovery from Data10.1145/364482218:5(1-16)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3644822
Han DHuang YLiu JLiao KLin K(2024)LSAB: User Behavioral Pattern Modeling in Sequential Recommendation by Learning Self-Attention BiasACM Transactions on Knowledge Discovery from Data10.1145/363262518:3(1-20)Online publication date: 13-Jan-2024
https://dl.acm.org/doi/10.1145/3632625
Tang XChen L(2024)GTRL: An Entity Group-Aware Temporal Knowledge Graph Representation Learning MethodIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.333416536:9(4707-4721)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1109/TKDE.2023.3334165
Chen XHusen MZhou Z(2024)CTHTC: A Hybrid Architecture for Temporal Knowledge Graph Completion2024 18th International Conference on Ubiquitous Information Management and Communication (IMCOM)10.1109/IMCOM60618.2024.10418441(1-8)Online publication date: 3-Jan-2024
https://doi.org/10.1109/IMCOM60618.2024.10418441
Zhang FChen HShi YCheng JLin J(2024)Joint framework for tensor decomposition-based temporal knowledge graph completionInformation Sciences: an International Journal10.1016/j.ins.2023.119853654:COnline publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1016/j.ins.2023.119853
Sun KJiang HHu YYin B(2024)Generating Graph-Based Rules for Enhancing Logical ReasoningAdvanced Intelligent Computing Technology and Applications10.1007/978-981-97-5615-5_12(143-156)Online publication date: 3-Aug-2024
https://doi.org/10.1007/978-981-97-5615-5_12
Chen ZFu LXiao SWang SPlant CGuo W(2023)Multi-View Graph Convolutional Networks with Differentiable Node SelectionACM Transactions on Knowledge Discovery from Data10.1145/360895418:1(1-21)Online publication date: 10-Aug-2023
https://dl.acm.org/doi/10.1145/3608954
Li HLv ZLi JXu ZWang YSun HSheng Z(2023)Traffic Flow Forecasting in the COVID-19: A Deep Spatial-temporal Model Based on Discrete Wavelet TransformationACM Transactions on Knowledge Discovery from Data10.1145/356475317:5(1-28)Online publication date: 27-Feb-2023
https://dl.acm.org/doi/10.1145/3564753
Bai LChai DZhu L(2023)RLATKnowledge-Based Systems10.1016/j.knosys.2023.110514269:COnline publication date: 7-Jun-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110514
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents