Meta-path reasoning of knowledge graph for commonsense question answering
Authors: 
Miao Zhang, Tingting He, Ming DongAuthors Info & Claims
Abstract
Commonsense question answering (CQA) requires understanding and reasoning over QA context and related commonsense knowledge, such as a structured Knowledge Graph (KG). Existing studies combine language models and graph neural networks to model inference. However, traditional knowledge graph are mostly concept-based, ignoring direct path evidence necessary for accurate reasoning. In this paper, we propose MRGNN (Meta-path Reasoning Graph Neural Network), a novel model that comprehensively captures sequential semantic information from concepts and paths. In MRGNN, meta-paths are introduced as direct inference evidence and an original graph neural network is adopted to aggregate features from both concepts and paths simultaneously. We conduct sufficient experiments on the CommonsenceQA and OpenBookQA datasets, showing the effectiveness of MRGNN. Also, we conduct further ablation experiments and explain the reasoning behavior through the case study.
References
[1]
Lin B Y, Chen X, Chen J, Ren X. KagNet: knowledge-aware graph networks for commonsense reasoning. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 2829–2839
[2]
Feng Y, Chen X, Lin B Y, Wang P, Yan J, Ren X. Scalable multi-hop relational reasoning for knowledge-aware question answering. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 1295–1309
[3]
Yasunaga M, Ren H, Bosselut A, Liang P, Leskovec J. QA-GNN: reasoning with language models and knowledge graphs for question answering. In: Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021, 535–546
[4]
Talmor A, Herzig J, Lourie N, Berant J. CommonsenseQA: a question answering challenge targeting commonsense knowledge. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4149–4158
[5]
Hao S and Zhang H Performance analysis of PHY layer for RIS-assisted wireless communication systems with retransmission protocols Journal of King Saud University-Computer and Information Sciences 2022 34 8 5388-5404
[6]
Devlin J, Chang M W, Lee K, Toutanova K. BERT: pretraining of deep bidirectional transformers for language understanding. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171–4186
[7]
Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R. ALBERT: a lite BERT for self-supervised learning of language representations. In: Proceedings of the 8th International Conference on Learning Representations. 2020
[8]
Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, and Liu P J Exploring the limits of transfer learning with a unified text-to-text transformer The Journal of Machine Learning Research 2020 21 1 140
[9]
Khashabi D, Min S, Khot T, Sabharwal A, Tafjord O, Clark P, Hajishirzi H. UNIFIEDQA: crossing format boundaries with a single QA system. In: Proceedings of Findings of the Association for Computational Linguistics: EMNLP 2020. 2020, 1896–1907
[10]
Speer R, Chin J, Havasi C. ConceptNet 5.5: an open multilingual graph of general knowledge. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017, 4444–4451
[11]
Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. RoBERTa: a robustly optimized BERT pretraining approach. 2016, arXiv preprint arXiv: 1907.11692
[12]
Shen T, Geng X, Qin T, Guo D, Tang D, Duan N, Long G, Jiang D. Multi-task learning for conversational question answering over a large-scale knowledge base. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 2442–2451
[13]
Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. 2017, arXiv preprint arXiv: 1609.02907
[14]
Schlichtkrull M, Kipf T N, Bloem P, Van Den Berg R, Titov I, Welling M. Modeling relational data with graph convolutional networks. In: Proceedings of the 15th International Conference on the Semantic Web. 2018, 593–607
[15]
Veličković P, Casanova A, Liò P, Cucurull G, Romero A, Bengio Y. Graph attention networks. In: Proceedings of the 6th International Conference on Learning Representations. 2018
[16]
Bordes A, Usunier N, Garcia-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 2787–2795
[17]
Sun Y and Han J Mining Heterogeneous Information Networks: Principles and Methodologies 2012 San Rafael Morgan & Claypool Publishers
[18]
Santoro A, Raposo D, Barrett D G T, Malinowski M, Pascanu R, Battaglia P W, Lillicrap T. A simple neural network module for relational reasoning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 4974–4983
[19]
Heilman M, Smith N A. Good question! statistical ranking for question generation. In: Proceedings of Human Language Technologies: Conference of the North American Chapter of the Association for Computational Linguistics. 2010, 609–617
[20]
Hochreiter S and Schmidhuber J Long short-term memory Neural Computation 1997 9 8 1735-1780
[21]
Mihaylov T, Clark P, Khot T, Sabharwal A. Can a suit of armor conduct electricity? A new dataset for open book question answering. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 2381–2391
[22]
Mihaylov T, Frank A. Knowledgeable reader: enhancing cloze-style reading comprehension with external commonsense knowledge. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 821–832
[23]
Monea A M, Marginean A N. Medical question entailment based on textual inference and fine-tuned BioMed-RoBERTa. In: Proceedings of the 17th IEEE International Conference on Intelligent Computer Communication and Processing. 2021, 319–326
[24]
Wang X, Kapanipathi P, Musa R, Yu M, Talamadupula K, Abdelaziz I, Chang M, Fokoue A, Makni B, Mattei N, Witbrock M. Improving natural language inference using external knowledge in the science questions domain. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence and the 31st Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. 2019, 7208–7215
[25]
Zhan X, Huang Y, Dong X, Cao Q, and Liang X PathReasoner: explainable reasoning paths for commonsense question answering Knowledge-Based Systems 2022 235 107612
[26]
Yan J, Raman M, Chan A, Zhang T, Rossi R, Zhao H, Kim S, Lipka N, Ren X. Learning contextualized knowledge structures for commonsense reasoning. In: Proceedings of the Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021, 4038–4051
[27]
Wang J, Li X, Tan Z, Zhao X, Xiao W. Relation-aware bidirectional path reasoning for commonsense question answering. In: Proceedings of the 25th Conference on Computational Natural Language Learning. 2021, 445–453
[28]
Saxena A, Tripathi A, Talukdar P P. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 4498–4507
[29]
Bao J, Duan N, Yan Z, Zhou M, Zhao T. Constraint-based question answering with knowledge graph. In: Proceedings of the COLING 2016, 26th International Conference on Computational Linguistics: Technical Papers. 2016, 2503–2514
[30]
Das R, Dhuliawala S, Zaheer M, Vilnis L, Durugkar I, Krishnamurthy A, Smola A, McCallum A. Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. In: Proceedings of the 6th International Conference on Learning Representations. 2018
[31]
Chen D, Yang M, Zheng H T, Li Y, Shen Y. Answer-enhanced path-aware relation detection over knowledge base. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019, 1021–1024
[32]
Ji S, Pan S, Cambria E, Marttinen P, and Yu P S A survey on knowledge graphs: representation, acquisition, and applications IEEE Transactions on Neural Networks and Learning Systems 2022 33 2 494-514
[33]
Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J. Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of 2008 ACM SIGMOD International Conference on Management of Data. 2008, 1247–1250
[34]
Kundu S, Khot T, Sabharwal A, Clark P. Exploiting explicit paths for multi-hop reading comprehension. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 2737–2747
[35]
Chen W, Xiong W, Yan X, Wang W Y. Variational knowledge graph reasoning. In: Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018, 1823–1832
[36]
Yih W T, Chang M W, He X, Gao J. Semantic parsing via staged query graph generation: question answering with knowledge base. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 1321–1331
[37]
Lan Y, Jiang J. Query graph generation for answering multi-hop complex questions from knowledge bases. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 969–974
Recommendations
Overview of knowledge reasoning for knowledge graph
AbstractKnowledge graphs are large-scale semantic networks that considerably impact knowledge representation. Mining hidden knowledge from existing data, including triplet knowledge reasoning, is a primary objective of knowledge graphs. With the ...
Text-Enhanced Question Answering over Knowledge Graph
IJCKG '21: Proceedings of the 10th International Joint Conference on Knowledge GraphsQuestion answering over knowledge graph is an important area of research within question answering. Existing methods mainly focus on the utilization of information in knowledge graphs and ignore the abundant external information of entities. However, ...
Query and Neighbor-Aware Reasoning Based Multi-hop Question Answering over Knowledge Graph
Knowledge Science, Engineering and ManagementAbstractMulti-hop Question Answering over Knowledge Graph (multi-hop KGQA) is a challenging task since it requires reasoning with multiple triplets over knowledge graph to find the correct answer entities. Benefiting from expeditious development of ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© Higher Education Press 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 12 August 2023
Accepted: 16 November 2022
Received: 03 June 2022
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 30 Nov 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in