Abstract
Multi-hop Knowledge Graph Question Answering (KGQA) aims to find the answer entity via a reasoning path consisting of multiple fact triples in the knowledge graph (KG). Most of end-to-end KGQA approaches only pay attention to answering one-hop simple questions and lack scalability and interpretability. Meanwhile, since the high cost for data annotations, the lack of intermediate supervision signals becomes a major challenge. To address these challenges, we propose a policy-based reinforcement learning model called RPGQA which converts the task of KGQA to a reasoning path generation task in the KG. Firstly, in order to improve the interpretability of the model, the agent in our model learns an effective policy to reason a path to the answer entity as the evidence for the question. Secondly, we design an algorithm for entity disambiguation during entity linking. After that, the topic entity in the question can be linked as the beginning of the reasoning path. Furthermore, we propose a reward shaping policy consisting of three parts to enhance intermediate supervision signals, which alleviates the problem of reward delay and sparsity of reward. Extensive experiments on multiple benchmark datasets have demonstrated the effectiveness of our model. RPGQA outperforms most of the state-of-art baselines on the multi-hop KGQA task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bast, H., Haussmann, E.: More accurate question answering on freebase. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 1431–1440 (2015)
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250 (2008)
Das, R., et al.: Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. In: 6th International Conference on Learning Representations (2017)
Dong, L., Wei, F., Zhou, M., Xu, K.: Question answering over freebase with multi-column convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 260–269 (2015)
Gao, H., et al.: CSIP: enhanced link prediction with context of social influence propagation. Big Data Res. 24, 100217 (2021)
Guo, S., et al.: Knowledge graph embedding preserving soft logical regularity. In: Proceedings of the 29th ACM International Conference on Information and Knowledge Management, pp. 425–434 (2020)
Li, X., Hu, S., Zou, L.: Knowledge based natural answer generation via masked-graph transformer. World Wide Web 25(3), 1403–1423 (2022)
Liang, C., Berant, J., Le, Q., Forbus, K.D., Lao, N.: Neural symbolic machines: learning semantic parsers on freebase with weak supervision. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 23–33. Association for Computational Linguistics (2017)
Lin, X.V., Socher, R., Xiong, C.: Multi-hop knowledge graph reasoning with reward shaping. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October– 4 November, 2018, pp. 3243–3253 (2018)
Miller, A., Fisch, A., Dodge, J., Karimi, A.H., Bordes, A., Weston, J.: Key-value memory networks for directly reading documents. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1400–1409 (2016)
Qiu, Y., Wang, Y., Jin, X., Zhang, K.: Stepwise reasoning for multi-relation question answering over knowledge graph with weak supervision. In: Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 474–482 (2020)
Qiu, Y., et al.: Hierarchical query graph generation for complex question answering over knowledge graph. In: Proceedings of the 29th ACM International Conference on Information and Knowledge Management, pp. 1285–1294 (2020)
Ren, H., Hu, W., Leskovec, J.: Query2box: reasoning over knowledge graphs in vector space using box embeddings. In: 8th International Conference on Learning Representations. OpenReview.net (2020)
Ren, H., Leskovec, J.: Beta embeddings for multi-hop logical reasoning in knowledge graphs. In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 (2020)
Saxena, A., Tripathi, A., Talukdar, P.: Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4498–4507 (2020)
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38
Shen, Y., Chen, J., Huang, P.S., Guo, Y., Gao, J.: M-walk: learning to walk over graphs using Monte Carlo tree search. In: Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, pp. 6787–6798 (2018)
Sun, H., Bedrax-Weiss, T., Cohen, W.W.: PullNet: open domain question answering with iterative retrieval on knowledge bases and text. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 2380–2390 (2019)
Sun, H., Dhingra, B., Zaheer, M., Mazaitis, K., Salakhutdinov, R., Cohen, W.W.: Open domain question answering using early fusion of knowledge bases and text. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4231–4242 (2018)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Wang, M., Wang, H., Li, B., Zhao, X., Wang, X.: Survey of key technologies of new generation knowledge graph. J. Comput. Res. Dev., 1–18 (2022). (Chinese)
Wu, J., Li, B., Ji, Y., Tian, J., Xiang, Y.: Text-enhanced knowledge graph representation model in hyperbolic space. In: Li, B., et al. (eds.) ADMA 2022. LNCS (LNAI), vol. 13088, pp. 137–149. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-95408-6_11
Xiong, W., Hoang, T., Wang, W.Y.: DeepPath: a reinforcement learning method for knowledge graph reasoning. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 564–573 (2017)
Xu, K., Reddy, S., Feng, Y., Huang, S., Zhao, D.: Question answering on freebase via relation extraction and textual evidence. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (2016)
Yih, W.t., Chang, M.W., He, X., Gao, J.: Semantic parsing via staged query graph generation: question answering with knowledge base. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, pp. 1321–1331 (2015)
Yin, W., Yu, M., Xiang, B., Zhou, B., Schütze, H.: Simple question answering by attentive convolutional neural network. In: COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, pp. 1746–1756 (2016)
Yu, H., Li, H., Mao, D., Cai, Q.: A relationship extraction method for domain knowledge graph construction. World Wide Web 23(2), 735–753 (2020). https://doi.org/10.1007/s11280-019-00765-y
Yu, M., Yin, W., Hasan, K.S., Santos, C.d., Xiang, B., Zhou, B.: Improved neural relation detection for knowledge base question answering. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 571–581 (2017)
Zhang, Y., et al.: Fine-grained evaluation of knowledge graph embedding model in knowledge enhancement downstream tasks. Big Data Res. 25, 100218 (2021)
Zhang, Y., Dai, H., Kozareva, Z., Smola, A.J., Song, L.: Variational reasoning for question answering with knowledge graph. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), The 30th Innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI 2018), pp. 6069–6076 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xiang, Y. et al. (2023). Reasoning Path Generation for Answering Multi-hop Questions Over Knowledge Graph. In: Li, B., Yue, L., Tao, C., Han, X., Calvanese, D., Amagasa, T. (eds) Web and Big Data. APWeb-WAIM 2022. Lecture Notes in Computer Science, vol 13422. Springer, Cham. https://doi.org/10.1007/978-3-031-25198-6_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-25198-6_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25197-9
Online ISBN: 978-3-031-25198-6
eBook Packages: Computer ScienceComputer Science (R0)