Abstract
Knowledge graph question answering (Q &A) aims to answer questions through a knowledge base (KB). When using a knowledge base as a data source for multihop Q &A, knowledge graph Q &A needs to obtain relevant entities, their relationships and the correct answer, but often the correct answer cannot be obtained through the reasoning path because of absent relationships. Currently, using pre-trained language models (PLM) and knowledge graphs (KG) has a good effect on complex problems. However, challenging problems remain; the relationships between problems and candidate entities need to be better represented, and joint reasoning must be performed in the relationship graph based on problems and entities. To solve these problems, we expand the relational graph by adding tail entities to the list of preselected entities through reverse relations and then add the processed problems and entities to the problem subgraph. To perform inference on a relational graph, we design an attention-based neural network module. To calculate the loss of the model’s inference process nodes, we use a modified Euclidean distance function as the loss function. To evaluate our model, we conducted experiments on the WebQSP and CWQ datasets, and the model obtained state-of-the-art results in both the KB-full and KB-half settings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Source code will be made available post acceptance.
References
Rajpurkar, P., Zhang, J., Lopyrev, K., et al.: Squad: 100,000+ questions for machine com-prehension of text. arXiv preprint arXiv:1606.05250 (2016)
Zhao, W., Chung, T., Goyal, A., Metallinou, A.: Simple question answering with subgraph ranking and joint-scoring. arXiv preprint arXiv:1904.04049 (2019)
Lan, Y., He, G., Jiang, J., et al.: A survey on complex knowledge base question answering: Methods, challenges and solutions. arXiv preprint arXiv:2105.11644 (2021)
Chen, D., Fisch, A., Weston, J., Bordes, A.: Reading Wikipedia to answer open-domain questions. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1870–1879, Vancouver, Canada. Association for Computational Linguistics (2017)
Jiang, K., Wu, D., Jiang, H.: FreebaseQA: a new factoid QA data set matching trivia-style question-answer pairs with freebase. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 318–323, Minneapolis, Minnesota. Association for Computational Linguistics (2019)
Antoine, B., et al.: Large-scale simple question answering with memory networks. arXiv preprint arXiv:1506.02075 (2015)
Yih, S.W., Chang, M.W., He, X., et al.: Semantic parsing via staged query graph generation: question answering with knowledge base. In: Proceedings of the Joint Conference of the 53rd Annual Meeting of the ACL and the 7th International Joint Conference on Natural Language Processing of the AFNLP (2015)
Devlin, J., Chang, M.W., Lee, K., et al.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Liu, Z., et al.: A robustly optimized BERT pre-training approach with post-training. In: Proceedings of the 20th Chinese National Conference on Computational Linguistics (2021)
Kassner, N., Schütze, H.: Negated and misprimed probes for pre-trained language models: birds can talk, but cannot fly. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7811–7818. Association for Computational Linguistics (2020)
Feng, Y., Chen, X., Lin, B.Y., Wang, P., Yan, J., Ren, X.: Scalable multi-hop relational reasoning for knowledge-aware question answering. arXiv preprint arXiv:2005.00646 (2020)
Yasunaga, M., Ren, H., Bosselut, A., et al.: QA-GNN: Reasoning with language models and knowledge graphs for question answering. arXiv preprint arXiv:2104.06378 (2021)
Mitchell, T., et al.: Never-ending learning. Commun. ACM 61(5), 103–115 (2018)
Vrandečić, D., Krötzsch, M.: Wikidata: a free collaborative knowledge-base. Commun. ACM 57(10), 78–85 (2014)
Färber, M., et al.: Linked data quality of dbpedia, freebase, opencyc, wikidata, and yago. Seman. Web 9(1), 77–129 (2018)
Sukhbaatar, S., Weston, J., Fergus, R.: End-to-end memory networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Martineau, J., Finin, T.: Delta tfidf: an improved feature space for sentiment analysis. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 3. no. 1 (2009)
Yih, W., Richardson, M., Meek, C., et al.: The value of semantic parse labeling for knowledge base question answering. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 201–206 (2016)
Sun, H., Bedrax-Weiss, T., Cohen, W.W.: Pullnet: Open domain question answering with iterative retrieval on knowledge bases and text. arXiv preprint arXiv:1904.09537 (2019)
Lin, B.Y., Chen, X., Chen, J., Ren, X.: KagNet: Knowledge-aware graph networks for commonsense reasoning. arXiv preprint arXiv:1909.02151 (2019)
Berant, J., Chou, A., Frostig, R., Liang, P.: Semantic parsing on freebase from question-answer pairs. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1533–1544 (2013)
Saha, A., Ansari, G.A., Laddha, A., Sankaranarayanan, K., Chakrabarti, S.: Complex program induction for querying knowledge bases in the absence of gold programs. Trans. Assoc. Comput. Linguist. 7, 185–200 (2019)
Zhang, Y., et al.: Variational reasoning for question answering with knowledge graph. In: Thirty-second AAAI Conference on Artificial Intelligence (2018)
Qiu, Y., Wang, Y., Jin, X., et al.: Stepwise reasoning for multi-relation question answering over knowledge graph with weak supervision. In: Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 474–482 (2020)
Zhou, M., Huang, M., Zhu, X.: An interpretable reasoning network for multi-relation question answering. arXiv preprint arXiv:1801.04726 (2018)
Cohen, W.W., Sun, H., Hofer, R.A., Siegler, M.: Scalable neural methods for reasoning with a symbolic knowledge base. arXiv preprint arXiv:2002.06115 (2020)
Lukovnikov, D., Fischer, A., Lehmann, J.: Pretrained transformers for simple question answering over knowledge graphs. In: Ghidini, C., et al. (eds.) ISWC 2019. LNCS, vol. 11778, pp. 470–486. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30793-6_27
Talmor, A., Berant, J.: The web as a knowledge-base for answering complex questions. arXiv preprint arXiv:1803.06643 (2018)
Miller, A., Fisch, A., Dodge, J., et al.: Key-value memory networks for directly reading documents. arXiv preprint arXiv:1606.03126 (2016)
Sun, H., Dhingra, B., Zaheer, M., et al.: Open domain question answering using early fusion of knowledge bases and text. arXiv preprint arXiv:1809.00782 (2018)
Saxena, A., Tripathi, A., Talukdar, P.: Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4498–4507 (2020)
Shi, J., Cao, S., Hou, L., et al.: TransferNet: An effective and transparent frame-work for multi-hop question answering over relation graph. arXiv preprint arXiv:2104.07302 (2021)
Acknowledgements
This work is supported by National Nature Science Foundation of China (No.61762024, No.62062029) and Innovation Project of GUET Graduate Educatio (No.2022YCXS091)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, F., Huang, H., Dong, R. (2023). Efficient Question Answering Based on Language Models and Knowledge Graphs. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14257. Springer, Cham. https://doi.org/10.1007/978-3-031-44216-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-44216-2_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44215-5
Online ISBN: 978-3-031-44216-2
eBook Packages: Computer ScienceComputer Science (R0)