Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Multi-granularity Hierarchical Feature Extraction for Question-Answering Understanding

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Question-answering understanding systems are of central importance to many natural language processing tasks. A successful question-answering system first needs to accurately mine the semantics of the problem text and then match the semantic similarity between the question and the answer. Most of the current pre-training language modes use joint coding of questions and answers, a pre-training language model to avoid the problem of feature extraction from multilevel text structure, it through unified advance training ignores text semantic expression in different particle size, different levels of semantic features, and to some extent avoiding the serious problem of semantic understanding. In this paper, we focus on the problem of multi-granularity and multi-level feature expression of text semantics in question and answer understanding, and design a question-answering understanding method for multi-granularity hierarchical features. First, we extract features from two aspects, the traditional language model and the deep matching model, and then fuse these features to construct the similarity matrix, and learn the similarity matrix by designing three different models. Finally, the similarity matrix is learned by three different models, and after sorting, the overall similarity is obtained from the similarity of multiple granularity features. Evaluated by testing on WikiQA public datasets, experiments show that the results of our method are improved by adding the multi-granularity hierarchical feature learning method compared with traditional deep learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The experiment data used to support the findings of this study have been deposited in the GITHUB repository https://github.com/mrlijun2017.

Notes

  1. https://trec.nist.gov/data/qamain.html

References

  1. Cambazoglu BB, Sanderson M, Scholer F, Croft B. A review of public datasets in question answering research. In: ACM SIGIR Forum, vol. 54. ACM New York, NY, USA; 2021. p. 1–23.

  2. Acheampong KN, Tian W. Advancement of textual answer triggering: cognitive boosting. IEEE Trans Emerg Top Comput. 2022;10(1):361–72.

    Article  Google Scholar 

  3. Etezadi R, Shamsfard M. The state of the art in open domain complex question answering: a survey. Appl Intell. 2022;1–21.

  4. Hao T, Li X, He Y, Wang FL, Qu Y. Recent progress in leveraging deep learning methods for question answering. Neural Comput Applic. 2022;1–19.

  5. Luo Y, Yang B, Xu D, Tian L. A survey: complex knowledge base question answering. In: 2022 IEEE 2nd International Conference on Information Communication and Software Engineering (ICICSE). IEEE; 2022. p. 46–52.

  6. Singh H, Nasery A, Mehta D, Agarwal A, Lamba J, Srinivasan BV. MIMOQA: multimodal input multimodal output question answering. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p. 5317–32, Online, June 2021. Association for Computational Linguistics.

  7. Risch J, Möller T, Gutsch J, Pietsch M. Semantic answer similarity for evaluating question answering models. In: Proceedings of the 3rd Workshop on Machine Reading for Question Answering, Punta Cana, Dominican Republic, November 2021. p. 149–57. Association for Computational Linguistics.

  8. Chakraborty N, Lukovnikov D, Maheshwari G, Trivedi P, Lehmann J, Fischer A. Introduction to neural network-based question answering over knowledge graphs. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 2021;11(3):e1389.

  9. Lan Y, He G, Jiang J, Jiang J, Zhao WX, Wen J-R. A survey on complex knowledge base question answering: methods, challenges and solutions. In: Zhou Z-H, editor. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 4483–91. International Joint Conferences on Artificial Intelligence Organization, 8 2021. Survey Track.

  10. Jia Z, Pramanik S, Roy RS, Weikum G. Complex temporal question answering on knowledge graphs. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021. p. 792–802.

  11. Ren H, Dai H, Dai B, Chen X, Yasunaga M, Sun H, Schuurmans D, Leskovec J, Zhou D. LEGO: latent execution-guided reasoning for multi-hop question answering on knowledge graphs. In: Meila M, Zhang T, editors. Proceedings of the 38th International Conference on Machine Learning, vol. 139 of Proceedings of Machine Learning Research. p. 8959–70. PMLR, 18–24 Jul 2021.

  12. Li L, Zhang M, Chao Z, Xiang J. Using context information to enhance simple question answering. World Wide Web. 2021;24(1):249–77.

    Article  Google Scholar 

  13. Chen H, Ye F, Fan Y, He Z, Jing Y, Zhang K, Wang XS. Staged query graph generation based on answer type for question answering over knowledge base. Knowl-Based Syst. 2022;253:109576.

  14. Zhang Q, Weng X, Zhou G, Zhang Y, Huang JX. ARL: an adaptive reinforcement learning framework for complex question answering over knowledge base. Inf Process Manag. 2022;59(3):102933.

  15. Song L, Li J, Liu J, Yang Y, Shang X, Sun M. Answering knowledge-based visual questions via the exploration of question purpose. Pattern Recogn. 2023;133.

  16. Singh D, Reddy S, Hamilton W, Dyer C, Yogatama D. End-to-end training of multi-document reader and retriever for open-domain question answering. In: Ranzato M, Beygelzimer A, Dauphin Y, Liang PS, Vaughan JW, editors. Advances in Neural Information Processing Systems, vol. 34. Curran Associates, Inc.; 2021. p. 25968–81.

  17. Yamada I, Asai A, Hajishirzi H. Efficient passage retrieval with hashing for open-domain question answering. arXiv:2106.00882 [Preprint]. 2021. Available from: http://arxiv.org/abs/2106.00882.

  18. Das R, Godbole A, Naik A, Tower E, Zaheer M, Hajishirzi H, Jia R, Mccallum A. Knowledge base question answering by case-based reasoning over subgraphs. In: Chaudhuri K, Jegelka S, Song L, Szepesvari C, Niu G, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, vol. 162 of Proceedings of Machine Learning Research. PMLR, 17–23 Jul 2022. p. 4777–93.

  19. Hsu H-H, Huang N-F. Xiao-Shih: a self-enriched question answering bot with machine learning on Chinese-based MOOCs. IEEE Trans Learn Technol. 2022;15(2):223–37.

    Article  Google Scholar 

  20. Nakano Y, Kawano S, Yoshino K, Sudoh K, Nakamura S. Pseudo ambiguous and clarifying questions based on sentence structures toward clarifying question answering system. In Proceedings of the Second DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering, Dublin, Ireland, May 2022. p. 31–40. Association for Computational Linguistics.

  21. Gomes J, de Mello RC, Ströele V, de Souza JF. A hereditary attentive template-based approach for complex knowledge base question answering systems. Expert Syst Appl. 2022;205:117725.

  22. Sovrano F, Vitali F. Explanatory artificial intelligence (YAI): human-centered explanations of explainable AI and complex data. Data Min Knowl Disc. 2022;1–28.

  23. Sachan DS, Patwary M, Shoeybi M, Kant N, Ping W, Hamilton WL, Catanzaro B. End-to-end training of neural retrievers for open-domain question answering. arXiv:2101.00408 [Preprint]. 2021. Available from: http://arxiv.org/abs/2101.00408.

  24. Alkhaldi T, Chu C, Kurohashi S. Flexibly focusing on supporting facts, using bridge links, and jointly training specialized modules for multi-hop question answering. IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2021;29:3216–25.

    Article  Google Scholar 

  25. Wu W, Zhu Z, Qi J, Wang W, Zhang G, Liu P. A dynamic graph expansion network for multi-hop knowledge base question answering. Neurocomputing. 2022.

  26. Hu J, Qian S, Fang Q, Xu C. Heterogeneous community question answering via social-aware multi-modal co-attention convolutional matching. IEEE Trans Multimedia. 2021;23:2321–34.

    Article  Google Scholar 

  27. Qiu C, Zhou G, Cai Z, Søgaard A. A global-local attentive relation detection model for knowledge-based question answering. IEEE Transactions on Artificial Intelligence. 2021;2(2):200–12.

  28. Aithal SG, Rao AB, Singh S. Automatic question-answer pairs generation and question similarity mechanism in question answering system. Appl Intell. 2021;51(11):8484–97.

  29. Etemadi R, Zihayat M, Feng K, Adelman J, Bagheri E. Embedding-based team formation for community question answering. Inf Sci. 2022.

  30. Li X, Cheng Y. Understanding the message passing in graph neural networks via power iteration clustering. Neural Netw. 2021;140:130–5.

    Article  Google Scholar 

  31. Figueroa A, Gómez-Pantoja C, Neumann G. Integrating heterogeneous sources for predicting question temporal anchors across Yahoo! Answers. Information Fusion. 2019;50:112–25.

  32. Wang Q, Wenjun W, Qi Y, Zhao Y. Deep Bayesian active learning for learning to rank: a case study in answer selection. IEEE Trans Knowl Data Eng. 2022;34(11):5251–62.

    Article  Google Scholar 

  33. Yoo D, Kweon IS. Learning loss for active learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019. p. 93–102.

  34. Feng WZ, Tang J. A ranking model for answer selection with deep matching features. Journal of Chinese Information Processing. 2019;33(1):118–24.

    Google Scholar 

  35. Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016. p. 785–94.

  36. Yang Y, Yih W-T, Meek C. WIKIQA: a challenge dataset for open-domain question answering. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015. p. 2013–18.

  37. Milne D, Witten IH. Learning to link with Wikipedia. In: Proceedings of the 17th ACM Conference on Information and Knowledge Management. 2008. p. 509–18.

  38. Feng M, Xiang B, Glass MR, Wang L, Zhou B. Applying deep learning to answer selection: a study and an open task. In: 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU). IEEE; 2015. p. 813–20.

  39. Tan M, dos Santos C, Xiang B, Zhou B. LSTM-based deep learning models for non-factoid answer selection. arXiv:1511.04108 [Preprint]. 2015. Available from: http://arxiv.org/abs/1511.04108.

  40. Fang Z, Liu J, Li Y, Qiao Y, Hanqing L. Improving visual question answering using dropout and enhanced question encoder. Pattern Recogn. 2019;90:404–14.

    Article  Google Scholar 

  41. dos Santos C, Tan M, Xiang B, Zhou B. Attentive pooling networks. arXiv:1602.03609 [Preprint]. 2016. Available from: http://arxiv.org/abs/1602.03609.

  42. Tay Y, Luu AT, Hui SC. Enabling efficient question answer retrieval via hyperbolic neural networks. CoRR abs/1707.07847. 2017.

  43. Wang Z, Hamza W, Florian R. Bilateral multi-perspective matching for natural language sentences. arXiv:1702.03814 [Preprint]. 2017. Available from: http://arxiv.org/abs/1702.03814.

  44. Bian W, Li S, Yang Z, Chen G, Lin Z. A compare-aggregate model with dynamic-clip attention for answer selection. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 2017. p. 1987–90.

  45. Shen G, Yang Y, Deng Z-H. Inter-weighted alignment network for sentence pair modeling. In: Proceedings of the 2017 Conference On Empirical Methods in Natural Language Processing. 2017. p. 1179–89.

  46. Tran QH, Lai T, Haffari G, Zukerman I, Bui T, Bui H. The context-dependent additive recurrent neural net. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long Papers). 2018. p. 1274–83.

Download references

Funding

This work was supported by the Guangxi Natural Science Foundation (No. 2022GXNSFBA035510),the Open Funds from Guilin University of Electronic Technology, Guangxi Key Laboratory of Image and Graphic Intelligent Processing (No. GIIP2207), the National Natural Science Foundation of China (No. 62267002, No. 62066009), the Foundation of Doctoral Research Initiation Project (No. UF20034Y), the Middle-aged and Young Teachers’ Basic Ability Promotion Project of Guangxi (No. 2021KY0222), and the Postdoctoral Science Foundation of Guangxi Province of China (No. C21RSC90SX03).

Author information

Authors and Affiliations

  1. School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin, 541004, Guangxi, China

    Xingguo Qin, Ya Zhou, Guimin Huang, Maolin Li & Jun Li

  2. Guangxi Key Laboratory of Image and Graphic Intelligence Processing, Guilin University of Electronic Technology, Guilin, 541004, Guangxi, China

    Guimin Huang & Jun Li

Authors
  1. Xingguo Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ya Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guimin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maolin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Li.

Ethics declarations

Ethics Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed Consent

Informed consent was not required as no humans or animals were involved.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qin, X., Zhou, Y., Huang, G. et al. Multi-granularity Hierarchical Feature Extraction for Question-Answering Understanding. Cogn Comput 15, 121–131 (2023). https://doi.org/10.1007/s12559-022-10102-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-022-10102-7

Keywords

Search

Navigation