Abstract
Machine reading comprehension (MRC) is a challenging NLP task that requires machines to model the complex interactions between questions and specific contexts. In Question-Answering (QA) tasks, most existing works rely on the powerful encoder of pre-trained language models (PrLM) in order to represent word/subword embeddings for extracting the answer. In this study, we present a novel method for enriching the context representation by exploiting the question-context interaction at the sentence level. In particular, we introduce the sentence-based question-context interaction (S-QCI) block, which combines two main layers such as the question-aware layer and the cross-sentence layer, to represent the sentence embedding of the context. The sentence information is then used to enrich question information for the context representation at the word level. The main idea is that the word units in the sentence, which have a high attention score of question-sentence interaction, can be enriched with more question information for the final output of the extractive-span MRC task. The experiment on NewsQA, a benchmark dataset in this research field, indicates that the proposed method has significant improvements compared with the baselines using PrLM and achieves new state-of-the-art results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Back, S., Chinthakindi, S.C., Kedia, A., Lee, H., Choo, J.: NeurQuRI: neural question requirement inspector for answerability prediction in machine reading comprehension. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020. OpenReview.net (2020). https://openreview.net/forum?id=ryxgsCVYPr
Chen, D., Fisch, A., Weston, J., Bordes, A.: Reading Wikipedia to answer open-domain questions. In: Barzilay, R., Kan, M. (eds.) Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers. pp. 1870–1879. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/P17-1171
Clark, K., Luong, M., Le, Q.V., Manning, C.D.: ELECTRA: pre-training text encoders as discriminators rather than generators. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020. OpenReview.net (2020). https://openreview.net/forum?id=r1xMH1BtvB
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/n19-1423
Hu, M., Wei, F., Peng, Y., Huang, Z., Yang, N., Li, D.: Read + verify: machine reading comprehension with unanswerable questions. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pp. 6529–6537. AAAI Press (2019). https://doi.org/10.1609/aaai.v33i01.33016529
Huang, Z., et al.: Recent trends in deep learning based open-domain textual question answering systems. IEEE Access 8, 94341–94356 (2020). https://doi.org/10.1109/ACCESS.2020.2988903
Kundu, S., Ng, H.T.: A question-focused multi-factor attention network for question answering. In: McIlraith, S.A., Weinberger, K.Q. (eds.) Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI 2018), the 30th innovative Applications of Artificial Intelligence (IAAI 2018), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI 2018), New Orleans, Louisiana, USA, February 2–7, 2018, pp. 5828–5835. AAAI Press (2018). https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17226
Liu, X., Shen, Y., Duh, K., Gao, J.: Stochastic answer networks for machine reading comprehension. In: Gurevych, I., Miyao, Y. (eds.) Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15–20, 2018, Volume 1: Long Papers, pp. 1694–1704. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/P18-1157, https://aclanthology.org/P18-1157/
Liu, Y., et al.: Roberta: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019). http://arxiv.org/abs/1907.11692
Phan, T., Nguyen, N.N., Bui, K.N.: HeterGraphLongSum: heterogeneous graph neural network with passage aggregation for extractive long document summarization. In: Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12–17, 2022, pp. 6248–6258. International Committee on Computational Linguistics (2022). https://aclanthology.org/2022.coling-1.545
Tay, Y., Luu, A.T., Hui, S.C., Su, J.: Densely connected attention propagation for reading comprehension. In: Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018(December), pp. 3–8, 2018. Montréal, Canada, pp. 4911–4922 (2018). https://proceedings.neurips.cc/paper/2018/hash/7b66b4fd401a271a1c7224027ce111bc-Abstract.html
Trischler, A., et al.: NewsQA: a machine comprehension dataset. In: Proceedings of the 2nd Workshop on Representation Learning for NLP, pp. 191–200. Association for Computational Linguistics, Vancouver, Canada, August 2017. https://doi.org/10.18653/v1/W17-2623, https://aclanthology.org/W17-2623
Vaswani, A., et al.: Attention is all you need. In: Guyon, I., von Luxburg, U., Bengio, S., Wallach, H.M., Fergus, R., Vishwanathan, S.V.N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017(December), pp. 4–9, 2017. Long Beach, CA, USA, pp. 5998–6008 (2017). https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
Wang, W., Yang, N., Wei, F., Chang, B., Zhou, M.: Gated self-matching networks for reading comprehension and question answering. In: Barzilay, R., Kan, M. (eds.) Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30–August 4, Volume 1: Long Papers, pp. 189–198. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/P17-1018, https://doi.org/10.18653/v1/P17-1018
Yang, Z., Dai, Z., Yang, Y., Carbonell, J.G., Salakhutdinov, R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding. In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019(December), pp. 8–14, 2019. Vancouver, BC, Canada, pp. 5754–5764 (2019). https://proceedings.neurips.cc/paper/2019/hash/dc6a7e655d7e5840e66733e9ee67cc69-Abstract.html
Zeng, C., Li, S., Li, Q., Hu, J., Hu, J.: A survey on machine reading comprehension-tasks, evaluation metrics and benchmark datasets. Appl. Sci. 10(21), 7640 (2020). https://doi.org/10.3390/app10217640, https://www.mdpi.com/2076-3417/10/21/7640
Zhang, Z., Yang, J., Zhao, H.: Retrospective reader for machine reading comprehension. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2–9, 2021, pp. 14506–14514. AAAI Press (2021). https://ojs.aaai.org/index.php/AAAI/article/view/17705
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Phan, TA., Ngo, H., Bui, KH.N. (2023). A Novel Question-Context Interaction Method for Machine Reading Comprehension. In: Nguyen, N.T., et al. Intelligent Information and Database Systems. ACIIDS 2023. Lecture Notes in Computer Science(), vol 13995. Springer, Singapore. https://doi.org/10.1007/978-981-99-5834-4_5
Download citation
DOI: https://doi.org/10.1007/978-981-99-5834-4_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-5833-7
Online ISBN: 978-981-99-5834-4
eBook Packages: Computer ScienceComputer Science (R0)