Abstract
Community question answering (CQA) becomes more and more popular in both academy and industry recently. However, a large number of answers often amass in question-answering communities. Hence, it is almost impossible for users to view item by item and select the most relevant one. As a result, answer selection becomes a very significant subtask of CQA. Hence, we propose question-answer dual attention fusion networks with the pre-trained model (BRETDAN) for the task of answer selection. Specifically, we apply BERT model, which has achieved a better result in GLUE leaderboard with deep transformer architectures as the encoder layer to do fine-tuning for question subjects, question bodies and answers, respectively, then the cross attention mechanism selecting out the most relevant answer for different questions. Finally, we apply dual attention fusion networks to filter the noise caused by introducing question and answer pairs. Specifically, the cross attention mechanism aims to extract interactive information between question subject and answer. In a similar way, the interactive information between question body and answer is also captured. Dual attention fusion aims to address the noise problem in the question and answer pairs. Experiments show that the BERTDAN model achieves significant performance on two datasets: SemEval-2015 and SemEval-2017, outperforming all baseline models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Roth, D.: Learning to resolve natural language ambiguities: a unified approach. In: AAAI/IAAI 1998, pp. 806–813 (1998)
Metzler, D., Croft, W.B.: Analysis of statistical question classification for fact-based questions. Inf. Retr. 8(3), 481–504 (2005)
Barrón-Cedeno, A., et al.: Thread-level information for comment classification in community question answering. In: ACL, pp. 687–693, Beijing, China (2015)
Joty, S., Màrquez, L., Nakov, P.: Joint learning with global inference for comment classification in community question answering. In: ACL, pp. 703–713, San Diego, California (2016)
Yang, M., et al.: Knowledge-enhanced hierarchical attention for community question answering with multi-task and adaptive learning, pp. 5349–5355. In: IJCAI (2019)
Deng, Y., et al.: Joint learning of answer selection and answer summary generation in community question answering, pp. 7651–7658. In: AAAI (2020)
Xie, Y., Shen, Y., et al.: Attentive user-engaged adversarial neural network for community question answering. In: AAAI, vol. 34, pp. 9322–9329 (2020)
Garg, S., Thuy, V., Moschitti, A.: Tanda: transfer and adapt pre-trained transformer models for answer sentence selection. In: AAAI, vol. 34, pp. 7780–7788 (2020)
Yang, M., Wenting, T., Qiang, Q., et al.: Advanced community question answering by leveraging external knowledge and multi-task learning. Knowl.-Based Syst. 171, 106–119 (2019)
Yang, H., et al.: AMQAN: adaptive multi-attention question-answer networks for answer selection. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds.) ECML PKDD 2020. LNCS (LNAI), vol. 12459, pp. 584–599. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67664-3_35
Wan, S., Lan, Y., Guo, J., et al.: A deep architecture for semantic matching with multiple positional sentence representations. In: AAAI, pp. 2835–2841 (2016)
Zhang, X., Li, S., Sha, L., Wang, H.: Attentive interactive neural networks for answer selection in community question answering. In: AAAI, vol. 31 (2017)
Devlin, J., Chang, M.W., et al.: Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL (2019)
Yu, A.W., et al.: Fast and accurate reading comprehension by combining self-attention and convolution. In: ICLR (2018)
Lin, Z., et al.: A structured self-attentive sentence embedding. In: ICLR (2017)
Chen, Q., et al.: Enhanced lstm for natural language inference[c]. In: ACL, pp. 1657–1668 (2017)
Mou, L., et al.: Natural language inference by tree-based convolution and heuristic matching[c]. In: ACL, pp. 130–136 (2016)
Ba, J., Kingma, D.P.: Adam: a method for stochastic optimization. In: ICLR (2015)
Tran, Q.H., Tran, D.V., Vu, T., Le Nguyen, M., Pham, S.B.: Jaist: combining multiple features for answer selection in community question answering. In: SemEval-2015, pp. 215–219, Denver, Colorado (2015)
Wu, W., Wang, H., Li, S.: Bi-directional gated memory networks for answer selection. In: Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, pp. 251–262 (2017)
Wu, G., Sheng, Y., Lan, M., Wu, Y.: Ecnu at semeval2017 task 3: using traditional and deep learning methods to address community question answering task. In: SemEval-2017, pp. 365–369 (2017)
Xiang, Y., Zhou, X., et al.: Incorporating label dependency for answer quality tagging in community question answering via cnn-lstm-crf. In: COLING, pp. 1231–1241, Osaka, Japan (2016)
Wu, W., Sun, X., Wang, H., et al.: Question condensing networks for answer selection in community question answering. In: ACL, pp. 1746–1755 (2018)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, H. et al. (2021). BERTDAN: Question-Answer Dual Attention Fusion Networks with Pre-trained Models for Answer Selection. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13110. Springer, Cham. https://doi.org/10.1007/978-3-030-92238-2_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-92238-2_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92237-5
Online ISBN: 978-3-030-92238-2
eBook Packages: Computer ScienceComputer Science (R0)