Abstract
Text matching based on deep learning models often suffer from the limitation of query term coverage problems. Inspired by the success of attention based models in machine translation, which the models can automatically search for parts of a sentence that are relevant to a target word, we propose a multi-level attention model with maximum matching matrix rank to simulate what human does when finding a good answer for a query question. Firstly, we apply a multi-attention mechanism to choose the high effect document words for every query words. Then an approach we called reciprocal relative standard deviation (RRSD) will calculate the matching coverage score for all query words. Experiments on both question-answer task and learning to rank task have achieved state-of-the-art results compared to traditional statistical methods and deep neural network methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yu, L., Hermann, K.M., Blunsom, P., et al.: Deep learning for answer sentence selection. Comput. Sci. (2014)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Comput. Sci. (2014)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks, 4, 3104–3112 (2014)
Guo J, Fan Y, Ai Q, et al.: A deep relevance matching model for Ad-hoc retrieval. In: ACM International on Conference on Information and Knowledge Management, pp. 55–64. ACM (2016)
Reed, G.F., Lynn, F., Meade, B.D.: Use of coefficient of variation in assessing variability of quantitative assays. Clin. Diagn. Lab. Immunol. 9(6), 1235–1239 (2002)
Pang, L., Lan, Y., Guo, J., et al.: Text matching as image recognition (2016)
Liu, T.Y.: Learning to rank for information retrieval. Acm Sigir. Forum 41(2), 904 (2010)
Yang, Y., Yih, W.T., Meek, C.: WikiQA: a challenge dataset for open-domain question answering. In: Conference on Empirical Methods in Natural Language Processing, pp. 2013–2018 (2015)
Qin, T., Liu, T.Y.: Introducing LETOR 4.0 datasets. Comput. Sci. (2013)
Zhai, C., Lafferty, J.: A study of smoothing methods for language models applied to Ad Hoc information retrieval. In: International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 334–342. ACM (2001)
Huang, P.S., He, X., Gao, J., et al.: Learning deep structured semantic models for web search using clickthrough data. In: ACM International Conference on Conference on Information & Knowledge Management, pp. 2333–2338. ACM (2013)
Shen, Y., He, X., Gao, J., et al.: Learning semantic representations using convolutional neural networks for web search. In: International Conference on World Wide Web, pp. 373–374. ACM (2014)
Hu, B., Lu, Z., Li, H., et al.: Convolutional neural network architectures for matching natural language sentences. In: International Conference on Neural Information Processing Systems. MIT Press, pp. 2042–2050 (2014)
Xiong, C., Dai, Z., Callan, J., et al.: End-to-end neural Ad-hoc ranking with Kernel pooling, pp. 55–64 (2017)
Wan, S., Lan, Y., Guo, J., et al.: A deep architecture for semantic matching with multiple positional sentence representations, pp. 2835–2841 (2015)
Yang, L., Ai, Q., Guo, J., et al.: aNMM: ranking short answer texts with attention-based neural matching model. In: ACM International on Conference on Information and Knowledge Management, pp. 287–296. ACM (2016)
Pang, L., Lan, Y., Guo, J., et al.: DeepRank: a new deep architecture for relevance ranking in information retrieval (2017)
Pang, L., Lan, Y., Guo, J., et al.: A deep investigation of deep IR models (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, Q., Wu, Y. (2018). A Multi-level Attention Model for Text Matching. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds) Artificial Neural Networks and Machine Learning – ICANN 2018. ICANN 2018. Lecture Notes in Computer Science(), vol 11139. Springer, Cham. https://doi.org/10.1007/978-3-030-01418-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-01418-6_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01417-9
Online ISBN: 978-3-030-01418-6
eBook Packages: Computer ScienceComputer Science (R0)