research-article

Enhanced question understanding with dynamic memory networks for textual question answering

Authors:

Ming LiAuthors Info & Claims

Expert Systems with Applications: An International Journal, Volume 80, Issue C

Pages 39 - 45

https://doi.org/10.1016/j.eswa.2017.03.006

Published: 01 September 2017 Publication History

Abstract

Introducing global and hierarchical salient features of inputs.Adopting a modified network to extract hierarchical salient features of a question.Finding a method to utilize these features to construct multiple feature sets. Memory networks show promising context understanding and reasoning capabilities in Textual Question Answering (Textual QA). We improve the previous dynamic memory networks to do Textual QA by processing inputs to simultaneously extract global and hierarchical salient features. We then use them to construct multiple feature sets at each reasoning step. Experiments were conducted on a public Textual Question Answering dataset (Facebook bAbI dataset) in two ways: with and without supervision from labels of supporting facts. Compared to previous works such as Dynamic Memory Networks, our models show better accuracy and stability.

References

[1]

J. Andreas, M. Rohrbach, T. Darrell, D. Klein, Learning to compose neural networks for question answering, 2016.

[2]

S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. Lawrence Zitnick, D. Parikh, Vqa: Visual question answering, 2015.

[3]

R. Bhati, S. Prasad, Open domain question answering system using cognitive computing, IEEE, 2016.

[4]

S. Chang, A. Pal, Routing questions for collaborative answering in community question answering, ACM, 2013.

[5]

K. Cho, B. van Merrienboer, D. Bahdanau, Y. Bengio, On the properties of neural machine translation: encoder-decoder approaches, 2014.

[6]

J. Chung, C. Gulcehre, K. Cho, Y. Bengio, Empirical evaluation of gated recurrent neural networks on sequence modeling, 2014.

[7]

Y.B. Dzmitry Bahdanau, C. Kyunghyun, Neural machine translation by jointly learning to align and translate, 2015.

[8]

J. Fan, A. Kalyanpur, D.C. Gondek, D. Ferrucci, Automatic knowledge extraction from documents, IBM Journal of Research and Development, 56 (2012) 5:1-5:10.

Digital Library

[9]

Graves, A., Wayne, G., & Danihelka, I. (2014). Neural turing machines. arXiv preprint. arXiv:1410.5401.

[10]

S. Hochreiter, J.J. Schmidhuber, Long short-term memory, Neural Computation, 9 (1997) 1735-1780.

Digital Library

[11]

B. Hu, Z. Lu, H. Li, Q. Chen, Convolutional neural network architectures for matching natural language sentences, 2014.

[12]

P. Jackson, Introduction to expert systems (3rd edition), Addison-Wesley, Boston, Massachusetts, 1998.

Digital Library

[13]

N. Kalchbrenner, P. Blunsom, Recurrent continuous translation models, 2013.

[14]

Y. Kim, Convolutional neural networks for sentence classification, 2014.

[15]

D. Kingma, J. Ba, Adam: A method for stochastic optimization, 2015.

[16]

A. Kumar, O. Irsoy, P. Ondruska, M. Iyyer, J. Bradbury, I. Gulrajani, R. Socher, Ask me anything: Dynamic memory networks for natural language processing, 2016.

[17]

S. Lai, L. Xu, K. Liu, J. Zhao, Recurrent Convolutional Neural Networks for Text Classification, 2015.

[18]

B. Loni, A survey of state-of-the-art methods on question classification, Electrical Engineering Mathematics & Computer Science (2011) 1-40.

[19]

M.-T. Luong, H. Pham, C.D. Manning, Effective approaches to attention-based neural machine translation, 2015.

[20]

F. Meng, Z. Lu, Z. Tu, H. Li, Q. Liu, A deep memory-based architecture for sequence-to-sequence learning, 2016.

[21]

A. Ng, Feature selection, L 1vs. L 2 regularization, and rotational invariance, 2004.

[22]

Peng, B., Lu, Z., Li, H., & Wong, K.-F. (2015). Towards neural network-based reasoning. arXiv preprint. arXiv:1508.05508.

[23]

T. Sahu, N. Nagwani, S. Verma, Multivariate beta mixture model for automatic identification of topical authoritative users in community question answering sites, IEEE Access (2016).

[24]

T.P. Sahu, N.K. Nagwani, S. Verma, Selecting best answer: An empirical analysis on community question answering sites, IEEE Access, 4 (2016) 4797-4808.

[25]

M. Schuster, K.K. Paliwal, Bidirectional recurrent neural networks, IEEE Transactions on Signal Processing, 45 (1997) 2673-2681.

Digital Library

[26]

N. Srivastava, G.E. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting., Journal of Machine Learning Research, 15 (2014) 1929-1958.

Digital Library

[27]

S. Sukhbaatar, A. Szlam, End-to-end memory networks, 2015.

[28]

H. Sun, H. Ma, W.-t. Yih, C.-T. Tsai, J. Liu, M.-W. Chang, Open domain question answering via semantic enrichment, ACM, 2015.

[29]

I. Sutskever, O. Vinyals, Q.V. Le, Sequence to sequence learning with neural networks, 2014.

[30]

O. Vinyals, A. Toshev, S. Bengio, D. Erhan, Show and tell: A neural image caption generator, 2015.

[31]

Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merrinboer, B., Joulin, A., & Mikolov, T. (2015). Towards AI-complete question answering: A set of prerequisite toy tasks. arXiv preprint. arXiv:1502.05698.

[32]

J. Weston, S. Chopra, A. Bordes, Memory networks, 2015.

[33]

C. Xiong, S. Merity, R. Socher, Dynamic memory networks for visual and textual question answering, 2016.

[34]

K. Xu, A. Courville, R.S. Zemel, Y. Bengio, Show, attend and tell : Neural image caption generation with visual attention, 2015.

[35]

Z. Yang, X. He, J. Gao, L. Deng, A. Smola, Stacked attention networks for image question answering, 2015.

[36]

Yu, Y., Zhang, W., Hang, C.-W., & Zhou, B. (2015). Empirical study on deep learning models for QA. arXiv preprint. arXiv:1510.07526.

Cited By

Park YKo YSeo J(2022)BERT-based response selection in dialogue systems using utterance attention mechanismsExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.118277209:COnline publication date: 15-Dec-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.118277
Li MLu XChen LWang J(2022)Knowledge map construction for question and answer archivesExpert Systems with Applications: An International Journal10.1016/j.eswa.2019.112923141:COnline publication date: 21-Apr-2022
https://dl.acm.org/doi/10.1016/j.eswa.2019.112923
Hao TLi XHe YWang FQu Y(2022)Recent progress in leveraging deep learning methods for question answeringNeural Computing and Applications10.1007/s00521-021-06748-334:4(2765-2783)Online publication date: 1-Feb-2022
https://dl.acm.org/doi/10.1007/s00521-021-06748-3
Show More Cited By

Recommendations

Lightweight web-based fact repositories for textual question answering
CIKM '07: Proceedings of the sixteenth ACM conference on Conference on information and knowledge management

Since answers to fact-seeking questions usually reside within small factual text nuggets, often "hidden" within full-length documents, their relevance to a question is not necessarily correlated to the relevance of the full-length document to the ...
Prominent feature extraction for evidence gathering in question answering
Soft computing and intelligent systems: Tools, techniques and applications

Question Answering (QA) research is a significant and challenging task in Natural Language Processing. QA aims to extract an exact answer from a relevant text snippet or a document. The motivation behind QA research is the need of user who is using state-...
QUESTION ANSWERING USING QUESTION CLASSIFICATION AND DOCUMENT TAGGING

Question answering (QA) is a relatively new area of research. We took the approach of designing a question answering system that is based on question classification and document tagging. Question classification extracts useful information from the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 80, Issue C

September 2017

356 pages

ISSN:0957-4174

Issue’s Table of Contents

Copyright © Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 September 2017

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Park YKo YSeo J(2022)BERT-based response selection in dialogue systems using utterance attention mechanismsExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.118277209:COnline publication date: 15-Dec-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.118277
Li MLu XChen LWang J(2022)Knowledge map construction for question and answer archivesExpert Systems with Applications: An International Journal10.1016/j.eswa.2019.112923141:COnline publication date: 21-Apr-2022
https://dl.acm.org/doi/10.1016/j.eswa.2019.112923
Hao TLi XHe YWang FQu Y(2022)Recent progress in leveraging deep learning methods for question answeringNeural Computing and Applications10.1007/s00521-021-06748-334:4(2765-2783)Online publication date: 1-Feb-2022
https://dl.acm.org/doi/10.1007/s00521-021-06748-3
Shen YLai EMohaghegh M(2020)Role of RNNs for Non-sequential Tasks in The Question Answering ContextProceedings of the 2020 4th International Symposium on Computer Science and Intelligent Control10.1145/3440084.3441216(1-6)Online publication date: 17-Nov-2020
https://dl.acm.org/doi/10.1145/3440084.3441216
Cai LZhou SYan XYuan R(2019)A Stacked BiLSTM Neural Network Based on Coattention Mechanism for Question AnsweringComputational Intelligence and Neuroscience10.1155/2019/95434902019Online publication date: 21-Aug-2019
https://dl.acm.org/doi/10.1155/2019/9543490

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents