Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/2806416.2806490acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Assessing the Impact of Syntactic and Semantic Structures for Answer Passages Reranking

Published: 17 October 2015 Publication History

Abstract

In this paper, we extensively study the use of syntactic and semantic structures obtained with shallow and deeper syntactic parsers in the answer passage reranking task. We propose several dependency-based structures enriched with Linked Open Data (LD) knowledge for representing pairs of questions and answer passages. We use such tree structures in learning to rank (L2R) algorithms based on tree kernel. The latter can represent questions and passages in a tree fragment space, where each substructure represents a powerful syntactic/semantic feature. Additionally since we define links between structures, tree kernels also generate relational features spanning question and passage structures. We derive very important findings, which can be useful to build state-of-the-art systems: (i) full syntactic dependencies can outperform shallow models also using external knowledge and (ii) the semantic information should be derived by effective and high-coverage resources, e.g., LD, and incorporated in syntactic structures to be effective. We demonstrate our findings by carrying out an extensive comparative experimentation on two different TREC QA corpora and one community question answer dataset, namely Answerbag. Our comparative analysis on well-defined answer selection benchmarks consistently demonstrates that our structural semantic models largely outperform the state of the art in passage reranking.

References

[1]
E. Aktolga, J. Allan, and D. A. Smith. Passage reranking for question answering using syntactic structures and answer types. In ECIR, 2011.
[2]
M. W. Bilotti, J. L. Elsas, J. Carbonell, and E. Nyberg. Rank learning for factoid question answering with linguistic and semantic constraints. In CIKM, 2010.
[3]
M. W. Bilotti and E. Nyberg. Improving text retrieval precision and answer accuracy in question answering systems. In (IR4QA) Workshop at COLING, 2008.
[4]
C. Bizer, J. Lehmann, G. Kobilarov, S. Auer, C. Becker, R. Cyganiak, and S. Hellmann. Dbpedia - a crystallization point for the web of data. Web Semantics, 7(3), 2009.
[5]
C. Bizer and A. Schultz. The berlin sparql benchmark. IJSWIS, 5(2):1--24, 2009.
[6]
B. Bohnet. Top accuracy and fast dependency parsing is not a contradiction. In COLING, 2010.
[7]
J. D. Choi and M. Palmer. Getting the most out of transition-based dependency parsing. In ACL, 2011.
[8]
D. Croce, A. Moschitti, and R. Basili. Structured lexical similarity via convolution kernels on dependency trees. In Proceedings of EMNLP, 2011.
[9]
A. Csomai and R. Mihalcea. Linking documents to encyclopedic knowledge. IEEE Intelligent Systems, 23(5):34--41, 2008.
[10]
R. E. de Castilho and I. Gurevych. A broad-coverage collection of portable NLP components for building shareable analysis pipelines. In OIAF4HLT Workshop (COLING), 2014.
[11]
J. Fan, D. Ferrucci, D. Gondek, and A. Kalyanpur. Prismatic: Inducing knowledge from a large scale lexicalized relation resource. In NAACL HLT, 2010.
[12]
C. Fellbaum, editor. WordNet: An Electronic Lexical Database. MIT Press, 1998.
[13]
D. Ferrucci, E. Brown, J. Chu-Carroll, J. Fan, D. Gondek, A. Kalyanpur, A. Lally, J. W. Murdock, E. Nyberg, J. Prager, N. Schlaefer, and C. Welty. Building Watson: An Overview of the DeepQA Project. AI Magazine, 31(3), 2010.
[14]
M. Heilman and N. A. Smith. Tree edit models for recognizing textual entailments, paraphrases, and answers to questions. In NAACL, 2010.
[15]
A. Hickl, J. Williams, J. Bensley, K. Roberts, Y. Shi, and B. Rink. Question answering with lcc chaucer at trec 2006. In TREC, 2006.
[16]
J. Hoffart, F. Suchanek, K. Berberich, and G. Weikum. Yago2: a spatially and temporally enhanced knowledge base from wikipedia. Artificial Intelligence, 2012.
[17]
J. Jeon, W. B. Croft, and J. H. Lee. Finding similar questions in large question and answer archives. In CIKM, 2005.
[18]
T. Joachims. Optimizing search engines using clickthrough data. In ACM SIGKDD KDD, 2002.
[19]
B. Katz and J. Lin. Selectively using relations to improve precision in question answering. In Workshop on NLP for QA (EACL), 2003.
[20]
X. Li and D. Roth. Learning question classifiers. In Proceedings of ACL, 2002.
[21]
C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, and D. McClosky. The Stanford CoreNLP natural language processing toolkit. In ACL, 2014.
[22]
M. C. McCord, J. W. Murdock, and B. Boguraev. Deep parsing in Watson. IBM Journal, 56(3), 2012.
[23]
D. Milne and I. Witten. An open-source toolkit for mining wikipedia. In NZCSRSC, volume 9, 2009.
[24]
A. Moschitti. Efficient convolution kernels for dependency and constituent syntactic trees. In ECML, 2006.
[25]
J. Nivre, J. Hall, J. Nilsson, A. Chanev, G. Eryigit, S. Kübler, S. Marinov, and E. Marsi. Maltparser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 2007.
[26]
M. Pasca. Open-Domain Question Answering from Large Text Collections. CSLI Publications, 2003.
[27]
V. Punyakanok and D. Roth. The use of classifiers in sequential inference. In NIPS, pages 995--1001, 2001.
[28]
F. Radlinski and T. Joachims. Query chains: Learning to rank from implicit feedback. CoRR, 2006.
[29]
A. Severyn and A. Moschitti. Automatic feature engineering for answer selection and extraction. In EMNLP, 2013.
[30]
A. Severyn and A. Moschitti. Learning to rank short text pairs with convolutional deep neural networks. In SIGIR, 2015.
[31]
A. Severyn, M. Nicosia, and A. Moschitti. Building structures from classifiers for passage reranking. In CIKM, 2013.
[32]
A. Severyn, M. Nicosia, and A. Moschitti. Learning adaptable patterns for passage reranking. CoNLL-2013, page 75, 2013.
[33]
J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
[34]
D. Shen and M. Lapata. Using semantic roles to improve question answering. In EMNLP-CoNLL, 2007.
[35]
S. Small, T. Strzalkowski, T. Liu, S. Ryan, R. Salkin, N. Shimizu, P. Kantor, D. Kelly, and N. Wacholder. Hitiqa: Towards analytical question answering. In COLING, 2004.
[36]
M. Surdeanu, M. Ciaramita, and H. Zaragoza. Learning to rank answers on large online QA collections. In ACL, 2008.
[37]
K. Tymoshenko, A. Moschitti, and A. Severyn. Encoding semantic resources in syntactic structures for passage reranking. In Proceedings of EACL, 2014.
[38]
E. M. Voorhees. Overview of the TREC 2001 Question Answering Track. In Proceedings of TREC, 2001.
[39]
E. M. Voorhees. Overview of the trec 2003 question answering track. In TREC 2003, 2003.
[40]
D. Wang and E. Nyberg. A long short-term memory model for answer sentence selection in question answering. In ACL, July 2015.
[41]
M. Wang and C. D. Manning. Probabilistic tree-edit models with structured latent variables for textual entailment and question answering. In ACL, 2010.
[42]
M. Wang, N. A. Smith, and T. Mitamura. What is the jeopardy model? a quasi-synchronous grammar for qa. In EMNLP-CoNLL, 2007.
[43]
P. C. Xuchen Yao, Benjamin Van Durme and C. Callison-Burch. Answer extraction as sequence tagging with tree edit distance. In NAACL, 2013.
[44]
X. Yao, B. Van Durme, and P. Clark. Answer extraction as sequence tagging with tree edit distance. In Proceedings of NAACL-HLT, pages 858--867, 2013.
[45]
W.-t. Yih, M.-W. Chang, C. Meek, and A. Pastusiak. Question answering using enhanced lexical semantic models. In ACL, 2013.

Cited By

View all
  • (2023)User Context-Aware Attention Networks for Answer SelectionWeb Information Systems Engineering – WISE 202310.1007/978-981-99-7254-8_6(67-81)Online publication date: 21-Oct-2023
  • (2022)Answer selection in community question answering exploiting knowledge graph and context informationSemantic Web10.3233/SW-22297013:3(339-356)Online publication date: 1-Jan-2022
  • (2022)Natural Language Processing with Improved Deep Learning Neural NetworksScientific Programming10.1155/2022/60286932022Online publication date: 1-Jan-2022
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management
October 2015
1998 pages
ISBN:9781450337946
DOI:10.1145/2806416
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2015

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. kernel methods
  2. learning to rank
  3. linked data
  4. question answering
  5. structural kernels

Qualifiers

  • Research-article

Funding Sources

  • IBM
  • H2020-ICT-2014-2

Conference

CIKM'15
Sponsor:

Acceptance Rates

CIKM '15 Paper Acceptance Rate 165 of 646 submissions, 26%;
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 18 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2023)User Context-Aware Attention Networks for Answer SelectionWeb Information Systems Engineering – WISE 202310.1007/978-981-99-7254-8_6(67-81)Online publication date: 21-Oct-2023
  • (2022)Answer selection in community question answering exploiting knowledge graph and context informationSemantic Web10.3233/SW-22297013:3(339-356)Online publication date: 1-Jan-2022
  • (2022)Natural Language Processing with Improved Deep Learning Neural NetworksScientific Programming10.1155/2022/60286932022Online publication date: 1-Jan-2022
  • (2022)BertHANK: hierarchical attention networks with enhanced knowledge and pre-trained model for answer selectionKnowledge and Information Systems10.1007/s10115-022-01703-764:8(2189-2213)Online publication date: 1-Aug-2022
  • (2020)Aspect-Level Sentiment Difference Feature Interaction Matching Model Based on Multi-round Decision MechanismAlgorithms and Architectures for Parallel Processing10.1007/978-3-030-60239-0_32(477-491)Online publication date: 2-Oct-2020
  • (2019)Learning pairwise patterns in Community Question AnsweringIntelligenza Artificiale10.3233/IA-17003412:2(49-65)Online publication date: 29-Jan-2019
  • (2019)A question-entailment approach to question answeringBMC Bioinformatics10.1186/s12859-019-3119-420:1Online publication date: 22-Oct-2019
  • (2019)A Deep Neural Network Framework for English Hindi Question AnsweringACM Transactions on Asian and Low-Resource Language Information Processing10.1145/335998819:2(1-22)Online publication date: 21-Nov-2019
  • (2019)What We Vote for? Answer Selection from User Expertise View in Community Question AnsweringThe World Wide Web Conference10.1145/3308558.3313510(1198-1209)Online publication date: 13-May-2019
  • (2019)Syntax Tree Aware Adversarial Question Rewriting for Answer Selection2019 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2019.8852107(1-8)Online publication date: Jul-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media