short-paper

Using Word Embeddings for Information Retrieval: How Collection and Term Normalization Choices Affect Performance

Authors:

Debasis Ganguly,

Srikanta Bedathur,

Mandar MitraAuthors Info & Claims

CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge Management

Pages 1835 - 1838

https://doi.org/10.1145/3269206.3269277

Published: 17 October 2018 Publication History

Abstract

Neural word embedding approaches, due to their ability to capture semantic meanings of vocabulary terms, have recently gained attention of the information retrieval (IR) community and have shown promising results in improving ad hoc retrieval performance. It has been observed that these approaches are sensitive to various choices made during the learning of word embeddings and their usage, often leading to poor reproducibility. We study the effect of varying following two parameters, viz., i) the term normalization and ii) the choice of training collection, on ad hoc retrieval performance with word2vec and fastText embeddings. We present quantitative estimates of similarity of word vectors obtained under different settings, and use embeddings based query expansion task to understand the effects of these parameters on IR effectiveness.

References

[1]

Qingyao Ai, Liu Yang, Jiafeng Guo, and W. Bruce Croft. 2016. Analysis of the Paragraph Vector Model for Information Retrieval. In Proc. of ICTIR'16 . 133--142.

Digital Library

[2]

Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching Word Vectors with Subword Information. TACL, Vol. 5 (2017), 135--146.

[3]

Fernando Diaz, Bhaskar Mitra, and Nick Craswell. 2016a. Query Expansion with Locally-Trained Word Embeddings. In Proc. of ACL'16 .

[4]

Fernando Diaz, Bhaskar Mitra, and Nick Craswell. 2016b. Query Expansion with Locally-Trained Word Embeddings. In Proc. of ACL'16 . 367--377.

[5]

Debasis Ganguly, Dwaipayan Roy, Mandar Mitra, and Gareth J. F. Jones. 2015. Word Embedding based Generalized Language Model for Information Retrieval. In Proc. of SIGIR'15 . 795--798.

Digital Library

[6]

Mihajlo Grbovic, Nemanja Djuric, Vladan Radosavljevic, Fabrizio Silvestri, and Narayan Bhamidipati. 2015. Context- and Content-aware Embeddings for Query Rewriting in Sponsored Search. In Proc. of SIGIR '15. 383--392.

Digital Library

[7]

Jiafeng Guo, Yixing Fan, Qingyao Ai, and W. Bruce Croft. 2016. A Deep Relevance Matching Model for Ad-hoc Retrieval. In Proc. of CIKM'16 . 55--64.

Digital Library

[8]

Donna K. Harman (Ed.). 1992. Overview of TREC-1. NIST .

[9]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Proc. NIPS '13 . 3111--3119.

Digital Library

[10]

Bhaskar Mitra, Fernando Diaz, and Nick Craswell. 2017. Learning to Match using Local and Distributed Representations of Text for Web Search. In Proc. of WWW'17. 1291--1299.

Digital Library

[11]

Dwaipayan Roy, Debasis Ganguly, Mandar Mitra, and Gareth J. F. Jones. 2016a. Word Vector Compositionality based Relevance Feedback using Kernel Density Estimation. In Proc. of CIKM'16. 1281--1290.

Digital Library

[12]

Dwaipayan Roy, Debjyoti Paul, Mandar Mitra, and Utpal Garain. 2016b. Using Word Embeddings for Automatic Query Expansion. In Proc. of NeuIR-2016 Workshop, collocated with SIGIR .

[13]

Hamed Zamani and W. Bruce Croft. 2016. Embedding-based Query Language Models. In Proc. of ICTIR'16. 147--156.

Digital Library

[14]

Guoqing Zheng and Jamie Callan. 2015. Learning to Reweight Terms with Distributed Representations. In Proc. of SIGIR '15. 575--584.

Digital Library

[15]

Guido Zuccon, Bevan Koopman, Peter Bruza, and Leif Azzopardi. 2015. Integrating and Evaluating Neural Word Embeddings in Information Retrieval. In Proc. of ADCS '15. 12:1--12:8.

Digital Library

Cited By

Faggioli GFerro NPerego RTonellotto NHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Dimension Importance Estimation for Dense Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657691(1318-1328)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657691
Liao SZhang RPoblete BMurdock VFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Bias Invariant Approaches for Improving Word Embedding FairnessProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614792(1400-1410)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614792
Wang XMacDonald CTonellotto NOunis I(2023)ColBERT-PRF: Semantic Pseudo-Relevance Feedback for Dense Passage and Document RetrievalACM Transactions on the Web10.1145/357240517:1(1-39)Online publication date: 16-Jan-2023
https://dl.acm.org/doi/10.1145/3572405
Show More Cited By

Index Terms

Using Word Embeddings for Information Retrieval: How Collection and Term Normalization Choices Affect Performance
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing

Recommendations

Query Expansion Using Word Embeddings
CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

We present a suite of query expansion methods that are based on word embeddings. Using Word2Vec's CBOW embedding approach, applied over the entire corpus on which search is performed, we select terms that are semantically related to the query. Our ...
Integrating and Evaluating Neural Word Embeddings in Information Retrieval
ADCS '15: Proceedings of the 20th Australasian Document Computing Symposium

Recent advances in neural language models have contributed new methods for learning distributed vector representations of words (also called word embeddings). Two such methods are the continuous bag-of-words model and the skipgram model. These methods ...
Applications of tf-idf concept to improve monolingual and cross-language information retrieval based on word embeddings
AISS '19: Proceedings of the 1st International Conference on Advanced Information Science and System

This work applied word embeddings for English monolingual information retrieval and Dutch-English cross-language information retrieval. Besides word embeddings, this work also applied tf-idf concept to increase result of relevant documents. We present ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge Management

October 2018

2362 pages

ISBN:9781450360142

DOI:10.1145/3269206

General Chair:
Alfredo Cuzzocrea
University of Trieste, Italy
,
Program Chairs:
James Allan
University of Massachusetts, USA
,
Norman Paton
University of Manchester, United Kingdom
,
Divesh Srivastava
AT&T Labs Research, USA
,
Rakesh Agrawal
Data Insights Lab, USA
,
Andrei Broder
Google Research, USA
,
Mohammed Zaki
Rensselaer Polytechnic Institute, USA
,
Selcuk Candan
Arizona State University, USA
,
Alexandros Labrinidis
University of Pittsburgh, USA
,
Assaf Schuster
Technion, Israel
,
Haixun Wang
Google Research, USA

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

CIKM '18

Sponsor:

CIKM '18: The 27th ACM International Conference on Information and Knowledge Management

October 22 - 26, 2018

Torino, Italy

Acceptance Rates

CIKM '18 Paper Acceptance Rate 147 of 826 submissions, 18%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
548
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)4

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Faggioli GFerro NPerego RTonellotto NHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Dimension Importance Estimation for Dense Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657691(1318-1328)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657691
Liao SZhang RPoblete BMurdock VFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Bias Invariant Approaches for Improving Word Embedding FairnessProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614792(1400-1410)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614792
Wang XMacDonald CTonellotto NOunis I(2023)ColBERT-PRF: Semantic Pseudo-Relevance Feedback for Dense Passage and Document RetrievalACM Transactions on the Web10.1145/357240517:1(1-39)Online publication date: 16-Jan-2023
https://dl.acm.org/doi/10.1145/3572405
Wang XEl-Gohary N(2023)Deep Learning–Based Named Entity Recognition and Resolution of Referential Ambiguities for Enhanced Information Extraction from Construction Safety RegulationsJournal of Computing in Civil Engineering10.1061/(ASCE)CP.1943-5487.000106437:5Online publication date: Sep-2023
https://doi.org/10.1061/(ASCE)CP.1943-5487.0001064
Moon JPosada-Quintero HChon K(2023)Genetic data visualization using literature text-based neural networksNeural Networks10.1016/j.neunet.2023.05.015165:C(562-595)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1016/j.neunet.2023.05.015
Moon JPosada-Quintero HChon K(2023)A literature embedding model for cardiovascular disease prediction using risk factors, symptoms, and genotype informationExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.118930213:PAOnline publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1016/j.eswa.2022.118930
Alshangiti MShi WLima ELiu XYu QRoychoudhury ACadar CKim M(2022)Hierarchical Bayesian multi-kernel learning for integrated classification and summarization of app reviewsProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3540250.3549174(558-569)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3540250.3549174
Roy DMitra MMayr PChowdhury A(2022)Local or Global? A Comparative Study on Applications of Embedding Models for Information RetrievalProceedings of the 5th Joint International Conference on Data Science & Management of Data (9th ACM IKDD CODS and 27th COMAD)10.1145/3493700.3493701(115-119)Online publication date: 8-Jan-2022
https://dl.acm.org/doi/10.1145/3493700.3493701
Datta SGanguly DGreene DMitra MSelcuk Candan KLiu HAkoglu LLuna Dong XTang J(2022)Deep-QPPProceedings of the Fifteenth ACM International Conference on Web Search and Data Mining10.1145/3488560.3498491(201-209)Online publication date: 11-Feb-2022
https://dl.acm.org/doi/10.1145/3488560.3498491
Li RZhao XMoens M(2022)A Brief Overview of Universal Sentence Representation Methods: A Linguistic ViewACM Computing Surveys10.1145/348285355:3(1-42)Online publication date: 26-Mar-2022
https://dl.acm.org/doi/10.1145/3482853
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents