poster

A machine learning approach for improved BM25 retrieval

Authors:

Krysta M. Svore,

Christopher J.C. BurgesAuthors Info & Claims

CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Pages 1811 - 1814

https://doi.org/10.1145/1645953.1646237

Published: 02 November 2009 Publication History

Abstract

Despite the widespread use of BM25, there have been few studies examining its effectiveness on a document description over single and multiple field combinations. We determine the effectiveness of BM25 on various document fields. We find that BM25 models relevance on popularity fields such as anchor text and query click information no better than a linear function of the field attributes. We also find query click information to be the single most important field for retrieval. In response, we develop a machine learning approach to BM25-style retrieval that learns, using LambdaRank, from the input attributes of BM25. Our model significantly improves retrieval effectiveness over BM25 and BM25F. Our data-driven approach is fast, effective, avoids the problem of parameter tuning, and can directly optimize for several common information retrieval measures. We demonstrate the advantages of our model on a very large real-world Web data collection.

References

[1]

E. Agichtein, E. Brill, and S. Dumais. Improving web search ranking by incorporating user behavior information. In ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 19--26, 2006.

Digital Library

[2]

C. Burges, R. Ragno, and Q. Le. Learning to rank with nonsmooth cost functions. In Advances in Neural Information Processing Systems (NIPS), 2006. See also MSR Technical Report MSR-TR-2006-60.

[3]

C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. In International Conference on Machine Learning (ICML), Bonn, Germany, 2005.

Digital Library

[4]

N. Craswell and D. Hawking. Overview of the TREC 2004 web track. In Proceedings of TREC 2004, 2004.

[5]

N. Craswell, D. Hawking, R. Wilkinson, and M. Wu. Overview of the TREC 2003 web track. In Proceedings of TREC 2003, 2003.

[6]

N. Craswell and M. Szummer. Random walk on the click graph. In ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2007.

Digital Library

[7]

P. Donmez, K. Svore, and C. Burges. On the local optimality of LambdaRank. In ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2009.

Digital Library

[8]

J. Gao, W. Yuan, X. Li, K. Deng, and J.-Y. Nie. Smoothing clickthrough data for web search ranking. In ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2009.

Digital Library

[9]

B. He and I. Ounis. On setting the hyper-parameters of term frequency normalization for information retrieval. ACM Transactions on Information Systems (TOIS), 25(3):13, 2007.

Digital Library

[10]

K. Jarvelin and J. Kekalainen. IR evaluation methods for retrieving highly relevant documents. In ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 41--48, 2000.

Digital Library

[11]

T. Joachims. Optimizing search engines using clickthrough data. In SIGKDD, pages 133--142, 2002.

Digital Library

[12]

D. Metzler. Generalized inverse document frequency. In ACM Conference on Information Knowledge Management (CIKM), 2008.

Digital Library

[13]

P. Ogilvie and J. Callan. Combining document representations for known item search. In ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2003.

Digital Library

[14]

Y. Rasolofo and J. Savoy. Term proximity scoring for keyword-based retrieval systems. In Proceedings of the 25th European Conference on IR Research (ECIR), 2003.

Digital Library

[15]

S. Robertson and S. Walker. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 345--354, 1994.

Digital Library

[16]

S. Robertson, H. Zaragoza, and M. Taylor. Simple BM25 extension to multiple weighted fields. In ACM Conference on Information Knowledge Management (CIKM), pages 42--49, 2004.

Digital Library

[17]

A. Singhal, C. Buckley, and M. Mitra. Pivoted document length normalization. In ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 21--29, 1996.

Digital Library

[18]

R. Song, M. Taylor, J.-R. Wen, H.-W. Hon, and Y. Yu. Viewing term proximity from a different perspective. Advances in Information Retrieval, Lecture Notes in Computer Science, 4956/2008:346--357, 2008.

Digital Library

[19]

K. Sparck-Jones, S. Walker, and S. Robertson. A probabilistic model of information retrieval: development and comparative experiments. Information Processing and Management, 36:809--840, 2000.

Digital Library

[20]

K. Svore and C. Burges. A machine learning approach improved bm25 retrieval. Microsoft Technical Report MSR-TR-2009-92, 2009.

[21]

M. Taylor, H. Zaragoza, N. Craswell, S. Robertson, and C. Burges. Optimisation methods for ranking functions with multiple parameters. In ACM Conference on Information Knowledge Management (CIKM), 2006.

Digital Library

[22]

R. Wilkinson. Effective retrieval of structured documents. In Research and Development in Information Retrieval, pages 311--317, 1994.

Digital Library

[23]

Q. Wu, C. Burges, K. Svore, and J. Gao. Ranking, boosting and model adaptation. Microsoft Technical Report MSR-TR-2008-109, 2008.

[24]

G. Xue, H.-J. Zeng, Z. Chen, Y. Yu, W.-Y. Ma, W. Xi, and W. Fan. Optimizing web search using web click-through information. In ACM Conference on Information Knowledge Management (CIKM), 2004.

Digital Library

[25]

Y. Yue and C. Burges. On using simultaneous perturbation stochastic approximation for IR measures, and the empirical optimality of LambdaRank. NIPS Machine Learning for Web Search Workshop, 2007.

Cited By

Zhao KZhang HLi JPan QLai LNie YZhang Z(2024)Social Network Forensics Analysis Model Based on Network Representation LearningEntropy10.3390/e2607057926:7(579)Online publication date: 7-Jul-2024
https://doi.org/10.3390/e26070579
Chen ZChen WXu JLiu ZZhang WFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Beyond Semantics: Learning a Behavior Augmented Relevance Model with Self-supervised LearningProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615457(4516-4522)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615457
Ni WShen QLiu TZeng QXu L(2023)Generating textual emergency plans for unconventional emergencies — A natural language processing approachSafety Science10.1016/j.ssci.2022.106047160(106047)Online publication date: Apr-2023
https://doi.org/10.1016/j.ssci.2022.106047
Show More Cited By

Index Terms

A machine learning approach for improved BM25 retrieval
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information retrieval

Recommendations

How good is a span of terms?: exploiting proximity to improve web retrieval
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval

Ranking search results is a fundamental problem in information retrieval. In this paper we explore whether the use of proximity and phrase information can improve web retrieval accuracy. We build on existing research by incorporating novel ranking ...
BM25t: a BM25 extension for focused information retrieval

This paper addresses the integration of XML tags into a term-weighting function for focused XML information retrieval (IR). Our model allows us to consider a certain kind of structural information: tags that represent a logical structure (e.g., title, ...
BM25-AH: Enhanced BM25 Algorithm for Domain-Specific Search Engine
iiWAS2019: Proceedings of the 21st International Conference on Information Integration and Web-based Applications & Services

The Virginia Military Institute (VMI) uses Google search to provide webpage search service in the VMI website. As Google search is a general-purpose service, it does not consider VMI-specific information and in turn, often fails to retrieve relevant ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

November 2009

2162 pages

ISBN:9781605585123

DOI:10.1145/1645953

General Chairs:
David Cheung
University of Hong Kong, Hong Kong
,
Il-Yeol Song
Drexel University, USA
,
Program Chairs:
Wesley Chu
UCLA, USA
,
Xiaohua Hu
Drexel University, USA
,
Jimmy Lin
University of Maryland, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

CIKM '09

Sponsor:

CIKM '09: Conference on Information and Knowledge Management

November 2 - 6, 2009

Hong Kong, China

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
779
Total Downloads

Downloads (Last 12 months)83
Downloads (Last 6 weeks)17

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhao KZhang HLi JPan QLai LNie YZhang Z(2024)Social Network Forensics Analysis Model Based on Network Representation LearningEntropy10.3390/e2607057926:7(579)Online publication date: 7-Jul-2024
https://doi.org/10.3390/e26070579
Chen ZChen WXu JLiu ZZhang WFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Beyond Semantics: Learning a Behavior Augmented Relevance Model with Self-supervised LearningProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615457(4516-4522)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615457
Ni WShen QLiu TZeng QXu L(2023)Generating textual emergency plans for unconventional emergencies — A natural language processing approachSafety Science10.1016/j.ssci.2022.106047160(106047)Online publication date: Apr-2023
https://doi.org/10.1016/j.ssci.2022.106047
Seo JLee TMoon HPark CEo SAiyanyo IPark KSo AAhn SPark J(2022)Dense-to-Question and Sparse-to-Answer: Hybrid Retriever System for Industrial Frequently Asked QuestionsMathematics10.3390/math1008133510:8(1335)Online publication date: 18-Apr-2022
https://doi.org/10.3390/math10081335
Zuluaga Cajiao ARosso Mateus A(2022)Graph-based Similarity for Document Retrieval in the Biomedical DomainProceedings of the 2022 7th International Conference on Machine Learning Technologies10.1145/3529399.3529428(180-184)Online publication date: 11-Mar-2022
https://dl.acm.org/doi/10.1145/3529399.3529428
Seo JOh DEo SPark CYang KMoon HPark KLim H(2022)PU-GEN: Enhancing generative commonsense reasoning for language models with human-centered knowledgeKnowledge-Based Systems10.1016/j.knosys.2022.109861256(109861)Online publication date: Nov-2022
https://doi.org/10.1016/j.knosys.2022.109861
Arabzadeh NVtyurina AYan XClarke C(2022)Shallow pooling for sparse labelsInformation Retrieval Journal10.1007/s10791-022-09411-025:4(365-385)Online publication date: 20-Jul-2022
https://doi.org/10.1007/s10791-022-09411-0
Shan XLiu CXia YChen QZhang YDing KLiang YLuo ALuo Y(2021)GLOW : Global Weighted Self-Attention Network for Web Search2021 IEEE International Conference on Big Data (Big Data)10.1109/BigData52589.2021.9671546(519-528)Online publication date: 15-Dec-2021
https://doi.org/10.1109/BigData52589.2021.9671546
Peng SXie XZhai JJia YGong Y(2021)A Page-topic Relevance Algorithm Based on BM25 and Paragraph-Semantic CorrelationJournal of Physics: Conference Series10.1088/1742-6596/1757/1/0121151757:1(012115)Online publication date: 1-Jan-2021
https://doi.org/10.1088/1742-6596/1757/1/012115
Souflas KKoutmos P(2021)Effects of Stratification and Preheat on Turbulent Flame Characteristics and StabilizationFlow, Turbulence and Combustion10.1007/s10494-021-00267-wOnline publication date: 11-May-2021
https://doi.org/10.1007/s10494-021-00267-w
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents