short-paper

Estimating query representativeness for query-performance prediction

Authors:

SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

Pages 853 - 856

https://doi.org/10.1145/2484028.2484107

Published: 28 July 2013 Publication History

Get Access

Abstract

The query-performance prediction (QPP) task is estimating retrieval effectiveness with no relevance judgments. We present a novel probabilistic framework for QPP that gives rise to an important aspect that was not addressed in previous work; namely, the extent to which the query effectively represents the information need for retrieval. Accordingly, we devise a few query-representativeness measures that utilize relevance language models. Experiments show that integrating the most effective measures with state-of-the-art predictors in our framework often yields prediction quality that significantly transcends that of using the predictors alone.

References

[1]

C. Buckley. Why current IR engines fail. In Proceedings of SIGIR, pages 584--585, 2004. Poster.

Digital Library

Google Scholar

[2]

D. Carmel and E. Yom-Tov. Estimating the Query Difficulty for Information Retrieval. Synthesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool Publishers, 2010.

Digital Library

Google Scholar

[3]

S. Cronen-Townsend, Y. Zhou, and W. B. Croft. Predicting query performance. In Proceedings of SIGIR, pages 299--306, 2002.

Digital Library

Google Scholar

[4]

C. Hauff, D. Hiemstra, and F. de Jong. A survey of pre-retrieval query performance predictors. In Proceedings of CIKM, pages 1419--1420, 2008.

Digital Library

Google Scholar

[5]

C. Hauff, V. Murdock, and R. Baeza-Yates. Improved query difficulty prediction for the web. In Proceedings of CIKM, pages 439--448, 2008.

Digital Library

Google Scholar

[6]

B. He and I. Ounis. Inferring query performance using pre-retrieval predictors. In Proceedings of SPIRE, pages 43--54, 2004.

Crossref

Google Scholar

[7]

O. Kurland, A. Shtok, S. Hummel, F. Raiber, D. Carmel, and O. Rom. Back to the roots: A probabilistic framework for query-performance prediction. In Proceedings of CIKM, pages 823--832, 2012.

Digital Library

Google Scholar

[8]

V. Lavrenko and W. B. Croft. Relevance-based language models. In Proceedings of SIGIR, pages 120--127, 2001.

Digital Library

Google Scholar

[9]

A. Shtok, O. Kurland, and D. Carmel. Predicting query performance by query-drift estimation. In Proceedings of ICTIR, pages 305--312, 2009.

Digital Library

Google Scholar

[10]

A. Shtok, O. Kurland, and D. Carmel. Using statistical decision theory and relevance models for query-performance prediction. In Proccedings of SIGIR, pages 259--266, 2010.

Digital Library

Google Scholar

[11]

F. Song and W. B. Croft. A general language model for information retrieval (poster abstract). In Proceedings of SIGIR, pages 279--280, 1999.

Digital Library

Google Scholar

[12]

J. H. Steiger. Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87(2):245--251, 1980.

Crossref

Google Scholar

[13]

C. Zhai and J. D. Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proceedings of SIGIR, pages 334--342, 2001.

Digital Library

Google Scholar

[14]

Y. Zhou and W. B. Croft. Query performance prediction in web search environments. In Proceedings of SIGIR, pages 543--550, 2007.

Digital Library

Google Scholar

Cited By

View all

Roitman HBalog KSetty VLioma CLiu YZhang MBerberich K(2020)ICTIR Tutorial: Modern Query Performance Prediction: Theory and PracticeProceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval10.1145/3409256.3409813(195-196)Online publication date: 14-Sep-2020
https://dl.acm.org/doi/10.1145/3409256.3409813
Zendel OShtok ARaiber FKurland OCulpepper JPiwowarski BChevalier MGaussier EMaarek YNie JScholer F(2019)Information Needs, Queries, and Query Performance PredictionProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3331184.3331253(395-404)Online publication date: 18-Jul-2019
https://dl.acm.org/doi/10.1145/3331184.3331253
Roitman HErera SWeiner BKamps JKanoulas Ede Rijke MFang HYilmaz E(2017)Robust Standard Deviation Estimation for Query Performance PredictionProceedings of the ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3121050.3121087(245-248)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1145/3121050.3121087
Show More Cited By

Index Terms

Estimating query representativeness for query-performance prediction
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Query-performance prediction: setting the expectations straight
SIGIR '14: Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval

The query-performance prediction task has been described as estimating retrieval effectiveness in the absence of relevance judgments. The expectations throughout the years were that improved prediction techniques would translate to improved retrieval ...
Query-Performance Prediction Using Minimal Relevance Feedback
ICTIR '13: Proceedings of the 2013 Conference on the Theory of Information Retrieval

There has been much work on devising query-performance prediction approaches that estimate search effectiveness without relevance judgments (i.e., zero feedback). Specifically, post-retrieval predictors analyze the result list of top-retrieved ...
Wikipedia-based query performance prediction
SIGIR '14: Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval

The query-performance prediction task is to estimate retrieval effectiveness with no relevance judgments. Pre-retrieval prediction methods operate prior to retrieval time. Hence, these predictors are often based on analyzing the query and the corpus ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

July 2013

1188 pages

ISBN:9781450320344

DOI:10.1145/2484028

General Chairs:
Gareth J.F. Jones
Dublin City University, Ireland
,
Páraic Sheridan
Dublin City University, Ireland
,
Program Chairs:
Diane Kelly
University of North Carolina, Chapel Hill, USA
,
Maarten de Rijke
University of Amsterdam, The Netherlands
,
Tetsuya Sakai
Microsoft Research Asia, China

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 July 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tag

query-performance prediction

Qualifiers

Short-paper

Conference

SIGIR '13

Sponsor:

SIGIR

SIGIR '13: The 36th International ACM SIGIR conference on research and development in Information Retrieval

July 28 - August 1, 2013

Dublin, Ireland

Acceptance Rates

SIGIR '13 Paper Acceptance Rate 73 of 366 submissions, 20%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
223
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Roitman HBalog KSetty VLioma CLiu YZhang MBerberich K(2020)ICTIR Tutorial: Modern Query Performance Prediction: Theory and PracticeProceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval10.1145/3409256.3409813(195-196)Online publication date: 14-Sep-2020
https://dl.acm.org/doi/10.1145/3409256.3409813
Zendel OShtok ARaiber FKurland OCulpepper JPiwowarski BChevalier MGaussier EMaarek YNie JScholer F(2019)Information Needs, Queries, and Query Performance PredictionProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3331184.3331253(395-404)Online publication date: 18-Jul-2019
https://dl.acm.org/doi/10.1145/3331184.3331253
Roitman HErera SWeiner BKamps JKanoulas Ede Rijke MFang HYilmaz E(2017)Robust Standard Deviation Estimation for Query Performance PredictionProceedings of the ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3121050.3121087(245-248)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1145/3121050.3121087
Pal DMitra MBhattacharya S(2014)Using Multiple Query Expansion Algorithms to Predict Query PerformanceProceedings of the 2014 Fourth International Conference of Emerging Applications of Information Technology10.1109/EAIT.2014.67(361-364)Online publication date: 19-Dec-2014
https://dl.acm.org/doi/10.1109/EAIT.2014.67

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Query-performance prediction: setting the expectations straight

Query-Performance Prediction Using Minimal Relevance Feedback

Wikipedia-based query performance prediction