Article

A geometric interpretation of r-precision and its correlation with average precision

Authors:

SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval

Pages 573 - 574

https://doi.org/10.1145/1076034.1076134

Published: 15 August 2005 Publication History

Get Access

Abstract

We consider two of the most commonly cited measures of retrieval performance: average precision and R-precision. It is well known that average precision and R-precision are highly correlated and similarly robust measures of performance, though the reasons for this are not entirely clear. In this paper, we give a geometric argument which shows that under a very reasonable set of assumptions, average precision and R-precision both approximate the area under the precision-recall curve, thus explaining their high correlation. We further demonstrate through the use of TREC data that the similarity or difference between average precision and R-precision is largely governed by the adherence to, or violation of, these reasonable assumptions.

References

[1]

C. Buckley and E. M. Voorhees. Evaluating evaluation measure stability. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 33--40. ACM Press, 2000.

Digital Library

Google Scholar

[2]

J. Tague-Sutcliffe and J. Blustein. A statistical analysis of the TREC-3 data. In Proceedings of the Third Text Retrieval Conference (TREC-3), pages 385--398, 1995.

Google Scholar

[3]

E. M. Voorhees and D. Harman. Overview of the seventh text retrieval conference (TREC-7). In Proceedings of the Seventh Text Retrieval Conference (TREC-7), pages 1--23, 1999.

Crossref

Google Scholar

Cited By

View all

Susanto BFerdiana RAdji T(2024)Performance of Traditional and Dense Vector Information Retrieval Models in Code Search2024 2nd International Conference on Software Engineering and Information Technology (ICoSEIT)10.1109/ICoSEIT60086.2024.10497512(52-57)Online publication date: 28-Feb-2024
https://doi.org/10.1109/ICoSEIT60086.2024.10497512
Aziz MWang APappu ABouchard HZhao YCarterette BLalmas MDemartini GZuccon GCulpepper JHuang ZTong H(2021)Leveraging Semantic Information to Facilitate the Discovery of Underserved PodcastsProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3481934(3707-3716)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3481934
Akabe KTakeuchi TAoki TNishimura K(2021)Information retrieval on oncology knowledge base using recursive paraphrase latticeJournal of Biomedical Informatics10.1016/j.jbi.2021.103705116:COnline publication date: 1-Apr-2021
https://dl.acm.org/doi/10.1016/j.jbi.2021.103705
Show More Cited By

Index Terms

A geometric interpretation of r-precision and its correlation with average precision
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results

Recommendations

Extending average precision to graded relevance judgments
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval

Evaluation metrics play a critical role both in the context of comparative evaluation of the performance of retrieval systems and in the context of learning-to-rank (LTR) as objective functions to be optimized. Many different evaluation metrics have ...
A new interpretation of average precision
SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval

We consider the question of whether Average Precision, as a measure of retrieval effectiveness, can be regarded as deriving from a model of user searching behaviour. It turns out that indeed it can be so regarded, under a very simple stochastic model of ...
Estimating average precision with incomplete and imperfect judgments
CIKM '06: Proceedings of the 15th ACM international conference on Information and knowledge management

We consider the problem of evaluating retrieval systems using incomplete judgment information. Buckley and Voorhees recently demonstrated that retrieval systems can be efficiently and effectively evaluated using incomplete judgments via the bpref ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval

August 2005

708 pages

ISBN:1595930345

DOI:10.1145/1076034

General Chairs:
Ricardo Baeza-Yates
University of Chile, Chile
,
Nivio Ziviani
Federal University of Minas Gerais, Brazil
,
Program Chairs:
Gary Marchionini
University of North Carolina, USA
,
Alistair Moffat
University of Melbourne, Australia
,
John Tait
University of Sunderland, UK

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 August 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGIR05

Sponsor:

SIGIR

SIGIR05: The 28th ACM/SIGIR International Symposium on Information Retrieval 2005

August 15 - 19, 2005

Salvador, Brazil

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
699
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)3

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Susanto BFerdiana RAdji T(2024)Performance of Traditional and Dense Vector Information Retrieval Models in Code Search2024 2nd International Conference on Software Engineering and Information Technology (ICoSEIT)10.1109/ICoSEIT60086.2024.10497512(52-57)Online publication date: 28-Feb-2024
https://doi.org/10.1109/ICoSEIT60086.2024.10497512
Aziz MWang APappu ABouchard HZhao YCarterette BLalmas MDemartini GZuccon GCulpepper JHuang ZTong H(2021)Leveraging Semantic Information to Facilitate the Discovery of Underserved PodcastsProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3481934(3707-3716)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3481934
Akabe KTakeuchi TAoki TNishimura K(2021)Information retrieval on oncology knowledge base using recursive paraphrase latticeJournal of Biomedical Informatics10.1016/j.jbi.2021.103705116:COnline publication date: 1-Apr-2021
https://dl.acm.org/doi/10.1016/j.jbi.2021.103705
Ferro NKim YSanderson M(2019)Using Collection Shards to Study Retrieval Performance Effect SizesACM Transactions on Information Systems10.1145/331036437:3(1-40)Online publication date: 19-Mar-2019
https://dl.acm.org/doi/10.1145/3310364
O'Neil GSaby LBand LGoodall J(2019)Effects of LiDAR DEM Smoothing and Conditioning Techniques on a Topography‐Based Wetland Identification ModelWater Resources Research10.1029/2019WR02478455:5(4343-4363)Online publication date: 27-May-2019
https://doi.org/10.1029/2019WR024784
Gupta SKutlu MKhetan VLease M(2019)Correlation, Prediction and Ranking of Evaluation Metrics in Information RetrievalAdvances in Information Retrieval10.1007/978-3-030-15712-8_41(636-651)Online publication date: 7-Apr-2019
https://doi.org/10.1007/978-3-030-15712-8_41
Mahmoud ABradshaw G(2015)Estimating Semantic Relatedness in Source CodeACM Transactions on Software Engineering and Methodology10.1145/282425125:1(1-35)Online publication date: 2-Dec-2015
https://dl.acm.org/doi/10.1145/2824251
Mahmoud ANiu N(2015)On the role of semantics in automated requirements tracingRequirements Engineering10.1007/s00766-013-0199-y20:3(281-300)Online publication date: 1-Sep-2015
https://dl.acm.org/doi/10.1007/s00766-013-0199-y
Keilwagen JGrosse IGrau J(2014)Area under Precision-Recall Curves for Weighted and Unweighted DataPLoS ONE10.1371/journal.pone.00922099:3(e92209)Online publication date: 20-Mar-2014
https://doi.org/10.1371/journal.pone.0092209
Mahmoud ANiu N(2014)Supporting requirements to code traceability through refactoringRequirements Engineering10.1007/s00766-013-0197-019:3(309-329)Online publication date: 1-Sep-2014
https://dl.acm.org/doi/10.1007/s00766-013-0197-0
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Extending average precision to graded relevance judgments

A new interpretation of average precision

Estimating average precision with incomplete and imperfect judgments