research-article

TREC-COVID: constructing a pandemic information retrieval test collection

Authors:

Kyle Lo,

Lucy Lu WangAuthors Info & Claims

ACM SIGIR Forum, Volume 54, Issue 1

Article No.: 1, Pages 1 - 12

https://doi.org/10.1145/3451964.3451965

Published: 19 February 2021 Publication History

Get Access

Abstract

TREC-COVID is a community evaluation designed to build a test collection that captures the information needs of biomedical researchers using the scientific literature during a pandemic. One of the key characteristics of pandemic search is the accelerated rate of change: the topics of interest evolve as the pandemic progresses and the scientific literature in the area explodes. The COVID-19 pandemic provides an opportunity to capture this progression as it happens. TREC-COVID, in creating a test collection around COVID-19 literature, is building infrastructure to support new research and technologies in pandemic search.

References

[1]

Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, Russell Reas, Jiangjiang Yang, Darrin Eide, Kathryn Funk, Rodney Kinney, Ziyang Liu, William. Merrill, Paul Mooney, Dewey A. Murdick, Devvret Rishi, Jerry Sheehan, Zhihong Shen, Brandon Stilson, Alex D. Wade, Kuansan Wang, Christopher Wilhelm, Boya Xie, Douglas M. Raymond, Daniel S. Weld, Oren Etzioni, and Sebastian Kohlmeier. Cord-19: The covid-19 open research dataset. ArXiv, abs/2004.10706, 2020.

Google Scholar

[2]

William Hersh and Ellen Voorhees. TREC Genomics special issue overview. Information Retrieval, 12:1--15, 2009.

Digital Library

Google Scholar

[3]

Ellen M. Voorhees and William Hersh. Overview of the TREC 2012 Medical Records track. In The Twenty-First Text REtrieval Conference Proceedings, 2012.

Google Scholar

[4]

Kirk Roberts, Matthew S. Simpson, Ellen Voorhees, and William Hersh. Overview of the TREC 2015 Clinical Decision Support Track. In The Twenty-Fourth Text REtrieval Conference Proceedings (TREC 2015). NIST Special Publication 500-319., 2015.

Google Scholar

[5]

Kirk Roberts, Dina Demner-Fushman, Ellen M. Voorhees, William R. Hersh, Steven Bedrick, Alexandar Lazar, and Shubham Pant. Overview of the TREC 2017 Precision Medicine Track. In The Twenty-Sixth Text REtrieval Conference Proceedings (TREC 2017). NIST Special Publication 500-324., 2017.

Google Scholar

[6]

Gerard Salton and Chris Buckley. Improving retrieval performance by relevance feedback. Journal of the Association for Information Science and Technology, 41:355--364, 1997.

Google Scholar

[7]

Ellen M. Voorhees. The philosophy of information retrieval evaluation. In Evaluation of Cross-Language Information Retrieval Systems. Proceedings of CLEF 2001, number 2406 in Lecture Notes in Computer Science, pages 355--370, 2002.

Digital Library

Google Scholar

[8]

Ellen M. Voorhees. On building fair and reusable test collections using bandit techniques. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM '18, page 407--416, 2018.

Digital Library

Google Scholar

Cited By

View all

Yang HLi SGonçalves T(2024)Enhancing Biomedical Question Answering with Large Language ModelsInformation10.3390/info1508049415:8(494)Online publication date: 19-Aug-2024
https://doi.org/10.3390/info15080494
Keller JBreuer TSchaer POosterhuis HBast HXiong C(2024)Evaluation of Temporal Change in IR Test CollectionsProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672530(3-13)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672530
Vast MVan Cooten BSoulier LPiwowarski BOosterhuis HBast HXiong C(2024)Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-EncodersProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672528(133-143)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672528
Show More Cited By

TREC-COVID: constructing a pandemic information retrieval test collection
1. Information systems
  1. Information retrieval

Recommendations

Challenges and opportunities in evolving TreC personal health record platform
PervasiveHealth '17: Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare

Electronic Health Records (EHR) have been one of the factors in transforming healthcare and health management by providing electronic access to information recorded on paper charts. However, increasing interest of patients to be actively involved in the ...
The TREC Medical Records Track
BCB'13: Proceedings of the International Conference on Bioinformatics, Computational Biology and Biomedical Informatics

The Text REtrieval Conference (TREC) is a series of annual workshops designed to build the infrastructure for large-scale evaluation of search systems and thus improve the state-of-the-art. Each workshop is organized around a set of "tracks", challenge ...
On the Quality of the TREC-COVID IR Test Collections
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Shared text collections continue to be vital infrastructure for IR research. The COVID-19 pandemic offered an opportunity to create a test collection that captured the rapidly changing information space during a pandemic, and the TREC-COVID effort was ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ACM SIGIR Forum Volume 54, Issue 1

June 2020

148 pages

ISSN:0163-5840

DOI:10.1145/3451964

Issue’s Table of Contents

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 February 2021

Published in SIGIR Volume 54, Issue 1

Check for updates

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

48
Total Citations
View Citations
555
Total Downloads

Downloads (Last 12 months)120
Downloads (Last 6 weeks)15

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yang HLi SGonçalves T(2024)Enhancing Biomedical Question Answering with Large Language ModelsInformation10.3390/info1508049415:8(494)Online publication date: 19-Aug-2024
https://doi.org/10.3390/info15080494
Keller JBreuer TSchaer POosterhuis HBast HXiong C(2024)Evaluation of Temporal Change in IR Test CollectionsProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672530(3-13)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672530
Vast MVan Cooten BSoulier LPiwowarski BOosterhuis HBast HXiong C(2024)Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-EncodersProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672528(133-143)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672528
Breuer TMaistro M(2024)Toward Evaluating the Reproducibility of Information Retrieval Systems with Simulated UsersProceedings of the 2nd ACM Conference on Reproducibility and Replicability10.1145/3641525.3663619(25-29)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3641525.3663619
Fröbe MScells HElstner TAkiki CGienapp LReimer JMacAvaney SStein BHagen MPotthast MHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Resources for Combining Teaching and Research in Information Retrieval CourseworkProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657886(1115-1125)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657886
Chang XMishra DMacdonald CMacAvaney SHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Neural Passage Quality Estimation for Static PruningProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657765(174-185)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657765
Pan KLi JWang WFei HSong HJi WLin JLiu XChua TTang SHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)I3: Intent-Introspective Retrieval Conditioned on InstructionsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657745(1839-1849)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657745
Kroll HKreutz CSackhoff PBalke WKlein MBen-David AJäschke RKelly M(2024)Enriching Simple Keyword Queries for Domain-Aware Narrative RetrievalProceedings of the 2023 ACM/IEEE Joint Conference on Digital Libraries10.1109/JCDL57899.2023.00029(143-154)Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1109/JCDL57899.2023.00029
Yang WZhang S(2024)Information Retrieval and Recommendation Framework Based on ChatGPT2024 3rd International Conference on Sentiment Analysis and Deep Learning (ICSADL)10.1109/ICSADL61749.2024.00068(378-385)Online publication date: 13-Mar-2024
https://doi.org/10.1109/ICSADL61749.2024.00068
Kilicoglu HEnsan FMcInnes BWang L(2024)Semantics-enabled biomedical literature analyticsJournal of Biomedical Informatics10.1016/j.jbi.2024.104588150(104588)Online publication date: Feb-2024
https://doi.org/10.1016/j.jbi.2024.104588
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Challenges and opportunities in evolving TreC personal health record platform

The TREC Medical Records Track

On the Quality of the TREC-COVID IR Test Collections