Article

Free access

A user-centred evaluation of ranking algorithms for interactive query expansion

Author:

Efthimis N. EfthimiadisAuthors Info & Claims

SIGIR '93: Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval

Pages 146 - 159

https://doi.org/10.1145/160688.160710

Published: 01 July 1993 Publication History

Abstract

The evaluation of 6 ranking algorithms for the ranking of terms for query expansion is discussed within the context of an investigation of interactive query expansion and relevance feedback in a real operational environment. The yardstick for the evaluation was provided by the user relevance judgements on the lists of the candidate terms for query expansion. The evaluation focuses on the similarities in the performance of the different algorithms and how the algorithms with similar performance treat terms.

References

[1]

Croft, W.B. and Harper, D.J. (1979) Using probabilistic models of document retrieval without relevance information, journal of Documenlalion 35 (4): 285-295; 1979.

[2]

Efthimiadis, E.N. (1992) Inleractzve Query Expanszon and Relevance Feedback for Document Retrzeval Systems. Ph.D. thesis, City University, London, U.K.

[3]

Harman, D. (1988) Towards interactive query expansion. In: i llh Inlerl~al~onal CoT~ference on Research and Development ~n JR, SIGIR 1988, Grenoble, France, Presses Universitaires de Grenoble, France, pp 321-331.

Digital Library

[4]

Harman, D. (1992) Relevance feedback revisited. In: 15th {nternatzonal Conference on Research and Development in IR, SIGIR 1992, Copenhagen, Danmark, June 21-24, 1992. Belkin, N., Ingwersen, P. L: Pejtersen, A.M., eds. ppl-10.

Digital Library

[5]

Harper, D.J. and van Rijsbergen, C.J. (1978) An evaluation of feedback in document retrieval using cooccurrence data. Journal of Documentatzon 34 (3): 189-216; 1978.

[6]

Ingwersen, P. (1984) A cognitive view of three selected online search facilities. Onhne Revzew, 8(5), 1984, pp.465-492.

[7]

Lehmann, E.L. (1975) Nonparame~mcs: slatzstzcal methods based on ranks. Oakland, CA' ttolden Hay

[8]

Martin, W.A. (1982) Helping the less experienced user. In: 6th IT~lernatzonal Onhne Meelz7~9. London, 7-9 December 1982. Oxford: Learned Information (Europe) Ltd. 1982, pp 67-76.

[9]

McGill, M., Koll, M. and Noreault, T. (1979) An evaluation of factors affecting document ranking by information retrieval systems. Technical report, Syracuse University, School of Information Studies, 1979.

[10]

Peat, H.J., and Willett, P. (1991) The limitations of term co- occurrence data for query expansion in document retrieval systems. JourT~al of the American Soczety for b~format~on SczeT~ce, 42:378-383.

[11]

Porter, M.F. (1982) Implementing a probabilistic information retrieval system. Informatzon Technology: Research and Development, 1982, 1(2), 131-156.

[12]

Porter, M.F. and Galpin, V. (1988) Relevance feedback in a public access catalogue for a research library: Muscat at the Scott Polar Research Institute. Program, 1988, 22(1), 1-20.

Digital Library

[13]

Ro, J.S. (1988) Evaluation of the applicability of ranking algorithms, Pt. I and Pt. II. Journal of the American Soczety for Informatzon Sczence, 39:73-78; 147- 160.

[14]

Robertson, S.E. (1977b) The probability ranking principle in IR. Journal of Documentatzon, 33, 1977, 294-304.

[15]

Robertson, S.E. (1986) On relevance weight estimation and query expansion. Journal of Documentatzon 42 (3): 182-188; 1986.

[16]

Robertson, S.E (1990) On term selection for query expansion. Journal of Documentatzon, 46(4), 359-364.

Digital Library

[17]

Robertson, S.E. and Spar& Jones, K. (1976) Relevance weighting of search terms. Journal of the American Society for Informalzon Sczence, 27(3), 1976, pp.129- 146.

[18]

Robertson, S.E., Bovey, J.D., Thompson, C.L. and Macaskill, M.J. (1986) Weighting, ranking and relevance feedback in a front-end system. Journal of Informatzon Science, 12, 1986, pp.71-75.

Digital Library

[19]

Sager, W.K.H. and Lockemann, P.C. (1976) Classification of ranking algorithms. Internatzonal Forum for Informatzon and Documentatzon, 1:12-25.

[20]

Salton, G., ed. (197t) The SMART retrieval system. Expemmenls zn automalzc document processing. Englewood Cliffs, NJ. Prentice-Hall.

Digital Library

[21]

Salton, G. (1975) Dynamic Informatzon and Lzbrary Processing. Englewood Cliffs: Prentice-Hall.

Digital Library

[22]

Smeaton, A.F. and van Rijsbergen, C.J. (1981) The nearest neighbour problem in information retrieval. An algorithm using upper bounds. Proceedzngs of the Fourth International Conference on Informatzon Storage and Retmeval, Oakland, CA, USA, 31 May- 2 June 1981. In: SIGIR Forum, 16(1): 83-87.

Digital Library

[23]

Smeaton, A.F. and van Rijsbergen, C.J. (1983) The retrieval effects of query expansion on a feedback document retrieval system. The Computer Journal, 26(3), 1983, pp.239-246.

[24]

Sparck Jones, K., ed. (1971) Automatic keyword classification for informatzon retrieval. London: Butterworths.

[25]

Sparck Jones, K. (1979a) Search term relevance weighting given little relevance information. 3ournal of Documentation 35 (1): 30-48; 1979.

[26]

Sparck Jones, K. (1979b) Experiments in relevance weighting of search terms. Informatzon Processing and Management 15 (3): 133-144; 1979.

[27]

Sparck Jones, K. (1988) A look back and a look forward. In: Proceedzngs of the 11th Internatzonal Conference on Research 8J Development zn Information Retrieval. June 13-15, 1988, Grenoble, France. Yves Chiaramella (ed.) ACM Press. 13-29.

Digital Library

[28]

van Rijsbergen, C.J. (1977) A theoretical basis for the use of co-occurrence data in information retrieval. Journal of Documcntatzon, 33:106-119.

[29]

van Rijsbergen, C.J. (1979) Information Retrieval. 2nd edition, London: Butterworth, 1979.

Digital Library

[30]

van Rijsbergen, C.J., Harper, D.J. and Porter, M.F. (1981) The selection of good search terms. Information Processing and Management, 17(2), 1981, pp.77-91.

[31]

Walker, S. and de Vere R. (1990) Improving subject retrzeval zn online catalogues: 2. Relevance feedback and query expansion. British Library Research Paper 72. London: British Library, 1990.

Cited By

Mishra SRyerkerk MLockerman YEis DRzeszotarski J(2024)Teachable Facets: A Framework of Interactive Machine Teaching for Information FilteringProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638289(178-188)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638289
Singh SSuleman H(2022)Investigating User Control to Mitigate Bias When Searching African Historical DataFrom Born-Physical to Born-Virtual: Augmenting Intelligence in Digital Libraries10.1007/978-3-031-21756-2_37(456-463)Online publication date: 7-Dec-2022
https://doi.org/10.1007/978-3-031-21756-2_37
Anjan Kumar KSatish Kumar TKrishna Prasad RRavi Kumar S(2021)Web Object Ranking for Location-Based Web Object SearchAdvances in Smart Communication and Imaging Systems10.1007/978-981-15-9938-5_16(151-165)Online publication date: 14-Apr-2021
https://doi.org/10.1007/978-981-15-9938-5_16
Show More Cited By

Index Terms

A user-centred evaluation of ranking algorithms for interactive query expansion
1. Information systems
  1. Information retrieval
    1. Document representation
    2. Information retrieval query processing
2. Theory of computation
  1. Semantics and reasoning
    1. Program reasoning
      1. Abstraction

Recommendations

Re-ranking search results using query logs
CIKM '06: Proceedings of the 15th ACM international conference on Information and knowledge management

This work addresses two common problems in search, frequently occurring with underspecified user queries: the top-ranked results for such queries may not contain documents relevant to the user's search intent, and fresh and relevant pages may not get ...
A polyrepresentational approach to interactive query expansion
JCDL '09: Proceedings of the 9th ACM/IEEE-CS joint conference on Digital libraries

Interactive Query Expansion (IQE) presents suggested terms to the user during their search to enable better Information Retrieval (IR). However, IQE terms are poorly used, and tend to lack information meaningful to the user. The lack of cognitive and ...
Query Expansion by Mining User Logs

Queries to search engines on the Web are usually short. They do not provide sufficient information for an effective selection of relevant documents. Previous research has proposed the utilization of query expansion to deal with this problem. However, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '93: Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval

July 1993

361 pages

ISBN:0897916050

DOI:10.1145/160688

Copyright © 1993 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 1993

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

SIGIR93

Sponsor:

SIGIR

SIGIR93: 16th International ACM/SIGIR '93 Conference on Research and Development in Information Retrieval

June 27 - July 1, 1993

Pennsylvania, Pittsburgh, USA

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
903
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)11

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mishra SRyerkerk MLockerman YEis DRzeszotarski J(2024)Teachable Facets: A Framework of Interactive Machine Teaching for Information FilteringProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638289(178-188)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638289
Singh SSuleman H(2022)Investigating User Control to Mitigate Bias When Searching African Historical DataFrom Born-Physical to Born-Virtual: Augmenting Intelligence in Digital Libraries10.1007/978-3-031-21756-2_37(456-463)Online publication date: 7-Dec-2022
https://doi.org/10.1007/978-3-031-21756-2_37
Anjan Kumar KSatish Kumar TKrishna Prasad RRavi Kumar S(2021)Web Object Ranking for Location-Based Web Object SearchAdvances in Smart Communication and Imaging Systems10.1007/978-981-15-9938-5_16(151-165)Online publication date: 14-Apr-2021
https://doi.org/10.1007/978-981-15-9938-5_16
Shekarpour SMarx EAuer SSheth ASingh SMarkovitch S(2017)RQUERYProceedings of the Thirty-First AAAI Conference on Artificial Intelligence10.5555/3298023.3298140(3936-3943)Online publication date: 4-Feb-2017
https://dl.acm.org/doi/10.5555/3298023.3298140
Bharat KHenzinger M(2017)Improved Algorithms for Topic Distillation in a Hyperlinked EnvironmentACM SIGIR Forum10.1145/3130348.313036751:2(194-201)Online publication date: 2-Aug-2017
https://dl.acm.org/doi/10.1145/3130348.3130367
Li JRao H(2017)Twitter as a Rapid Response News Service: An Exploration in the Context of the 2008 China EarthquakeTHE ELECTRONIC JOURNAL OF INFORMATION SYSTEMS IN DEVELOPING COUNTRIES10.1002/j.1681-4835.2010.tb00300.x42:1(1-22)Online publication date: 5-Dec-2017
https://doi.org/10.1002/j.1681-4835.2010.tb00300.x
Durao FBayyapu KXu GDolog PLage R(2014)Expanding user's query with tag-neighbors for effective medical information retrievalMultimedia Tools and Applications10.1007/s11042-012-1316-571:2(905-929)Online publication date: 1-Jul-2014
https://dl.acm.org/doi/10.1007/s11042-012-1316-5
Durao FBayyapu KXu GDolog PLage R(2013)Medical Information Retrieval Enhanced with User’s Query Expanded with Tag-NeighborsHandbook of Medical and Healthcare Technologies10.1007/978-1-4614-8495-0_2(17-40)Online publication date: 21-Nov-2013
https://doi.org/10.1007/978-1-4614-8495-0_2
Carpineto CRomano G(2012)A Survey of Automatic Query Expansion in Information RetrievalACM Computing Surveys10.1145/2071389.207139044:1(1-50)Online publication date: 1-Jan-2012
https://dl.acm.org/doi/10.1145/2071389.2071390
Durao FBayyapu KXu GDolog PLage R(2011)Using Tag-Neighbors for Query Expansion in Medical Information RetrievalProceedings of the 2011 International Conference on Information Science and Applications10.1109/ICISA.2011.5772324(1-9)Online publication date: 26-Apr-2011
https://dl.acm.org/doi/10.1109/ICISA.2011.5772324
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents