Article

User-Oriented Evaluation Methods for Interactive Web Search Interfaces

Authors:

Xue Dong YangAuthors Info & Claims

WI-IATW '07: Proceedings of the 2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops

Pages 239 - 243

Published: 02 November 2007 Publication History

Abstract

Although significant efforts have been devoted to the study and evaluation of information retrieval systems from an algorithmic perspective, far less work has been performed on the evaluation of these systems from the user's perspective. This is certainly the case for Web information retrieval, where the major search engines continue to utilise interfaces that have not changed substantially since their introduction. One of the challenges in developing new Web search interfaces is the evaluation of these systems in comparison to one another, as well as in comparison to the popular Web search engines. In this paper, we highlight some of the methods used in the literature for evaluating Web search systems, and present a summary of the methods that we have found to be effective in dealing with the challenges of evaluating intelligent and interactive Web search interfaces.

References

[1]

E. Berenci, C. Carpineto, V. Giannini, and S. Mizzaro. Effectiveness of keyword-based display and selection of retrieval results for interactive searches. International Journal on Digital Libraries, 3(3):249-260, 2000.

[2]

H. Greisdorf and A. Spink. Median measure: an approach to IR systems evaluation. Information Processing and Management, 37(6):843-857, 2001.

Digital Library

[3]

M. A. Hearst and J. O. Pedersen. Reexamining the cluster hypothesis: Scatter/gather on retrieval results. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, 1996.

Digital Library

[4]

O. Hoeber and X. D. Yang. A comparative user study of web search interfaces: HotMap, Concept Highlighter, and Google. In Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, 2006.

Digital Library

[5]

O. Hoeber and X. D. Yang. Interactive web information retrieval using WordBars. In Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, 2006.

Digital Library

[6]

O. Hoeber and X. D. Yang. The visual exploration of web search results using HotMap. In Proceedings of the International Conference on Information Visualization, 2006.

Digital Library

[7]

O. Hoeber and X. D. Yang. Evaluating wordbars in exploratory web search scenarios. Information Processing and Management, 2007 (to appear).

Digital Library

[8]

P. J.-H. Hu, P.-C. Ma, and P. Y. Chau. Evaluation of user interface designs for information retrieval systems: a computer-based experiment. Decision Support Systems, 27(1):125-143, 1999.

Digital Library

[9]

B. J. Jansen and U. Pooch. A review of Web searching studies and a framework for future research. Journal of the American Society for Information Science and Technology, 52(3):235-246, 2001.

Digital Library

[10]

J. Nielsen. Usability Engineering. Academic Press, 1993.

Digital Library

[11]

S. Nowicki. Student vs. search engine: Undergraduates rank results for relevance. Libraries and the Academy, 3(3):503- 515, 2003.

[12]

J. Preece, Y. Rogers, and H. Sharp. Interaction Design: beyond human-computer interaction. Johyn Wiley & Sons, 2002.

Digital Library

[13]

M. B. Rosson and J. M. Carroll. Usability Engineering: scenario-based development of human-computer interaction. Morgan Kaufmann, 2002.

Digital Library

[14]

B. Shneiderman. Designing the User Interface. Addison-Wesley, 1998.

Digital Library

[15]

A. Spink. A user-centered approach to evaluating human interaction with web search engines: an exploratory study. Information Processing and Management, 38(3):401-426, 2002.

Digital Library

[16]

A. Spink, D. Wolfram, B. J. Jansen, and T. Saracevic. Searching the Web: The public and their queries. Journal of the American Society for Information Science and Technology, 52(3):226-234, 2001.

[17]

L. T. Su. A comprehensive and systematic model of user evaluation of Web search enginges: II. an evaluation by undergraduates. Journal of the American Society for Information Science and Technology, 54(13):1193-1223, 2003.

Digital Library

[18]

A. G. Sutcliffe, M. Ennis, and J. Hu. Evaluating the effectiveness of visual user interfaces for information retrieval. International Journal on Human-Computer Studies, 53(5):741-763, 2000.

Digital Library

[19]

L. Vaughan. New measures for search engine evaluation proposed and tested. Information Processing and Management, 40(4):677-691, 2004.

Digital Library

[20]

O. Zamir and O. Etzioni. Grouper: a dynamic clustering interface to web search results. Computer Networks: The International Journal of Computer and Telecommunications Networking, 31(11-16):1361-1374, 1999.

Digital Library

Cited By

Elbedweihy KWrigley SClough PCiravegna F(2015)An overview of semantic search evaluation initiativesWeb Semantics: Science, Services and Agents on the World Wide Web10.1016/j.websem.2014.10.00130:C(82-105)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.1016/j.websem.2014.10.001
Castano SFerrara AMontanelli S(2012)Structured data clouding across multiple websInformation Systems10.1016/j.is.2011.10.00337:4(352-371)Online publication date: 1-Jun-2012
https://dl.acm.org/doi/10.1016/j.is.2011.10.003

Index Terms

User-Oriented Evaluation Methods for Interactive Web Search Interfaces

Recommendations

User Evaluation Methods for Visual Web Search Interfaces
IV '09: Proceedings of the 2009 13th International Conference Information Visualisation

In recent years, numerous visual Web search interfaces have been developed in the research community. However, the user evaluations of these interfaces have been performed using a wide range of methods, making it difficult to compare and verify the ...
An overview of Web search evaluation methods

Web search evaluation is the process of measuring the effectiveness of a Web search system. Such an evaluation helps in identifying the most effective one and helps the users to find the required information with less effort. Web search systems have ...
A Comparative User Study of Web Search Interfaces: HotMap, Concept Highlighter, and Google
WI '06: Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence

Users of traditional web search engines commonly find it difficult to evaluate the results of their web searches. We suggest the use of information visualization and interactive visual manipulation as methods for improving the ability of users to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WI-IATW '07: Proceedings of the 2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops

November 2007

513 pages

ISBN:0769530281

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence

Publisher

IEEE Computer Society

United States

Publication History

Published: 02 November 2007

Check for updates

Author Tag

evaluation methodsWeb searchhuman-computer interaction

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
167
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Elbedweihy KWrigley SClough PCiravegna F(2015)An overview of semantic search evaluation initiativesWeb Semantics: Science, Services and Agents on the World Wide Web10.1016/j.websem.2014.10.00130:C(82-105)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.1016/j.websem.2014.10.001
Castano SFerrara AMontanelli S(2012)Structured data clouding across multiple websInformation Systems10.1016/j.is.2011.10.00337:4(352-371)Online publication date: 1-Jun-2012
https://dl.acm.org/doi/10.1016/j.is.2011.10.003

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents