research-article

Why People Search for Images using Web Search Engines

Authors:

Maarten de Rijke,

Shaoping MaAuthors Info & Claims

WSDM '18: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

Pages 655 - 663

https://doi.org/10.1145/3159652.3159686

Published: 02 February 2018 Publication History

Abstract

What are the intents or goals behind human interactions with image search engines? Knowing why people search for images is of major concern to Web image search engines because user satisfaction may vary as intent varies. Previous analyses of image search behavior have mostly been query-based, focusing on what images people search for, rather than intent-based, that is, why people search for images. To date, there is no thorough investigation of how different image search intents affect users» search behavior. In this paper, we address the following questions: (1)Why do people search for images in text-based Web image search systems? (2)How does image search behavior change with user intent? (3)Can we predict user intent effectively from interactions during the early stages of a search session? To this end, we conduct both a lab-based user study and a commercial search log analysis. We show that user intents in image search can be grouped into three classes: Explore/Learn, Entertain, and Locate/Acquire. Our lab-based user study reveals different user behavior patterns under these three intents, such as first click time, query reformulation, dwell time and mouse movement on the result page. Based on user interaction features during the early stages of an image search session, that is, before mouse scroll, we develop an intent classifier that is able to achieve promising results for classifying intents into our three intent classes. Given that all features can be obtained online and unobtrusively, the predicted intents can provide guidance for choosing ranking methods immediately after scrolling.

References

[1]

Eugene Agichtein, Eric Brill, and Susan Dumais. 2006. Improving web search ranking by incorporating user behavior information. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 19--26.

Digital Library

[2]

Icek Ajzen. 2002. Perceived behavioral control, self-efficacy, locus of control, and the theory of planned behavior. Journal of Applied Social Psychology 32 (2002), 665--683.

[3]

Paul André, Edward Cutrell, Desney Tan, and Greg Smith. 2009. Designing novel image search interfaces by understanding unique characteristics and usage. Human-Computer Interaction--INTERACT 2009 (2009), 340--353.

Digital Library

[4]

Alexey Borisov, Ilya Markov, Maarten de Rijke, and Pavel Serdyukov. 2016. A context-aware time model for web search. In SIGIR 2016: 39th international ACM SIGIR conference on Research and development in information retrieval. ACM, 205--214.

Digital Library

[5]

Andrei Broder. 2002. A taxonomy of web search. In ACM Sigir forum, Vol. 36. ACM, 3--10.

Digital Library

[6]

Huanhuan Cao, Daxin Jiang, Jian Pei, Qi He, Zhen Liao, Enhong Chen, and Hang Li. 2008. Context-aware query suggestion by mining click-through and session data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 875--883.

Digital Library

[7]

Justin Cheng, Caroline Lo, and Jure Leskovec. 2017. Predicting intent using activity logs: How goal specificity and temporal range affect user behavior. In Proceedings of the 26th International Conference on World Wide Web Companion. International World Wide Web Conferences Steering Committee, 593--601.

Digital Library

[8]

I CNNIC. 2016. The 38th Statistical report on internet development in China. China Internet Network Information Center (CNNIC), China (2016).

[9]

Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. 2008. An experimental comparison of click position-bias models. In Proceedings of the 2008 international conference on web search and data mining. ACM, 87--94.

Digital Library

[10]

Honghua Kathy Dai, Lingzhi Zhao, Zaiqing Nie, Ji-Rong Wen, Lee Wang, and Ying Li. 2006. Detecting online commercial intention (OCI). In Proceedings of the 15th international conference on World Wide Web. ACM, 829--837.

Digital Library

[11]

Zhicheng Dou, Ruihua Song, and Ji-Rong Wen. 2007. A large-scale evaluation and analysis of personalized search strategies. In Proceedings of the 16th international conference on World Wide Web. ACM, 581--590.

Digital Library

[12]

Joseph L. Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin 76, 5 (1971), 378--382.

[13]

Abby Goodrum and Amanda Spink. 1999. Visual information seeking: A study of image queries on the world wide web. In Proceedings of the ASIS Annual Meeting, Vol. 36. ERIC, 665--74.

[14]

Sha Hu, Zhicheng Dou, Xiaojie Wang, Tetsuya Sakai, and Ji-Rong Wen. 2015. Search result diversification based on hierarchical intents. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 63--72.

Digital Library

[15]

Xian-Sheng Hua, Linjun Yang, Jingdong Wang, Jing Wang, Ming Ye, Kuansan Wang, Yong Rui, and Jin Li. 2013. Clickage: Towards bridging semantic and intent gaps via mining click logs of search engines. In Proceedings of the 21st ACM international conference on Multimedia. ACM, 243--252.

Digital Library

[16]

Bouke Huurnink, Laura Hollink, Wietske Van Den Heuvel, and Maarten de Rijke. 2010. Search behavior of media professionals at an audiovisual archive: A transaction log analysis. Journal of the Association for Information Science and Technology 61, 6 (2010), 1180--1197.

Digital Library

[17]

Vidit Jain and Manik Varma. 2011. Learning to re-rank: query-dependent image re-ranking using click data. In Proceedings of the 20th international conference on World wide web. ACM, 277--286.

Digital Library

[18]

Bernard J. Jansen. 2008. Searching for digital images on the web. Journal of Documentation 64, 1 (2008), 81--101.

[19]

Bernard J. Jansen, Amanda Spink, and Bhuva Narayan. 2007. Query modifications patterns during web searching. In Information Technology, 2007. ITNG'07. Fourth International Conference on. IEEE, 439--444.

Digital Library

[20]

Bernard J. Jansen, Amanda Spink, and Tefko Saracevic. 2000. Real life, real users, and real needs: A study and analysis of user queries on the web. Information Processing & Management 36, 2 (2000), 207--227.

Digital Library

[21]

Jyun-Yu Jiang, Yen-Yu Ke, Pao-Yu Chien, and Pu-Jen Cheng. 2014. Learning user reformulation behavior for query auto-completion. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. ACM, 445--454.

Digital Library

[22]

Diane Kelly, Jaime Arguello, Ashlee Edwards, and Wan-ching Wu. 2015. Development and evaluation of search tasks for IIR experiments using a cognitive complexity framework. In Proceedings of the 2015 International Conference on The Theory of Information Retrieval. ACM, 101--110.

Digital Library

[23]

Christoph Kofler and Mathias Lux. 2009. Dynamic presentation adaptation based on user intent classification. In Proceedings of the 17th ACM international conference on Multimedia. ACM, 1117--1118.

Digital Library

[24]

Christoph Kofler and Mathias Lux. 2009. An exploratory study on the explicitness of user intentions in digital photo retrieval. In Proceedings of the 9th International Conference on Knowledge Management and Knowledge Technologies.

[25]

J. Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics 33, 1 (1977), 159--174.

[26]

Yanen Li, Anlei Dong, Hongning Wang, Hongbo Deng, Yi Chang, and ChengXiang Zhai. 2014. A two-dimensional click model for query auto-completion. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. ACM, 455--464.

Digital Library

[27]

Jingjing Liu, Michael J Cole, Chang Liu, Ralf Bierig, Jacek Gwizdka, Nicholas J Belkin, Jun Zhang, and Xiangmin Zhang. 2010. Search behaviors in different task types. In Proceedings of the 10th annual joint conference on Digital libraries. ACM, 69--78.

Digital Library

[28]

Yiqun Liu, Xiaohui Xie, Chao Wang, Jian-Yun Nie, Min Zhang, and Shaoping Ma. 2016. Time-aware click model. ACM Transactions on Information Systems (TOIS) 35, 3 (2016), Article 16.

Digital Library

[29]

Mathias Lux, Christoph Kofler, and Oge Marques. 2010. A classification scheme for user intentions in image search. In CHI'10 Extended Abstracts on Human Factors in Computing Systems. ACM, 3913--3918.

Digital Library

[30]

Llew Mason, Jonathan Baxter, Peter L Bartlett, and Marcus R Frean. 2000. Boosting algorithms as gradient descent. In Advances in neural information processing systems. 512--518.

Digital Library

[31]

Neil O'Hare, Paloma de Juan, Rossano Schifanella, Yunlong He, Dawei Yin, and Yi Chang. 2016. Leveraging user interaction signals for web image search. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 559--568.

Digital Library

[32]

Bing Pan, Helene A. Hembrooke, Geri K. Gay, Laura A. Granka, Matthew K. Feusner, and Jill K. Newman. 2004. The determinants of web page viewing behavior: an eye-tracking study. In Proceedings of the 2004 symposium on Eye tracking research & applications. ACM, 147--154.

Digital Library

[33]

Jaimie Y. Park, Neil O'Hare, Rossano Schifanella, Alejandro Jaimes, and Chin-Wan Chung. 2015. A large-scale study of user image search behavior on the web. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 985--994.

Digital Library

[34]

Hsiao-Tieh Pu. 2005. A comparative analysis of web image and textual queries. Online Information Review 29, 5 (2005), 457--467.

[35]

Davood Rafiei, Krishna Bharat, and Anand Shukla. 2010. Diversifying web search results. In Proceedings of the 19th international conference on World wide web. ACM, 781--790.

Digital Library

[36]

Miriam Redi, Frank Z. Liu, and Neil O'Hare. 2017. Bridging the aesthetic gap: The wild beauty of web imagery. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval. ACM, 242--250.

Digital Library

[37]

Daniel E. Rose and Danny Levinson. 2004. Understanding user goals in web search. In Proceedings of the 13th international conference on World Wide Web. ACM, 13--19.

Digital Library

[38]

Daniel M Russell, Diane Tang, Melanie Kellar, and Robin Jeffries. 2009. Task behaviors during web search: The dificulty of assigning labels. In System Sciences, 2009. HICSS'09. 42nd Hawaii International Conference on. IEEE, 1--5.

Digital Library

[39]

Mohammad Soleymani, Michael Riegler, and Pål Halvorsen. 2017. Multimodal analysis of image search intent: Intent recognition in image search from user behavior and visual content. In ACM International Conference on Multimedia Retrieval. 251--259.

Digital Library

[40]

Bilyana Taneva, Mouna Kacimi, and Gerhard Weikum. 2010. Gathering and ranking photos of named entities with high precision, high recall, and diversity. In Proceedings of the third ACM international conference on Web search and data mining. ACM, 431--440.

Digital Library

[41]

Dian Tjondronegoro, Amanda Spink, and Bernard J. Jansen. 2009. A study and comparison of multimedia Web searching: 1997--2006. Journal of the American Society for Information Science and Technology 60, 9 (2009), 1756--1768.

Digital Library

[42]

Reinier H. van Leuken, Lluis Garcia, Ximena Olivares, and Roelof van Zwol. 2009. Visual diversification of image search results. In Proceedings of the 18th international conference on World wide web. ACM, 341--350.

Digital Library

[43]

Kuansan Wang, Nikolas Gloy, and Xiaolong Li. 2010. Inferring search behaviors using partially observable markov (pom) model. In Proceedings of the third ACM international conference on Web search and data mining. ACM, 211--220.

Digital Library

[44]

Xiaohui Xie, Yiqun Liu, Xiaochuan Wang, Meng Wang, Zhijing Wu, Yingying Wu, Min Zhang, and Shaoping Ma. 2017. Investigating examination behavior of image search users. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM.

Digital Library

[45]

Danqing Xu, Yiqun Liu, Min Zhang, Shaoping Ma, and Liyun Ru. 2012. Incorporating revisiting behaviors into click models. In Proceedings of the fifth ACM international conference on Web search and data mining. ACM, 303--312.

Digital Library

[46]

Jinxi Xu and W. Bruce Croft. 1996. Query expansion using local and global document analysis. In Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 4--11.

Digital Library

[47]

Zheng-Jun Zha, Linjun Yang, Tao Mei, Meng Wang, Zengfu Wang, Tat-Seng Chua, and Xian-Sheng Hua. 2010. Visual query suggestion: Towards capturing user intent in internet image search. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 6, 3 (2010), 13.

Digital Library

Cited By

Hajizadeh Saffar ASitbon LHoogstrate MAbbas ARoomkham SMiller D(2024)Human and Large Language Model Intent Detection in Image-Based Self-Expression of People with Intellectual DisabilityProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638347(199-208)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638347
Sharma DSingh BAgarwal SGarg LKim CJung K(2023)A Survey of Detection and Mitigation for Fake Images on Social Media PlatformsApplied Sciences10.3390/app13191098013:19(10980)Online publication date: 5-Oct-2023
https://doi.org/10.3390/app131910980
Shao YLi HWu YLiu YAi QMao JMa YMa S(2023)An Intent Taxonomy of Legal Case RetrievalACM Transactions on Information Systems10.1145/362609342:2(1-27)Online publication date: 29-Sep-2023
https://dl.acm.org/doi/10.1145/3626093
Show More Cited By

Index Terms

Why People Search for Images using Web Search Engines
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing
      1. Query intent
      2. Query log analysis
    2. Specialized information retrieval
      1. Multimedia and multimodal retrieval
        Image search

Recommendations

User Behavior Modeling for Web Image Search
WSDM '19: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining

Web-based image search engines differ from Web search engines greatly. The intents or goals behind human interactions with image search engines are different. In image search, users mainly search images instead of Web pages or online services. It is ...
Overlap Among Major Web Search Engines
ITNG '06: Proceedings of the Third International Conference on Information Technology: New Generations

Our study examined the overlap among results retrieved by three major Web search engines for a large set of more than 10,316 queries. Previous smaller studies have discussed the lack of overlap in results returned by Web search engines for the same ...
Mining query subtopics from search log data
SIGIR '12: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval

Most queries in web search are ambiguous and multifaceted. Identifying the major senses and facets of queries from search log data, referred to as query subtopic mining in this paper, is a very important issue in web search. Through search log analysis, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '18: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

February 2018

821 pages

ISBN:9781450355810

DOI:10.1145/3159652

General Chairs:
Yi Chang
Jilin University, Huawei Inc.
,
Chengxiang Zhai
University of Illinois Urbana-Champaign
,
Program Chairs:
Yan Liu
University of Southern California
,
Yoelle Maarek
Amazon

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 February 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

the Netherlands Organisation for Scientific Research (NWO)
Natural Science Foundation of China
National Key Basic Research Program
the European Community's Seventh Framework Programme (FP7/2007-2013)

Conference

WSDM 2018

Sponsor:

WSDM 2018: The Eleventh ACM International Conference on Web Search and Data Mining

February 5 - 9, 2018

CA, Marina Del Rey, USA

Acceptance Rates

WSDM '18 Paper Acceptance Rate 81 of 514 submissions, 16%;

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
369
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)5

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hajizadeh Saffar ASitbon LHoogstrate MAbbas ARoomkham SMiller D(2024)Human and Large Language Model Intent Detection in Image-Based Self-Expression of People with Intellectual DisabilityProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638347(199-208)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638347
Sharma DSingh BAgarwal SGarg LKim CJung K(2023)A Survey of Detection and Mitigation for Fake Images on Social Media PlatformsApplied Sciences10.3390/app13191098013:19(10980)Online publication date: 5-Oct-2023
https://doi.org/10.3390/app131910980
Shao YLi HWu YLiu YAi QMao JMa YMa S(2023)An Intent Taxonomy of Legal Case RetrievalACM Transactions on Information Systems10.1145/362609342:2(1-27)Online publication date: 29-Sep-2023
https://dl.acm.org/doi/10.1145/3626093
Zou JAliannejadi MKanoulas EPera MLiu Y(2023)Users Meet Clarifying Questions: Toward a Better Understanding of User Interactions for Search ClarificationACM Transactions on Information Systems10.1145/352411041:1(1-25)Online publication date: 9-Jan-2023
https://dl.acm.org/doi/10.1145/3524110
Sitbon LBrereton MBircanin F(2023)Reframing search and recommendation as opportunities for communication for people with intellectual disabilityHuman–Computer Interaction10.1080/07370024.2023.2247394(1-19)Online publication date: 28-Aug-2023
https://doi.org/10.1080/07370024.2023.2247394
Zou JSun ALong CAliannejadi MKanoulas E(2023)Asking Clarifying Questions: To benefit or to disturb users in Web search?Information Processing & Management10.1016/j.ipm.2022.10317660:2(103176)Online publication date: Mar-2023
https://doi.org/10.1016/j.ipm.2022.103176
Ulloa RRichter AMakhortykh MUrman AKacperski C(2022)Representativeness and face-ism: Gender bias in image searchNew Media & Society10.1177/1461444822110069926:6(3541-3567)Online publication date: 19-Jun-2022
https://doi.org/10.1177/14614448221100699
Wei ZPaliyawan PThawonmas R(2022)A Card Game for Collecting Human-Perceived Similarity Data of Artwork ImagesIEEE Access10.1109/ACCESS.2022.314272510(8103-8111)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3142725
Kumpulainen SRuotsalainen H(2022)Searching Wartime Photograph Archive for Serious Leisure PurposesLinking Theory and Practice of Digital Libraries10.1007/978-3-031-16802-4_7(81-92)Online publication date: 20-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-16802-4_7
Daghighi AChen J(2021)Robustness Comparison of Scheduling Algorithms in MapReduce FrameworkIntelligent Computing10.1007/978-3-030-80119-9_30(494-508)Online publication date: 13-Jul-2021
https://doi.org/10.1007/978-3-030-80119-9_30
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents