research-article

Leveraging User Interaction Signals for Web Image Search

Authors:

Paloma de Juan,

Rossano Schifanella,

Yi ChangAuthors Info & Claims

SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

Pages 559 - 568

https://doi.org/10.1145/2911451.2911532

Published: 07 July 2016 Publication History

Abstract

User interfaces for web image search engine results differ significantly from interfaces for traditional (text) web search results, supporting a richer interaction. In particular, users can see an enlarged image preview by hovering over a result image, and an `image preview' page allows users to browse further enlarged versions of the results, and to click-through to the referral page where the image is embedded. No existing work investigates the utility of these interactions as implicit relevance feedback for improving search ranking, beyond using clicks on images displayed in the search results page. In this paper we propose a number of implicit relevance feedback features based on these additional interactions: hover-through rate, 'converted-hover' rate, referral page click through, and a number of dwell time features. Also, since images are never self-contained, but always embedded in a referral page, we posit that clicks on other images that are embedded on the same referral webpage as a given image can carry useful relevance information about that image. We also posit that query-independent versions of implicit feedback features, while not expected to capture topical relevance, will carry feedback about the quality or attractiveness of images, an important dimension of relevance for web image search. In an extensive set of ranking experiments in a learning to rank framework, using a large annotated corpus, the proposed features give statistically significant gains of over 2% compared to a state of the art baseline that uses standard click features.

References

[1]

E. Agichtein, E. Brill, and S. Dumais. Improving web search ranking by incorporating user behavior information. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '06, pages 19--26, New York, NY, USA, 2006. ACM.

Digital Library

[2]

P. Andre, E. Cutrell, D. S. Tan, and G. Smith. Designing novel image search interfaces by understanding unique characteristics and usage. In INTERACT 2009, pages 340--353. Springer-Verlag, 2009.

Digital Library

[3]

M. Bilenko and R. W. White. Mining the search trails of surfing crowds: Identifying relevant websites from user activity. In Proceedings of the 17th International Conference on World Wide Web, WWW '08, pages 51--60, New York, NY, USA, 2008. ACM.

Digital Library

[4]

N. Craswell and M. Szummer. Random walks on the click graph. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '07, pages 239--246, New York, NY, USA, 2007. ACM.

Digital Library

[5]

N. Craswell, O. Zoeter, M. Taylor, and B. Ramsey. An experimental comparison of click position-bias models. In Proceedings of the 2008 International Conference on Web Search and Data Mining, WSDM '08, pages 87--94, New York, NY, USA, 2008. ACM.

Digital Library

[6]

J. H. Friedman. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29:1189--1232, 2000.

[7]

B. Geng, L. Yang, C. Xu, X.-S. Hua, and S. Li. The role of attractiveness in web image search. In Proceedings of the 19th ACM International Conference on Multimedia, MM '11, pages 63--72, New York, NY, USA, 2011. ACM.

Digital Library

[8]

J. Huang, T. Lin, and R. W. White. No search result left behind: branching behavior with browser tabs. In WSDM, 2012.

Digital Library

[9]

V. Jain and M. Varma. Learning to re-rank: Query-dependent image re-ranking using click data. In Proceedings of the 20th International Conference on World Wide Web, WWW '11, pages 277--286, New York, NY, USA, 2011. ACM.

Digital Library

[10]

B. J. Jansen, A. Spink, and J. O. Pedersen. The effect of specialized multimedia collections on web searching. J. Web Eng., 3(3--4):182--199, 2004.

Digital Library

[11]

K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM TOIS.

[12]

T. Joachims. Optimizing search engines using clickthrough data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '02, pages 133--142, New York, NY, USA, 2002. ACM.

Digital Library

[13]

T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '05, pages 154--161, New York, NY, USA, 2005. ACM.

Digital Library

[14]

D. Kelly and N. J. Belkin. Display time as implicit feedback: Understanding task effects. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '04, pages 377--384, New York, NY, USA, 2004. ACM.

Digital Library

[15]

Y. Kim, A. Hassan, R. W. White, and I. Zitouni. Modeling dwell time to predict click-level satisfaction. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining, WSDM '14, pages 193--202, New York, NY, USA, 2014. ACM.

Digital Library

[16]

C. Liu, J. Liu, N. Belkin, M. Cole, and J. Gwizdka. Using dwell time as an implicit measure of usefulness in different task types. Proceedings of the American Society for Information Science and Technology, 2011.

[17]

S. Maniu, N. O'Hare, L. Aiello, L. Chiarandini, and A. Jaimes. Search behaviour on photo sharing platforms. In ICME 2013, pages 1--6, 2013.

[18]

D. Metzler and W. B. Croft. A markov random field model for term dependencies. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '05, pages 472--479, New York, NY, USA, 2005. ACM.

Digital Library

[19]

J. Y. Park, N. O'Hare, R. Schifanella, A. Jaimes, and C.-W. Chung. A large-scale study of user image search behavior on the web. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, pages 985--994, New York, NY, USA, 2015. ACM.

Digital Library

[20]

S. Robertson and H. Zaragoza. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3(4):333--389, Apr. 2009.

Digital Library

[21]

G. Smith and H. Ashman. Evaluating implicit judgements from Image search interactions. In Proceedings of the WebSci'09: Society On-Line, Mar. 2009.

[22]

Y. Song, H. Ma, H. Wang, and K. Wang. Exploring and exploiting user search behavior on mobile and tablet devices to improve search relevance. In Proceedings of the 22Nd International Conference on World Wide Web, WWW '13, pages 1201--1212, Republic and Canton of Geneva, Switzerland, 2013. International World Wide Web Conferences Steering Committee.

Digital Library

[23]

M. Trevisiol, L. Chiarandini, L. M. Aiello, and A. Jaimes. Image ranking based on user browsing behavior. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '12, pages 445--454, New York, NY, USA, 2012. ACM.

Digital Library

[24]

T. Tsikrika, C. Diou, A. P. de Vries, and A. Delopoulos. Image annotation using clickthrough data. In Proceedings of the ACM International Conference on Image and Video Retrieval, CIVR '09, pages 14:1--14:8, New York, NY, USA, 2009. ACM.

Digital Library

[25]

C. Wang, Y. Liu, M. Zhang, S. Ma, M. Zheng, J. Qian, and K. Zhang. Incorporating vertical results into search click models. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '13, pages 503--512, New York, NY, USA, 2013. ACM.

Digital Library

[26]

R. W. White and J. Huang. Assessing the scenic route: Measuring the value of search trails in web logs. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '10, pages 587--594, New York, NY, USA, 2010. ACM.

Digital Library

[27]

S. Xu, H. Jiang, and F. C. M. Lau. Mining user dwell time for personalized web search re-ranking. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence - Volume Volume Three, IJCAI'11, pages 2367--2372. AAAI Press, 2011.

Digital Library

[28]

S. Xu, Y. Zhu, H. Jiang, and F. C. M. Lau. A user-oriented webpage ranking algorithm based on user attention time. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, July 13--17, 2008, pages 1255--1260, 2008.

Digital Library

[29]

X. Yi, L. Hong, E. Zhong, N. N. Liu, and S. Rajan. Beyond clicks: Dwell time for personalization. In Proceedings of the 8th ACM Conference on Recommender Systems, RecSys '14, pages 113--120, New York, NY, USA, 2014. ACM.

Digital Library

[30]

J. Yu, D. Tao, M. Wang, and Y. Rui. Learning to rank using user clicks and visual features for image retrieval. IEEE Trans. Cybernetics, 45(4):767--779, 2015.

[31]

Z. Zheng, K. Chen, G. Sun, and H. Zha. A regression framework for learning ranking functions using relative relevance judgments. In Proceedings of SIGIR '07.

Digital Library

Cited By

Rehse JAbb LBerg GBormann CKampik TWarmuth C(2024)User Behavior MiningBusiness & Information Systems Engineering10.1007/s12599-023-00848-1Online publication date: 5-Jan-2024
https://doi.org/10.1007/s12599-023-00848-1
Zhang RXie XMao JLiu YZhang MMa S(2021)Constructing a Comparison-based Click Model for Web SearchProceedings of the Web Conference 202110.1145/3442381.3449918(270-283)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3449918
Xie XMao JLiu Yde Rijke MHuang JChang YCheng XKamps JMurdock VWen JLiu Y(2020)Modeling User Behavior for Vertical Search: Images, Apps and ProductsProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401423(2440-2443)Online publication date: 25-Jul-2020
https://dl.acm.org/doi/10.1145/3397271.3401423
Show More Cited By

Index Terms

Leveraging User Interaction Signals for Web Image Search
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking
    2. Specialized information retrieval
      1. Multimedia and multimodal retrieval
        Image search
  2. World Wide Web
    1. Web searching and information discovery
      1. Content ranking
      2. Web search engines

Recommendations

Constructing an Interaction Behavior Model for Web Image Search
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

User interaction behavior is a valuable source of implicit relevance feedback. In Web image search a different type of search result presentation is used than in general Web search, which leads to different interaction mechanisms and user behavior. For ...
Why People Search for Images using Web Search Engines
WSDM '18: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

What are the intents or goals behind human interactions with image search engines? Knowing why people search for images is of major concern to Web image search engines because user satisfaction may vary as intent varies. Previous analyses of image ...
User Behavior Modeling for Web Image Search
WSDM '19: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining

Web-based image search engines differ from Web search engines greatly. The intents or goals behind human interactions with image search engines are different. In image search, users mainly search images instead of Web pages or online services. It is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

July 2016

1296 pages

ISBN:9781450340694

DOI:10.1145/2911451

General Chairs:
Raffaele Perego
ISTI-CNR, Italy
,
Fabrizio Sebastiani
Qatar Computing Research Institute, HBKU, Qatar
,
Program Chairs:
Javed Aslam
Northeastern University, US
,
Ian Ruthven
University of Strathclyde, UK
,
Justin Zobel
University of Melbourne, Australia

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGIR '16

Sponsor:

SIGIR

SIGIR '16: The 39th International ACM SIGIR conference on research and development in Information Retrieval

July 17 - 21, 2016

Pisa, Italy

Acceptance Rates

SIGIR '16 Paper Acceptance Rate 62 of 341 submissions, 18%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
406
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rehse JAbb LBerg GBormann CKampik TWarmuth C(2024)User Behavior MiningBusiness & Information Systems Engineering10.1007/s12599-023-00848-1Online publication date: 5-Jan-2024
https://doi.org/10.1007/s12599-023-00848-1
Zhang RXie XMao JLiu YZhang MMa S(2021)Constructing a Comparison-based Click Model for Web SearchProceedings of the Web Conference 202110.1145/3442381.3449918(270-283)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3449918
Xie XMao JLiu Yde Rijke MHuang JChang YCheng XKamps JMurdock VWen JLiu Y(2020)Modeling User Behavior for Vertical Search: Images, Apps and ProductsProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401423(2440-2443)Online publication date: 25-Jul-2020
https://dl.acm.org/doi/10.1145/3397271.3401423
Xie XMao JLiu Yde Rijke MChen HZhang MMa SHuang JChang YCheng XKamps JMurdock VWen JLiu Y(2020)Preference-based Evaluation Metrics for Web Image SearchProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401146(369-378)Online publication date: 25-Jul-2020
https://dl.acm.org/doi/10.1145/3397271.3401146
Ukkonen AJoona PRuotsalo THuang JChang YCheng XKamps JMurdock VWen JLiu Y(2020)Generating Images Instead of Retrieving ThemProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401129(1329-1338)Online publication date: 25-Jul-2020
https://dl.acm.org/doi/10.1145/3397271.3401129
Wu ZSanderson MCambazoglu BCroft WScholer Fd'Aquin MDietze SHauff CCurry ECudre Mauroux P(2020)Providing Direct Answers in Search Results: A Study of User BehaviorProceedings of the 29th ACM International Conference on Information & Knowledge Management10.1145/3340531.3412017(1635-1644)Online publication date: 19-Oct-2020
https://dl.acm.org/doi/10.1145/3340531.3412017
Wu ZZhou KLiu YZhang MMa S(2019)Does Diversity Affect User Satisfaction in Image SearchACM Transactions on Information Systems10.1145/332011837:3(1-30)Online publication date: 8-May-2019
https://dl.acm.org/doi/10.1145/3320118
Shao YLiu YZhang FZhang MMa S(2019)On Annotation Methodologies for Image Search EvaluationACM Transactions on Information Systems10.1145/330999437:3(1-32)Online publication date: 27-Mar-2019
https://dl.acm.org/doi/10.1145/3309994
Xie XMao JLiu Yde Rijke MShao YYe ZZhang MMa S(2019)Grid-based Evaluation Metrics for Web Image SearchThe World Wide Web Conference10.1145/3308558.3313514(2103-2114)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313514
Zhang FZhou KShao YLuo CZhang MMa SCollins-Thompson KMei QDavison BLiu YYilmaz E(2018)How Well do Offline and Online Evaluation Metrics Measure User Satisfaction in Web Image Search?The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval10.1145/3209978.3210059(615-624)Online publication date: 27-Jun-2018
https://dl.acm.org/doi/10.1145/3209978.3210059
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents