short-paper

Studying Ranking-Incentivized Web Dynamics

Authors:

Moshe Tennenholtz,

Oren KurlandAuthors Info & Claims

SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 2093 - 2096

https://doi.org/10.1145/3397271.3401300

Published: 25 July 2020 Publication History

Abstract

The ranking incentives of many authors of Web pages play an important role in the Web dynamics. That is, authors who opt to have their pages highly ranked for queries of interest often respond to rankings for these queries by manipulating their pages; the goal is to improve the pages' future rankings. Various theoretical aspects of this dynamics have recently been studied using game theory. However, empirical analysis of the dynamics is highly constrained due to lack of publicly available datasets. We present an initial such dataset that is based on TREC's ClueWeb09 dataset. Specifically, we used the WayBack Machine of the Internet Archive to build a document collection that contains past snapshots of ClueWeb documents which are highly ranked by some initial search performed for ClueWeb queries. Temporal analysis of document changes in this dataset reveals that findings recently presented for small-scale controlled ranking competitions between documents' authors also hold for Web data. Specifically, documents' authors tend to mimic the content of documents that were highly ranked in the past, and this practice can result in improved ranking.

References

[1]

Ablimit Aji, Yu Wang, Eugene Agichtein, and Evgeniy Gabrilovich. 2010. Using the past to score the present: extending term weighting models through revision history analysis. In Proc. of CIKM. 629--638.

Digital Library

[2]

Ran Ben-Basat, Moshe Tennenholtz, and Oren Kurland. 2017. A Game Theoretic Analysis of the Adversarial Retrieval Setting. J. Artif. Intell. Res. 60 (2017), 1127--1164.

[3]

Omer Ben-Porat and Moshe Tennenholtz. 2018. A Game-Theoretic Approach to Recommendation Systems with Strategic Content Providers. In Proc. of NIPS. 1110--1120.

[4]

Michael Bendersky, W Bruce Croft, and Yanlei Diao. 2011. Quality-biased ranking of web documents. In Proc. of WSDM. 95--104.

Digital Library

[5]

Carlos Castillo and Brian D. Davison. 2010. Adversarial Web Search. Foundations and Trends in Information Retrieval 4, 5 (2010), 377--486.

Digital Library

[6]

Charles L. A. Clarke, Nick Craswell, and Ian Soboroff. 2009. Overview of the TREC 2009 Web Track. In Proc. of TREC.

[7]

Gordon V. Cormack, Mark D. Smucker, and Charles L. A. Clarke. 2010. Efficient and Effective Spam Filtering and Re-ranking for Large Web Datasets. arXiv: 1004.5168 [cs.IR]

[8]

Jonathan L. Elsas and Susan T. Dumais. 2010. Leveraging Temporal Dynamics of Document Content in Relevance Ranking. In Proc. of WSDM. 1--10.

[9]

Gregory Goren, Oren Kurland, Moshe Tennenholtz, and Fiana Raiber. 2018. Ranking Robustness Under Adversarial Document Manipulations. In Proc. of SIGIR.395--404.

Digital Library

[10]

Zoltán Gyöngyi and Hector Garcia-Molina. 2005. Web Spam Taxonomy. In Proc. of AIR Web 2005. 39--47.

[11]

John D. Lafferty and Chengxiang Zhai. 2001. Document language models, query models, and risk minimization for information retrieval. In Proc. of SIGIR. 111--119.

[12]

Sérgio Nunes, Cristina Ribeiro, and Gabriel David. 2011. Term Weighting Based on Document Revision History. JASIST 62 (12 2011), 2471--2478.

[13]

Kira Radinsky and Paul N. Bennett. 2013. Predicting content change on the web. In Proc. of WSDM. 415--424.

[14]

Kira Radinsky, Fernando Diaz, Susan T. Dumais, Milad Shokouhi, Anlei Dong, and Yi Chang. 2013. Temporal web dynamics and its application to information retrieval. In Proc. of WSDM. 781--782.

Digital Library

[15]

Nimrod Raifer, Fiana Raiber, Moshe Tennenholtz, and Oren Kurland. 2017. Information Retrieval Meets Game Theory: The Ranking Competition Between Documents' Authors. In Proc. of SIGIR. 465--474.

Digital Library

[16]

Stephen E. Robertson. 1977. The Probability Ranking Principle in IR. Journal of Documentation(1977), 294--304. Reprinted in K. Sparck Jones and P. Willett (eds), Readings in Information Retrieval, pp. 281--286, 1997.

[17]

Moshe Tennenholtz and Oren Kurland. 2019. Rethinking search engines and recommendation systems: a game theoretic perspective. Commun. ACM 62, 12(2019), 66--75.

Digital Library

Cited By

Vasilisky ZKurland OTennenholtz MRaiber FYoshioka MKiseleva JAliannejadi M(2023)Content-Based Relevance Estimation in Retrieval Settings with Ranking-Incentivized Document ManipulationsProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3578337.3605124(205-214)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3578337.3605124
Li RLi JMitra BDiaz FBiega A(2022)Exposing Query Identification for Search TransparencyProceedings of the ACM Web Conference 202210.1145/3485447.3512262(3662-3672)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3485447.3512262

Index Terms

Studying Ranking-Incentivized Web Dynamics
1. Information systems
  1. Information retrieval
    1. Search engine architectures and scalability
      1. Adversarial retrieval

Recommendations

Ranking-Incentivized Quality Preserving Content Modification
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

The Web is a canonical example of a competitive retrieval setting where many documents' authors consistently modify their documents to promote them in rankings. We present an automatic method for quality-preserving modification of document content --- ...
Context-aware ranking in web search
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval

The context of a search query often provides a search engine meaningful hints for answering the current query better. Previous studies on context-aware search were either focused on the development of context models or limited to a relatively small ...
Evaluation of web search engine based on ranking of results and its features

Search engines help the user to surf the web. Due to the vast number of web pages, it is highly impossible for the user to retrieve the appropriate web page he needs. Thus, web search ranking algorithms play an important role in ranking web pages, so ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2020

2548 pages

ISBN:9781450380164

DOI:10.1145/3397271

General Chairs:
Jimmy Huang
York University, Canada
,
Yi Chang
Jilin University, China
,
Xueqi Cheng
Chinese Academy of Sciences, China
,
Program Chairs:
Jaap Kamps
University of Amsterdam, Netherlands
,
Vanessa Murdock
Amazon, U.S.A.
,
Ji-Rong Wen
Renmin University of China, China
,
Yiqun Liu
Tsinghua University, China

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Short-paper

Funding Sources

Horizon 2020 Framework Programme

Conference

SIGIR '20

Sponsor:

SIGIR

SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval

July 25 - 30, 2020

Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
90
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Vasilisky ZKurland OTennenholtz MRaiber FYoshioka MKiseleva JAliannejadi M(2023)Content-Based Relevance Estimation in Retrieval Settings with Ranking-Incentivized Document ManipulationsProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3578337.3605124(205-214)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3578337.3605124
Li RLi JMitra BDiaz FBiega A(2022)Exposing Query Identification for Search TransparencyProceedings of the ACM Web Conference 202210.1145/3485447.3512262(3662-3672)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3485447.3512262

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents