research-article

Open access

The Bandwagon Effect: Not Just Another Bias

Authors:

Norman Knyazev,

Harrie OosterhuisAuthors Info & Claims

ICTIR '22: Proceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval

Pages 243 - 253

https://doi.org/10.1145/3539813.3545142

Published: 25 August 2022 Publication History

Abstract

Optimizing recommender systems based on user interaction data is mainly seen as a problem of dealing with selection bias, where most existing work assumes that interactions from different users are independent. However, it has been shown that in reality user feedback is often influenced by earlier interactions of other users, e.g. via average ratings, number of views or sales per item, etc. This phenomenon is known as the bandwagon effect.

In contrast with previous literature, we argue that the bandwagon effect should not be seen as a problem of statistical bias. In fact, we prove that this effect leaves both individual interactions and their sample mean unbiased. Nevertheless, we show that it can make estimators inconsistent, introducing a distinct set of problems for convergence in relevance estimation. Our theoretical analysis investigates the conditions under which the bandwagon effect poses a consistency problem and explores several approaches for mitigating these issues. This work aims to show that the bandwagon effect poses an underinvestigated open problem that is fundamentally distinct from the well-studied selection bias in recommendation.

References

[1]

Gediminas Adomavicius, Jesse C. Bockstedt, Shawn Curley, and Jingjing Zhang. 2016. Understanding Effects of Personalized vs. Aggregate Ratings on User Preferences. CEUR Workshop Proceedings 1679 (2016), 14--21.

[2]

Solomon. E. Asch. 1951. Effects of Group Pressure upon the Modification and Distortion of Judgments. In Groups, Leadership and Men; Research in Human Relations. Carnegie Press, 177--190.

[3]

Christian Borghesi and Jean-Philippe Bouchaud. 2010. Spatial Correlations in Vote Statistics: A Diffusive Field Model for Decision-Making. The European Physical Journal B 75, 3 (2010), 395--404.

[4]

Rocío Cañamares and Pablo Castells. 2018. Should I Follow The Crowd? A Probabilistic Analysis of the Effectiveness of Popularity in Recommender Systems. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 415--424.

Digital Library

[5]

Franco Caparrelli, Anna M. D'Arcangelis, and Alexander Cassuto. 2004. Herding in the Italian Stock Market: A Case of Behavioral Finance. Journal of Behavioral Finance 5, 4 (2004), 222--230.

[6]

Aleksandr Chuklin, Ilya Markov, and Maarten de Rijke. 2015. Click Models for Web Search. Morgan & Claypool Publishers.

[7]

Dan Cosley, Shyong K. Lam, Istvan Albert, Joseph A. Konstan, and John Riedl. 2003. Is Seeing Believing? How Recommender Interfaces Affect Users' Opinions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 585--592.

[8]

Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. 2008. An Experimental Comparison of Click Position-Bias Models. In Proceedings of the 2008 International Conference on Web Search and Data Mining. 87--94.

Digital Library

[9]

Martin J. Crowder. 1976. Maximum Likelihood Estimation for Dependent Observations. Journal of the Royal Statistical Society. Series B (Methodological) 38, 1 (1976), 45--53.

[10]

Utpal M. Dholakia and Kerry Soltysinski. 2001. Coveted or Overlooked? The Psychology of Bidding for Comparable Listings in Digital Auctions. Marketing Letters 12, 3 (2001), 225--237.

[11]

Erasmo M. Ferreira, Anderson K. Kohara, and Javier Sesma. 2017. New Properties of the Lerch's Transcendent. Journal of Number Theory 172 (2017), 21--31.

[12]

Andrew J. Flanagin and Miriam J. Metzger. 2013. Trusting Expert- versus User-Generated Ratings Online: The Role of Information Volume, Valence, and Consumer Characteristics. Computers in Human Behavior 29, 4 (2013), 1626--1634.

Digital Library

[13]

W. Wayne Fu and Clarice C. Sim. 2011. Aggregate Bandwagon Effect on Online Videos' Viewership: Value Uncertainty, Popularity Cues, and Heuristics. Journal of the American Society for Information Science and Technology 62, 12 (2011), 2382--2395.

Digital Library

[14]

Aurel Galántai. 2000. The Theory of Newton's Method. J. Comput. Appl. Math. 124, 1 (2000), 25--44.

Digital Library

[15]

Ward A. Hanson and Daniel S. Putler. 1996. Hits and Misses: Herd Behavior and Online Product Popularity. Marketing Letters 7, 4 (1996), 297--305.

[16]

José M. Hernández-Lobato, Neil Houlsby, and Zoubin Ghahramani. 2014. Probabilistic Matrix Factorization with Non-random Missing Data. In Proceedings of the 31st International Conference on Machine Learning. 1512--1520.

[17]

Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative Filtering for Implicit Feedback Datasets. In 2008 Eighth IEEE International Conference on Data Mining. 263--272.

Digital Library

[18]

Jin Huang, Harrie Oosterhuis, Maarten de Rijke, and Herke van Hoof. 2020. Keeping Dataset Biases out of the Simulation: A Debiased Simulator for Reinforcement Learning based Recommender Systems. In Fourteenth ACM Conference on Recommender Systems. 190--199.

Digital Library

[19]

Jen-Hung Huang and Yi-Fen Chen. 2006. Herding in Online Product Choice. Psychology & Marketing 23, 5 (2006), 413--428.

[20]

Ilija Ilievski and Sujoy Roy. 2013. Personalized News Recommendation Based On Implicit Feedback. In Proceedings of the 2013 International News Recommender Systems Workshop and Challenge. 10--15.

Digital Library

[21]

Dietmar Jannach, Lukas Lerche, and Markus Zanker. 2018. Recommending Based on Implicit Feedback. In Social Information Access. Springer, 510--569.

[22]

Olivier Jeunen and Bart Goethals. 2021. Pessimistic Reward Models for Off-Policy Learning in Recommendation. In Fifteenth ACM Conference on Recommender Systems. 63--74.

[23]

Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased Learning-to-Rank with Biased Feedback. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. 781--789.

Digital Library

[24]

Jinyoung Kim and AndrewGambino. 2016. DoWe Trust the Crowd or Information System? Effects of Personalization and Bandwagon Cues on Users' Attitudes and Behavioral Intentions toward a Restaurant RecommendationWebsite. Computers in Human Behavior 65 (2016), 369--379.

Digital Library

[25]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. Computer 42, 8 (2009), 30--37.

Digital Library

[26]

László Kozma. 2021. Useful Inequalities Cheat Sheet. Retrieved February 14, 2022 from http://www.lkozma.net/inequalities_cheat_sheet/ineq.pdf

[27]

Bibb Latané and SharonWolf. 1981. The Social Impact of Majorities and Minorities. Psychological Review 88, 5 (1981), 438--453.

[28]

Isaac Leonard. 2012. Notes on the Riemann Zeta Function. MATH 324, Summer 2012, Elementary Number Theory, University of Alberta. Retrieved February 14, 2022 from http://www.math.ualberta.ca/~isaac/math324/s12/zeta.pdf

[29]

Roderick J. A. Little and Donald B. Rubin. 2019. Statistical Analysis with Missing Data. John Wiley & Sons.

[30]

Yiming Liu, Xuezhi Cao, and Yong Yu. 2016. Are You Influenced by Others When Rating?: Improve Rating Prediction by Conformity Modeling. In Proceedings of the 10th ACM Conference on Recommender Systems. 269--272.

Digital Library

[31]

M. Loève. 1977. Probability Concepts. In Probability Theory I. Springer, 151--176.

[32]

Benjamin M. Marlin and Richard S Zemel. 2009. Collaborative Prediction and Ranking with Non-Random Missing Data. In Proceedings of the Third ACM Conference on Recommender Systems. 5--12.

[33]

Harrie Oosterhuis. 2020. Learning from User Interactions with Rankings: A Unification of the Field. Ph.D. Dissertation. Informatics Institute, University of Amsterdam.

[34]

Harrie Oosterhuis and Maarten de Rijke. 2020. Policy-Aware Unbiased Learning to Rank for Top-k Rankings. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2020), 489--498.

Digital Library

[35]

Bruno Pradel, Nicolas Usunier, and Patrick Gallinari. 2012. Ranking with Non-Random Missing Satings: Influence of Popularity and Positivity on Evaluation Metrics. In Proceedings of the Sixth ACM Conference on Recommender Systems. 147--154.

Digital Library

[36]

Zhen Qin, Honglei Zhuang, Rolf Jagerman, Xinyu Qian, Po Hu, Dan Chary Chen, XuanhuiWang, Michael Bendersky, and Marc Najork. 2021. Bootstrapping Recommendations at ChromeWeb Store. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 3483--3491.

[37]

Ramsey M. Raafat, Nick Chater, and Chris Frith. 2009. Herding in Humans. Trends in Cognitive Sciences 13, 10 (2009), 420--428.

[38]

Francesco Ricci, Lior Rokach, and Bracha Shapira. 2015. Recommender Systems: Introduction and Challenges. In Recommender Systems Handbook. Springer, 1--34.

[39]

Yuta Saito. 2020. Doubly Robust Estimator for Ranking Metrics with Post-Click Conversions. In Fourteenth ACM Conference on Recommender Systems. 92--100.

[40]

Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020. Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback. In Proceedings of the 13th International Conference on Web Search and Data Mining. 501--509.

Digital Library

[41]

Matthew J. Salganik, Peter S. Dodds, and Duncan J. Watts. 2006. Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market. Science 311, 5762 (2006), 854--856.

[42]

Matthew J. Salganik and Duncan J. Watts. 2008. Leading the Herd Astray: An Experimental Study of Self-fulfilling Prophecies in an Artificial Cultural Market. Social Psychology Quarterly 71, 4 (2008), 338--355.

[43]

Masahiro Sato, Janmajay Singh, Sho Takemori, and Qian Zhang. 2021. Causality-Aware Neighborhood Methods for Recommender Systems. In European Conference on Information Retrieval. 603--618.

[44]

Masahiro Sato, Sho Takemori, Janmajay Singh, and Tomoko Ohkuma. 2020. Unbiased Learning for the Causal Effect of Recommendation. Fourteenth ACM Conference on Recommender Systems (2020), 378--387.

[45]

Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations As Treatments: Debiasing Learning and Evaluation. In Proceedings of the 33rd International Conference on International Conference on Machine Learning. 1670--1679.

[46]

Herbert A. Simon. 1954. Bandwagon and Underdog Effects and the Possibility of Election Predictions. The Public Opinion Quarterly 18, 3 (1954), 245--253.

[47]

Harald Steck. 2011. Item Popularity and Recommendation Accuracy. In Proceedings of the Fifth ACM Conference on Recommender Systems. 125--132.

Digital Library

[48]

Harald Steck. 2013. Evaluation of Recommendations: Rating-Prediction and Ranking. In Proceedings of the Seventh ACM Conference on Recommender Systems. 213--220.

Digital Library

[49]

Alexander L. Strehl, John Langford, Lihong Li, and Sham M. Kakade. 2010. Learning from Logged Implicit Exploration Data. In Proceedings of the 23rd International Conference on Neural Information Processing Systems. 2217--2225.

[50]

Xiaoyuan Su and Taghi M. Khoshgoftaar. 2009. A Survey of Collaborative Filtering Techniques. Advances in Artificial Intelligence (2009).

[51]

S. Shyam Sundar, Anne Oeldorf-Hirsch, and Qian Xu. 2008. The Bandwagon Effect of Collaborative Filtering Technology. In CHI '08 Extended Abstracts on Human Factors in Computing Systems. 3453--3458.

[52]

Adith Swaminathan and Thorsten Joachims. 2015. Counterfactual Risk Minimization: Learning from Logged Bandit Feedback. In Proceedings of the 32nd International Conference on Machine Learning. 814--823.

[53]

Adith Swaminathan and Thorsten Joachims. 2015. The Self-Normalized Estimator for Counterfactual Learning. In Advances in Neural Information Processing Systems, Vol. 28. 3231--3239.

[54]

Duyu Tang, Bing Qin, Ting Liu, and Yuekui Yang. 2015. User Modeling with Neural Network for Review Rating Prediction. In Proceedings of the 24th International Conference on Artificial Intelligence. 1340--1346.

[55]

Ali Vardasbi, Maarten de Rijke, and Ilya Markov. 2020. Cascade Model-Based Propensity Estimation for Counterfactual Learning to Rank. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2089--2092.

Digital Library

[56]

Ting Wang, Dashun Wang, and Fei Wang. 2014. Quantifying Herding Effects in Crowd Wisdom. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1087--1096.

Digital Library

[57]

Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. 2019. Doubly Robust Joint Learning for Recommendation on Data Missing Not at Random. In International Conference on Machine Learning. 6638--6647.

[58]

YixinWang, Dawen Liang, Laurent Charlin, and David M. Blei. 2020. Causal Inference for Recommender Systems. In Fourteenth ACM Conference on Recommender Systems. 426--431.

[59]

Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, and Xiangnan He. 2021. Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (2021), 1791--1800.

Digital Library

[60]

Hong Xie and Mingze Zhong. 2020. Robust Product Rating Rules Against Herding Effects: Theory and Applications. In 2020 IEEE International Conference on Data Mining. 1352--1357.

[61]

Hong Xie, Mingze Zhong, Yongkun Li, and John C. S. Lui. 2021. Understanding Persuasion Cascades in Online Product Rating Systems: Modeling, Analysis, and Inference. ACM Transactions on Knowledge Discovery from Data 15, 3 (2021), 1--29.

[62]

Longqi Yang, Yin Cui, Yuan Xuan, Chenyang Wang, Serge Belongie, and Deborah Estrin. 2018. Unbiased Offline Recommender Evaluation for Missing-Notat-Random Implicit Feedback. In Proceedings of the 12th ACM Conference on Recommender Systems. 279--287.

Digital Library

[63]

Bowen Yuan, Yaxu Liu, Jui-Yang Hsia, Zhenhua Dong, and Chih-Jen Lin. 2020. Unbiased Ad Click Prediction for Position-aware Advertising Systems. In Fourteenth ACM Conference on Recommender Systems. 368--377.

[64]

Xiaoying Zhang, Hong Xie, Junzhou Zhao, and John C. S. Lui. 2020. Understanding Assimilation-Contrast Effects in Online Rating Systems: Modelling, Debiasing, and Applications. ACM Transactions on Information Systems 38, 1 (2020), 1--25.

Digital Library

[65]

Xiaoying Zhang, Junzhou Zhao, and John C.S. Lui. 2017. Modeling the Assimilation-Contrast Effects in Online Product Rating Systems: Debiasing and Recommendations. In Proceedings of the Eleventh ACM Conference on Recommender Systems. 98--106.

[66]

Yang Zhang, Fuli Feng, Xiangnan He, TianxinWei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021. Causal Intervention for Leveraging Popularity Bias in Recommendation. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021), 11--20.

Digital Library

[67]

Mingze Zhong, Hong Xie, and Qingsheng Zhu. 2021. Quantifying Assimilate- Contrast Effects in Online Rating Systems: Modeling, Analysis and Application. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2369--2377.

Digital Library

[68]

Honglei Zhuang, Zhen Qin, Xuanhui Wang, Michael Bendersky, Xinyu Qian, Po Hu, and Dan Chary Chen. 2021. Cross-Positional Attention for Debiasing Clicks. In Proceedings of the Web Conference 2021. 788--797.

Digital Library

Cited By

Ruiz De villa ASottocornola GCoba LLucchesi FSkorulski B(2024)Ranking the causal impact of recommendations under collider bias in k-spots recommender systemsACM Transactions on Recommender Systems10.1145/36431392:2(1-29)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1145/3643139
Huang JOosterhuis HMansoury Mvan Hoof Hde Rijke MHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Going Beyond Popularity and Positivity Bias: Correcting for Multifactorial Bias in Recommender SystemsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657749(416-426)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657749
Zhang YSang LZhang YHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Exploring the Individuality and Collectivity of Intents behind Interactions for Graph Collaborative FilteringProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657738(1253-1262)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657738
Show More Cited By

Index Terms

The Bandwagon Effect: Not Just Another Bias
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking
      1. Learning to rank
  2. World Wide Web
    1. Web searching and information discovery
      1. Social recommendation
2. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Time series analysis

Recommendations

It Is Different When Items Are Older: Debiasing Recommendations When Selection Bias and User Preferences Are Dynamic
WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

User interactions with recommender systems (RSs) are affected by user selection bias, e.g., users are more likely to rate popular items (popularity bias) or items that they expect to enjoy beforehand (positivity bias). Methods exist for mitigating the ...
Bias and Debias in Recommender System: A Survey and Future Directions
While recent years have witnessed a rapid growth of research papers on recommender system (RS), most of the papers focus on inventing machine learning models to better fit user behavior data. However, user behavior data is observational rather than ...
Bandwagon Effect in Facebook Discussion Groups
ASE BD&SI '15: Proceedings of the ASE BigData & SocialInformatics 2015

A core issue in online social networks is identifying content that will result in greater traffic and interaction. In this paper we use a large Facebook dataset and the concepts of bandwagon effect and information cascade from the field of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICTIR '22: Proceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval

August 2022

289 pages

ISBN:9781450394123

DOI:10.1145/3539813

Program Chairs:
Fabio Crestani
Università della Svizzera Italiana - USI, Switzerland
,
Gabriella Pasi
Univ. Milano-Bicocca, Italy
,
Eric Gaussier
Univ. Grenoble-Alpes, France

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 August 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Google Inc

Conference

ICTIR '22

Sponsor:

SIGIR

ICTIR '22: The 2022 ACM SIGIR International Conference on the Theory of Information Retrieval

July 11 - 12, 2022

Madrid, Spain

Acceptance Rates

ICTIR '22 Paper Acceptance Rate 32 of 80 submissions, 40%;

Overall Acceptance Rate 235 of 527 submissions, 45%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
583
Total Downloads

Downloads (Last 12 months)345
Downloads (Last 6 weeks)56

Reflects downloads up to 19 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ruiz De villa ASottocornola GCoba LLucchesi FSkorulski B(2024)Ranking the causal impact of recommendations under collider bias in k-spots recommender systemsACM Transactions on Recommender Systems10.1145/36431392:2(1-29)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1145/3643139
Huang JOosterhuis HMansoury Mvan Hoof Hde Rijke MHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Going Beyond Popularity and Positivity Bias: Correcting for Multifactorial Bias in Recommender SystemsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657749(416-426)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657749
Zhang YSang LZhang YHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Exploring the Individuality and Collectivity of Intents behind Interactions for Graph Collaborative FilteringProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657738(1253-1262)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657738
Han LWei ZZhang S(2024)The Marketing Value of User-Generated Content in the Mobile IndustryProceedings of the 7th International Conference on Economic Management and Green Development10.1007/978-981-97-0523-8_107(1130-1142)Online publication date: 27-Feb-2024
https://doi.org/10.1007/978-981-97-0523-8_107
Knyazev NOosterhuis H(2023)A Lightweight Method for Modeling Confidence in Recommendations with Learned Beta DistributionsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608788(306-317)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608788

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents