short-paper

Quantifying and Leveraging User Fatigue for Interventions in Recommender Systems

Authors:

Hitesh Sagtani,

Madan Gopal Jhawar,

Rishabh MehrotraAuthors Info & Claims

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 2293 - 2297

https://doi.org/10.1145/3539618.3592044

Published: 18 July 2023 Publication History

Abstract

Predicting churn and designing intervention strategies are crucial for online platforms to maintain user engagement. We hypothesize that predicting churn, i.e. users leaving from the system without further return, is often a delayed act, and it might get too late for the system to intervene. We propose detecting early signs of users losing interest, allowing time for intervention, and introduce a new formulation ofuser fatigue as short-term dissatisfaction, providing early signals to predict long-term churn. We identify behavioral signals predicting fatigue and develop models for fatigue prediction. Furthermore, we leverage the predicted fatigue estimates to develop fatigue-aware ad-load balancing intervention strategy that reduces churn, improving short- and long-term user retention. Results from deployed recommendation system and multiple live A/B tests across over 80 million users generating over 200 million sessions highlight gains for user engagement and platform strategic metrics.

References

[1]

Christian Abbet, Meryem M'hamdi, Athanasios Giannakopoulos, Robert West, Andreea Hossmann, Michael Baeriswyl, and Claudiu Musat. 2018. Churn intent detection in multilingual chatbot conversations and social media. arXiv preprint arXiv:1808.08432 (2018).

[2]

Essam Abou el Kassem, Shereen Ali Hussein, Alaa Mostafa Abdelrahman, and Fahad Kamal Alsheref. 2020. Customer Churn Prediction Model and Identifying Features to Increase Customer Retention based on User Generated Content. IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, 5 (2020).

[3]

Ionut Brandusoiu and Gavril Toderean. 2013. Churn prediction modeling in mobile telecommunications industry using decision trees. Journal of Computer Science and Control Systems, Vol. 6, 1 (2013), 14.

[4]

Praveen Chandar, Brian St. Thomas, Lucas Maystre, Vijay Pappu, Roberto Sanchis-Ojeda, Tiffany Wu, Ben Carterette, Mounia Lalmas, and Tony Jebara. 2022. Using Survival Models to Estimate User Engagement in Online Experiments. In Proceedings of the ACM Web Conference 2022. 3186--3195.

Digital Library

[5]

Preeti K Dalvi, Siddhi K Khandge, Ashish Deomore, Aditya Bankar, and VA Kanade. 2016. Analysis of customer churn prediction in telecom industry using decision trees and logistic regression. In 2016 Symposium on Colossal Data Analysis and Networking (CDAN). IEEE, 1--4.

[6]

Kushal S Dave, Vishal Vaingankar, Sumanth Kolar, and Vasudeva Varma. 2013. Timespent based models for predicting user retention. In Proceedings of the 22nd international conference on World Wide Web. 331--342.

Digital Library

[7]

Simon Demediuk, Alexandra Murrin, David Bulger, Michael Hitchens, Anders Drachen, William L Raffe, and Marco Tamassia. 2018. Player retention in league of legends: a study using survival analysis. In Proceedings of the Australasian computer science week multiconference. 1--9.

Digital Library

[8]

Georges Dupret and Mounia Lalmas. 2013. Absence time and user engagement: evaluating ranking functions. In Proceedings of the sixth ACM international conference on Web search and data mining. 173--182.

Digital Library

[9]

Mohammed Hassouna, Ali Tarhini, Tariq Elyas, and Mohammad Saeed AbouTrab. 2016. Customer churn in mobile markets a comparison of techniques. arXiv preprint arXiv:1607.07792 (2016).

[10]

Tobias Hatt and Stefan Feuerriegel. 2020. Early detection of user exits from clickstream data: A markov modulated marked point process model. In Proceedings of The Web Conference 2020. 1671--1681.

Digital Library

[11]

Sebastiaan Höppner, Eugen Stripling, Bart Baesens, Seppe vanden Broucke, and Tim Verdonck. 2020. Profit driven decision trees for churn prediction. European journal of operational research, Vol. 284, 3 (2020), 920--933.

[12]

Dmitry Lagun and Mounia Lalmas. 2016. Understanding and measuring user engagement and attention in online news reading. In Proceedings of the ACM International Conference on Web Search and Data Mining. 113--122.

Digital Library

[13]

Mounia Lalmas and Liangjie Hong. 2018. Tutorial on metrics of user engagement: Applications to news, search and E-commerce. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. 781--782.

Digital Library

[14]

Jiayu Li, Hongyu Lu, Chenyang Wang, Weizhi Ma, Min Zhang, Xiangyu Zhao, Wei Qi, Yiqun Liu, and Shaoping Ma. 2021. A Difficulty-Aware Framework for Churn Prediction and Intervention in Games. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 943--952.

Digital Library

[15]

Yunfei Lu, Linyun Yu, Peng Cui, Chengxi Zang, Renzhe Xu, Yihao Liu, Lei Li, and Wenwu Zhu. 2019. Uncovering the co-driven mechanism of social and content links in user churn phenomena. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3093--3101.

Digital Library

[16]

Milovs Milosević, Nenad Zivić, and Igor Andjelković. 2017. Early churn prediction with personalized targeting in mobile social games. Expert Systems with Applications, Vol. 83 (2017), 326--332.

Digital Library

[17]

Maria Óskarsdóttir, Tine Van Calster, Bart Baesens, Wilfried Lemahieu, and Jan Vanthienen. 2018. Time series for early churn detection: Using similarity based classification for dynamic networks. Expert Systems with Applications, Vol. 106 (2018), 55--65.

[18]

Jagat Sastry Pudipeddi, Leman Akoglu, and Hanghang Tong. 2014. User churn in focused question answering sites: characterizations and prediction. In Proceedings of the 23rd International conference on world wide web. 469--474.

Digital Library

[19]

Vineeth Rakesh, Ajith Pudiyavitil, and Jaideep Chandrashekar. 2020. User Modeling and Churn Prediction in Over-the-top Media Services. In ORSUM@ RecSys.

[20]

Shini Renjith. 2015. An integrated framework to recommend personalized retention actions to control B2C E-commerce customer churn. arXiv preprint arXiv:1511.06975 (2015).

[21]

Petra Posedel Simović, Davor Horvatic, and Edward W Sun. 2021. Classifying variety of customer's online engagement for churn prediction with mixed-penalty logistic regression. arXiv preprint arXiv:2105.07671 (2021).

[22]

Siqi Wu, Marian-Andrei Rizoiu, and Lexing Xie. 2018. Beyond views: Measuring and predicting engagement in online videos. In Twelfth international AAAI conference on web and social media.

[23]

Xiaojun Wu and Sufang Meng. 2016. E-commerce customer churn prediction based on improved SMOTE and AdaBoost. In 2016 13th International conference on service systems and service management (ICSSSM). IEEE, 1--5.

[24]

Carl Yang, Xiaolin Shi, Luo Jie, and Jiawei Han. 2018. I know you'll be back: Interpretable new user clustering and churn prediction on a mobile social application. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 914--922.

Digital Library

[25]

Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu, and Suju Rajan. 2014. Beyond clicks: dwell time for personalization. In Proceedings of the 8th ACM Conference on Recommender systems. 113--120.

Digital Library

[26]

Xiaobing Yu, Shunsheng Guo, Jun Guo, and Xiaorong Huang. 2011. An extended support vector machine forecasting framework for customer churn in e-commerce. Expert Systems with Applications, Vol. 38, 3 (2011), 1425--1430.

Digital Library

Cited By

Jeunen OMandav JPotapov IAgarwal NVaid SShi WUstimenko A(2024)Multi-Objective Recommendation via Multivariate Policy LearningProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688132(712-721)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688132
Echterhoff JMelkote AKancherla SMcAuley J(2024)Avoiding Decision Fatigue with AI-Assisted Decision-MakingProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659569(1-11)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3627043.3659569
Sagtani HJhawar MMehrotra RJeunen OAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Ad-load Balancing via Off-policy Learning in a Content MarketplaceProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635846(586-595)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635846
Show More Cited By

Index Terms

Quantifying and Leveraging User Fatigue for Interventions in Recommender Systems
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems
    2. Users and interactive retrieval
      1. Personalization

Recommendations

User Fatigue in Online News Recommendation
WWW '16: Proceedings of the 25th International Conference on World Wide Web

Many aspects and properties of Recommender Systems have been well studied in the past decade, however, the impact of User Fatigue has been mostly ignored in the literature. User fatigue represents the phenomenon that a user quickly loses the interest on ...
Multi-granularity Fatigue in Recommendation
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Personalized recommendation aims to provide appropriate items according to user preferences mainly from their behaviors. Excessive homogeneous user behaviors on similar items will lead to fatigue, which may decrease user activeness and degrade user ...
Modeling User Fatigue for Sequential Recommendation
SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

Recommender systems filter out information that meets user interests. However, users may be tired of the recommendations that are too similar to the content they have been exposed to in a short historical period, which is the so-called user fatigue. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2023

3567 pages

ISBN:9781450394086

DOI:10.1145/3539618

General Chairs:
Hsin-Hsi Chen
National Taiwan University
,
Wei-Jou (Edward) Duh
National Taiwan University
,
Hen-Hsen Huang
Academia Sinica
,
Program Chairs:
Makoto P. Kato
Spotify
,
Josiane Mothe
Universite de Toulouse
,
Barbara Poblete
University of Chile and Amazon Visiting Academic

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

SIGIR '23

Sponsor:

SIGIR

SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 23 - 27, 2023

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
198
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)5

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jeunen OMandav JPotapov IAgarwal NVaid SShi WUstimenko A(2024)Multi-Objective Recommendation via Multivariate Policy LearningProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688132(712-721)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688132
Echterhoff JMelkote AKancherla SMcAuley J(2024)Avoiding Decision Fatigue with AI-Assisted Decision-MakingProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659569(1-11)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3627043.3659569
Sagtani HJhawar MMehrotra RJeunen OAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Ad-load Balancing via Off-policy Learning in a Content MarketplaceProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635846(586-595)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635846
Charles GKanani R(2024)Antecedents of social commerce purchase intention: evidence from Tanzanian social media usersCogent Business & Management10.1080/23311975.2024.244740912:1Online publication date: 30-Dec-2024
https://doi.org/10.1080/23311975.2024.2447409
Sagtani HJeunen OUstimenko A(2024)Learning-to-Rank with Nested FeedbackAdvances in Information Retrieval10.1007/978-3-031-56063-7_22(306-315)Online publication date: 24-Mar-2024
https://dl.acm.org/doi/10.1007/978-3-031-56063-7_22

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten