tutorial

Open access

Recent Advances in the Foundations and Applications of Unbiased Learning to Rank

Authors:

Shashank Gupta,

Harrie OosterhuisAuthors Info & Claims

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 3440 - 3443

https://doi.org/10.1145/3539618.3594247

Published: 18 July 2023 Publication History

Abstract

Since its inception, the field of unbiased learning to rank (ULTR) has remained very active and has seen several impactful advancements in recent years. This tutorial provides both an introduction to the core concepts of the field and an overview of recent advancements in its foundations along with several applications of its methods.

The tutorial is divided into four parts: Firstly, we give an overview of the different forms of bias that can be addressed with ULTR methods. Secondly, we present a comprehensive discussion of the latest estimation techniques in the ULTR field. Thirdly, we survey published results of ULTR in real-world applications. Fourthly, we discuss the connection between ULTR and fairness in ranking. We end by briefly reflecting on the future of ULTR research and its applications.

This tutorial is intended to benefit both researchers and industry practitioners who are interested in developing new ULTR solutions or utilizing them in real-world applications.

References

[1]

Aman Agarwal, Xuanhui Wang, Cheng Li, Michael Bendersky, and Marc Najork. 2019. Addressing Trust Bias for Unbiased Learning-to-rank. In The World Wide Web Conference. 4--14.

[2]

Qingyao Ai, Jiaxin Mao, Yiqun Liu, and W Bruce Croft. 2018. Unbiased learning to rank: Theory and practice. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2305--2306.

[3]

Asia J Biega, Krishna P Gummadi, and Gerhard Weikum. 2018. Equity of attention: Amortizing individual fairness in rankings. In The 41st international acm sigir conference on research & development in information retrieval. 405--414.

[4]

Adam Block, Rahul Kidambi, Daniel N Hill, Thorsten Joachims, and Inderjit S Dhillon. 2022. Counterfactual Learning To Rank for Utility-Maximizing Query Autocompletion. arXiv preprint arXiv:2204.10936 (2022).

[5]

Olivier Chapelle and Yi Chang. 2011. Yahoo! Learning to Rank Challenge Overview. In Proceedings of the learning to rank challenge. PMLR, 1--24.

[6]

Jiawei Chen, Xiang Wang, Fuli Feng, and Xiangnan He. 2021. Bias Issues and Solutions in Recommender System: Tutorial on the RecSys 2021. In Proceedings of the 15th ACM Conference on Recommender Systems. 825--827.

Digital Library

[7]

Ruey-Cheng Chen, Qingyao Ai, Gaya Jayasinghe, and W Bruce Croft. 2019. Correcting for recency bias in job recommendation. In Proceedings of the 28th ACM international conference on information and knowledge management. 2185--2188.

Digital Library

[8]

Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. 2008. An Experimental Comparison of Click Position-bias Models. In Proceedings of the 2008 international conference on web search and data mining. 87--94.

Digital Library

[9]

Miroslav Dudík, John Langford, and Lihong Li. 2011. Doubly robust policy evaluation and learning. arXiv preprint arXiv:1103.4601 (2011).

[10]

Artem Grotov and Maarten De Rijke. 2016. Online learning to rank for information retrieval: Sigir 2016 tutorial. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 1215--1218.

Digital Library

[11]

Huifeng Guo, Jinkai Yu, Qing Liu, Ruiming Tang, and Yuzhou Zhang. 2019. PAL: a position-bias aware learning framework for CTR prediction in live recommender systems. In Proceedings of the 13th ACM Conference on Recommender Systems. 452--456.

Digital Library

[12]

Shashank Gupta, Harrie Oosterhuis, and Maarten de Rijke. 2023. Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization. In SIGIR 2023: 46th international ACM SIGIR Conference on Research and Development in Information Retrieval. ACM.

Digital Library

[13]

Ziniu Hu, Yang Wang, Qu Peng, and Hang Li. 2019. Unbiased lambdamart: an unbiased pairwise learning-to-rank algorithm. In The World Wide Web Conference. 2830--2836.

Digital Library

[14]

Jiawei Huang and Nan Jiang. 2020. From importance sampling to doubly robust policy gradient. In International Conference on Machine Learning. PMLR, 4434--4443.

[15]

Nan Jiang and Lihong Li. 2016. Doubly robust off-policy value evaluation for reinforcement learning. In International Conference on Machine Learning. PMLR, 652--661.

[16]

Thorsten Joachims. 2002. Optimizing Search Engines Using Clickthrough Data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. 133--142.

Digital Library

[17]

Thorsten Joachims and Adith Swaminathan. 2016. Counterfactual Evaluation and Learning for Search, Recommendation and Ad Placement. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 1199--1201.

Digital Library

[18]

Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased Learning-to-rank with Biased Feedback. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. 781--789.

Digital Library

[19]

Haruka Kiyohara, Yuta Saito, Tatsuya Matsuhiro, Yusuke Narita, Nobuyuki Shimizu, and Yasuo Yamamoto. 2022. Doubly robust off-policy evaluation for ranking policies under the cascade behavior model. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 487--497.

Digital Library

[20]

Lihong Li, Wei Chu, John Langford, and Xuanhui Wang. 2011. Unbiased Offline Evaluation of Contextual-Bandit-based News Article Recommendation Algorithms. In Proceedings of the fourth ACM international conference on Web Search and Data Mining. 297--306.

Digital Library

[21]

Claudio Lucchese, Franco Maria Nardini, Rama Kumar Pasumarthi, Sebastian Bruch, Michael Bendersky, Xuanhui Wang, Harrie Oosterhuis, Rolf Jagerman, and Maarten de Rijke. 2019. Learning to rank in theory and practice: from gradient boosting to neural networks and unbiased learning. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1419--1420.

Digital Library

[22]

Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. 2020. Controlling fairness and bias in dynamic learning-to-rank. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval. 429--438.

Digital Library

[23]

Harrie Oosterhuis. 2020. Learning from User Interactions with Rankings: A Unification of the Field. Ph.,D. Dissertation. Informatics Institute, University of Amsterdam.

[24]

Harrie Oosterhuis. 2022a. Doubly-Robust Estimation for Unbiased Learning-to-Rank from Position-Biased Click Feedback. arXiv preprint arXiv:2203.17118 (2022).

[25]

Harrie Oosterhuis. 2022b. Reaching the End of Unbiasedness: Uncovering Implicit Limitations of Click-Based Learning to Rank. In Proceedings of the 2022 ACM SIGIR International Conference on the Theory of Information Retrieval. ACM.

Digital Library

[26]

Harrie Oosterhuis. 2023. Doubly Robust Estimation for Correcting Position Bias in Click Feedback for Unbiased Learning to Rank. ACM Transactions on Information Systems, Vol. 41, 3 (2023), 1--33.

Digital Library

[27]

Harrie Oosterhuis and Maarten de Rijke. 2018. Differentiable unbiased online learning to rank. In Proceedings of the 27th ACM international conference on information and knowledge management. 1293--1302.

Digital Library

[28]

Harrie Oosterhuis and Maarten de Rijke. 2020a. Policy-aware Unbiased Learning to Rank for Top-k Rankings. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 489--498.

Digital Library

[29]

Harrie Oosterhuis and Maarten de Rijke. 2020b. Taking the Counterfactual Online: Efficient and Unbiased Online Evaluation for Ranking. In Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval. 137--144.

Digital Library

[30]

Harrie Oosterhuis and Maarten de Rijke. 2021. Unifying Online and Counterfactual Learning to Rank: A Novel Counterfactual Estimator that Effectively Utilizes Online Interventions. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 463--471.

Digital Library

[31]

Harrie Oosterhuis, Rolf Jagerman, and Maarten de Rijke. 2020. Unbiased Learning to Rank: Counterfactual and Online Approaches. In Companion Proceedings of the Web Conference 2020. 299--300.

Digital Library

[32]

Zohreh Ovaisi, Ragib Ahsan, Yifan Zhang, Kathryn Vasilaky, and Elena Zheleva. 2020. Correcting for selection bias in learning-to-rank systems. In Proceedings of The Web Conference 2020. 1863--1873.

Digital Library

[33]

Zohreh Ovaisi, Kathryn Vasilaky, and Elena Zheleva. 2021. Propensity-Independent Bias Recovery in Offline Learning-to-Rank Systems. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1763--1767.

Digital Library

[34]

Yuta Saito. 2020. Doubly robust estimator for ranking metrics with post-click conversions. In Proceedings of the 14th ACM Conference on Recommender Systems. 92--100.

Digital Library

[35]

Yuta Saito and Thorsten Joachims. 2021. Counterfactual Learning and Evaluation for Recommender Systems: Foundations, Implementations, and Recent Advances. In Fifteenth ACM Conference on Recommender Systems. 828--830.

Digital Library

[36]

Yuta Saito and Thorsten Joachims. 2022a. Counterfactual Evaluation and Learning for Interactive Systems: Foundations, Implementations, and Recent Advances. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4824--4825.

Digital Library

[37]

Yuta Saito and Thorsten Joachims. 2022b. Off-policy evaluation for large action spaces via embeddings. arXiv preprint arXiv:2202.06317 (2022).

[38]

Mark Sanderson, Monica Lestari Paramita, Paul Clough, and Evangelos Kanoulas. 2010. Do User Preferences and Evaluation Measures Line Up?. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. 555--562.

Digital Library

[39]

Fatemeh Sarvi, Maria Heuss, Mohammad Aliannejadi, Sebastian Schelter, and Maarten de Rijke. 2021. Understanding and Mitigating the Effect of Outliers in Fair Ranking. arXiv preprint arXiv:2112.11251 (2021).

[40]

Anne Schuth, Harrie Oosterhuis, Shimon Whiteson, and Maarten de Rijke. 2016. Multileave gradient descent for fast online learning to rank. In proceedings of the ninth ACM international conference on web search and data mining. 457--466.

Digital Library

[41]

Ashudeep Singh and Thorsten Joachims. 2018. Fairness of exposure in rankings. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2219--2228.

Digital Library

[42]

Ashudeep Singh and Thorsten Joachims. 2019. Policy Learning for Fairness in Ranking. arXiv preprint arXiv:1902.04056 (2019).

[43]

Ashudeep Singh, David Kempe, and Thorsten Joachims. 2021. Fairness in ranking under uncertainty. Advances in Neural Information Processing Systems, Vol. 34 (2021), 11896--11908.

[44]

Mucun Tian, Chu Guo, Vito Claudio Ostuni, and Zhen Zhu. 2020. Counterfactual Learning to Rank using Heterogeneous Treatment Effect Estimation. ArXiv, Vol. abs/2007.09798 (2020).

[45]

Ali Vardasbi, Maarten de Rijke, and Ilya Markov. 2020a. Cascade model-based propensity estimation for counterfactual learning to rank. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2089--2092.

Digital Library

[46]

Ali Vardasbi, Harrie Oosterhuis, and Maarten de Rijke. 2020b. When Inverse Propensity Scoring does not Work: Affine Corrections for Unbiased Learning to Rank. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 1475--1484.

Digital Library

[47]

Ali Vardasbi, Fatemeh Sarvi, and Maarten de Rijke. 2022. Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Ranking. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (Madrid, Spain) (SIGIR '22). Association for Computing Machinery, New York, NY, USA, 715--725. https://doi.org/10.1145/3477495.3532045

Digital Library

[48]

Xuanhui Wang, Michael Bendersky, Donald Metzler, and Marc Najork. 2016. Learning to Rank with Selection Bias in Personal Search. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 115--124.

Digital Library

[49]

Xuanhui Wang, Nadav Golbandi, Michael Bendersky, Donald Metzler, and Marc Najork. 2018. Position Bias Estimation for Unbiased Learning to Rank in Personal Search. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. 610--618.

Digital Library

[50]

Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. 2019. Doubly robust joint learning for recommendation on data missing not at random. In International Conference on Machine Learning. PMLR, 6638--6647.

[51]

Xinwei Wu, Hechang Chen, Jiashu Zhao, Li He, Dawei Yin, and Yi Chang. 2021. Unbiased learning to rank in feeds recommendation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 490--498.

Digital Library

[52]

Himank Yadav, Zhengxiao Du, and Thorsten Joachims. 2021. Policy-Gradient Training of Fair and Unbiased Ranking Functions. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1044--1053.

Digital Library

[53]

Le Yan, Zhen Qin, Honglei Zhuang, Xuanhui Wang, Mike Bendersky, and Marc Najork. 2022. Revisiting two tower models for unbiased learning to rank. (2022).

[54]

Tao Yang and Qingyao Ai. 2021. Maximizing Marginal Fairness for Dynamic Learning to Rank. In Proceedings of the Web Conference 2021 (Ljubljana, Slovenia) (WWW '21). Association for Computing Machinery, New York, NY, USA, 137--145. https://doi.org/10.1145/3442381.3449901

Digital Library

[55]

Yisong Yue and Thorsten Joachims. 2009. Interactively optimizing information retrieval systems as a dueling bandits problem. In Proceedings of the 26th Annual International Conference on Machine Learning. 1201--1208.

Digital Library

[56]

Haiyuan Zhao, Jun Xu, Xiao Zhang, Guohao Cai, Zhenhua Dong, and Ji-Rong Wen. 2022. Unbiased Top-k Learning to Rank with Causal Likelihood Decomposition. arXiv preprint arXiv:2204.00815 (2022).

[57]

Zhi Zheng, Zhaopeng Qiu, Tong Xu, Xian Wu, Xiangyu Zhao, Enhong Chen, and Hui Xiong. 2022. CBR: Context Bias aware Recommendation for Debiasing User Modeling and Click Prediction. In Proceedings of the ACM Web Conference 2022. 2268--2276.

Digital Library

[58]

Honglei Zhuang, Zhen Qin, Xuanhui Wang, Michael Bendersky, Xinyu Qian, Po Hu, and Dan Chary Chen. 2021. Cross-positional attention for debiasing clicks. In Proceedings of the Web Conference 2021. 788--797.

Digital Library

Cited By

Deffayet RThonet THwang DLehoux VRenders Jde Rijke M(2024)SARDINE: Simulator for Automated Recommendation in Dynamic and Interactive EnvironmentsACM Transactions on Recommender Systems10.1145/36564812:3(1-34)Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1145/3656481
Shirokikh MShenbin IAlekseev AVolodkevich AVasilev ANikolenko S(2024)User Response Modeling in Recommender Systems: A SurveyJournal of Mathematical Sciences10.1007/s10958-024-07431-3285:2(255-293)Online publication date: 8-Nov-2024
https://doi.org/10.1007/s10958-024-07431-3
Gupta SOosterhuis Hde Rijke MYoshioka MKiseleva JAliannejadi M(2023)A Deep Generative Recommendation Method for Unbiased Learning from Implicit FeedbackProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3578337.3605114(87-93)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3578337.3605114

Index Terms

Recent Advances in the Foundations and Applications of Unbiased Learning to Rank
1. Information systems
  1. Information retrieval

Recommendations

Unbiased Learning to Rank: On Recent Advances and Practical Applications
WSDM '24: Proceedings of the 17th ACM International Conference on Web Search and Data Mining

Since its inception, the field of unbiased learning to rank (ULTR) has remained very active and has seen several impactful advancements in recent years. This tutorial provides both an introduction to the core concepts of the field and an overview of ...
Recent Advancements in Unbiased Learning to Rank
FIRE '23: Proceedings of the 15th Annual Meeting of the Forum for Information Retrieval Evaluation

Since its inception, the field of unbiased learning to rank (ULTR) has remained very active and has seen several impactful advancements in recent years. This tutorial provides both an introduction to the core concepts of the field and an overview of ...
Policy-Aware Unbiased Learning to Rank for Top-k Rankings
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

Counterfactual Learning to Rank (LTR) methods optimize ranking systems using logged user interactions that contain interaction biases. Existing methods are only unbiased if users are presented with all relevant items in every ranking. There is currently ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2023

3567 pages

ISBN:9781450394086

DOI:10.1145/3539618

General Chairs:
Hsin-Hsi Chen
National Taiwan University
,
Wei-Jou (Edward) Duh
National Taiwan University
,
Hen-Hsen Huang
Academia Sinica
,
Program Chairs:
Makoto P. Kato
Spotify
,
Josiane Mothe
Universite de Toulouse
,
Barbara Poblete
University of Chile and Amazon Visiting Academic

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2023

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

SIGIR '23

Sponsor:

SIGIR

SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 23 - 27, 2023

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
361
Total Downloads

Downloads (Last 12 months)222
Downloads (Last 6 weeks)27

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Deffayet RThonet THwang DLehoux VRenders Jde Rijke M(2024)SARDINE: Simulator for Automated Recommendation in Dynamic and Interactive EnvironmentsACM Transactions on Recommender Systems10.1145/36564812:3(1-34)Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1145/3656481
Shirokikh MShenbin IAlekseev AVolodkevich AVasilev ANikolenko S(2024)User Response Modeling in Recommender Systems: A SurveyJournal of Mathematical Sciences10.1007/s10958-024-07431-3285:2(255-293)Online publication date: 8-Nov-2024
https://doi.org/10.1007/s10958-024-07431-3
Gupta SOosterhuis Hde Rijke MYoshioka MKiseleva JAliannejadi M(2023)A Deep Generative Recommendation Method for Unbiased Learning from Implicit FeedbackProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3578337.3605114(87-93)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3578337.3605114

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten