research-article

Reformulating CTR Prediction: Learning Invariant Feature Interactions for Recommendation

Authors:

Yongdong ZhangAuthors Info & Claims

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 1386 - 1395

https://doi.org/10.1145/3539618.3591755

Published: 18 July 2023 Publication History

Abstract

Click-Through Rate (CTR) prediction plays a core role in recommender systems, serving as the final-stage filter to rank items for a user. The key to addressing the CTR task is learning feature interactions that are useful for prediction, which is typically achieved by fitting historical click data with the Empirical Risk Minimization (ERM) paradigm. Representative methods include Factorization Machines and Deep Interest Network, which have achieved wide success in industrial applications. However, such a manner inevitably learns unstable feature interactions, i.e., the ones that exhibit strong correlations in historical data but generalize poorly for future serving.

In this work, we reformulate the CTR task --- instead of pursuing ERM on historical data, we split the historical data chronologically into several periods (a.k.a, environments), aiming to learn feature interactions that are stable across periods. Such feature interactions are supposed to generalize better to predict future behavior data. Nevertheless, a technical challenge is that existing invariant learning solutions like Invariant Risk Minimization are not applicable, since the click data entangles both environment-invariant and environment-specific correlations. To address this dilemma, we propose Disentangled Invariant Learning (DIL) which disentangles feature embeddings to capture the two types of correlations separately. To improve the modeling efficiency, we further design LightDIL which performs the disentanglement at the higher level of the feature field. Extensive experiments demonstrate the effectiveness of DIL in learning stable feature interactions for CTR.

References

[1]

Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar. 2020. Invariant Risk Minimization Games. In International Conference on Machine Learning. PMLR, 145--155.

[2]

Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. 2019. Invariant risk minimization. arXiv preprint arXiv:1907.02893 (2019).

[3]

Jun-Hyun Bae, Inchul Choi, and Minho Lee. 2021. Meta-Learned Invariant Risk Minimization. arXiv preprint arXiv:2103.12947 (2021).

[4]

Weijie Bian, Kailun Wu, Lejian Ren, Qi Pi, Yujing Zhang, Can Xiao, Xiang-Rong Sheng, Yong-Nan Zhu, Zhangming Chan, Na Mou, et al. 2022. CAN: Feature Co-Action Network for Click-Through Rate Prediction. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 57--65.

Digital Library

[5]

Elliot Creager, Jörn-Henrik Jacobsen, and Richard Zemel. 2021. Environment Inference for Invariant Learning. In International Conference on Machine Learning. 2189--2200.

[6]

Xiaoyu Du, Zike Wu, Fuli Feng, Xiangnan He, and Jinhui Tang. 2022. Invariant Representation Learning for Multimedia Recommendation. In Proceedings of the 30th ACM International Conference on Multimedia. 619--628.

Digital Library

[7]

Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In Proceedings of the 34th International Conference on Machine Learning. 1126--1135.

[8]

Chen Gao, Yinfeng Li, Quanming Yao, Depeng Jin, and Yong Li. 2021. Progressive Feature Interaction Search for Deep Sparse Network. Advances in Neural Information Processing Systems, Vol. 34 (2021).

[9]

Chen Gao, Yu Zheng, Wenjie Wang, Fuli Feng, Xiangnan He, and Yong Li. 2022. Causal Inference in Recommender Systems: A Survey and Future Directions. arXiv preprint arXiv:2208.12397 (2022).

[10]

Clark Glymour, Kun Zhang, and Peter Spirtes. 2019. Review of causal discovery methods based on graphical models. Frontiers in genetics, Vol. 10 (2019), 524.

[11]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. 1725--1731.

[12]

Wei Guo, Rong Su, Renhao Tan, Huifeng Guo, Yingxue Zhang, Zhirong Liu, Ruiming Tang, and Xiuqiang He. 2021. Dual graph enhanced embedding neural network for ctr prediction. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 496--504.

Digital Library

[13]

F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context. Acm Transactions on Interactive Intelligent Systems, Vol. 5, 4 (2015), 1--19.

Digital Library

[14]

Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 355--364.

Digital Library

[15]

Yue He, Peng Cui, Jianxin Ma, Hao Zou, Xiaowei Wang, Hongxia Yang, and Philip S Yu. 2020. Learning Stable Graphs from Multiple Environments with Selection Bias. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2194--2202.

Digital Library

[16]

Yue He, Zimu Wang, Peng Cui, Hao Zou, Yafeng Zhang, Qiang Cui, and Yong Jiang. 2022. CausPref: Causal Preference Learning for Out-of-Distribution Recommendation. In Proceedings of the ACM Web Conference 2022. 410--421.

Digital Library

[17]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[18]

David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. 2021. Out-of-distribution Generalization via Risk Extrapolation (rex). In International Conference on Machine Learning. PMLR, 5815--5826.

[19]

Feng Li, Zhenrui Chen, Pengjie Wang, Yi Ren, Di Zhang, and Xiaoyu Zhu. 2019. Graph intention network for click-through rate prediction in sponsored search. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 961--964.

Digital Library

[20]

Yunqi Li, Hanxiong Chen, Juntao Tan, and Yongfeng Zhang. 2022. Causal factorization machine for robust recommendation. In Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries. 1--9.

Digital Library

[21]

Fengming Lin, Xiaolei Fang, and Zheming Gao. 2022. Distributionally robust optimization: A review on theory and applications. Numerical Algebra, Control and Optimization, Vol. 12, 1 (2022), 159--212.

[22]

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal Loss for Dense Object Detection. In Proceedings of the IEEE International Conference on Computer Vision. 2980--2988.

[23]

Bin Liu, Ruiming Tang, Yingzhi Chen, Jinkai Yu, Huifeng Guo, and Yuzhou Zhang. 2019. Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction. In The World Wide Web Conference. 1119--1129.

Digital Library

[24]

Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, and Yong Yu. 2020. Autofis: Automatic feature interaction selection in factorization models for click-through rate prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2636--2645.

Digital Library

[25]

Jiashuo Liu, Zheyuan Hu, Peng Cui, Bo Li, and Zheyan Shen. 2021. Heterogeneous Risk Minimization. In International Conference on Machine Learning. 6804--6814.

[26]

Qiang Liu, Feng Yu, Shu Wu, and Liang Wang. 2015. A convolutional click prediction model. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 1743--1746.

Digital Library

[27]

Erxue Min, Yu Rong, Tingyang Xu, Yatao Bian, Da Luo, Kangyi Lin, Junzhou Huang, Sophia Ananiadou, and Peilin Zhao. 2022. Neighbour Interaction based Click-Through Rate Prediction via Graph-masked Transformer. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 353--362.

Digital Library

[28]

Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu. 2018. Field-Weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising. In Proceedings of the 2018 World Wide Web Conference. 1349--1357.

Digital Library

[29]

Vihari Piratla, Praneeth Netrapalli, and Sunita Sarawagi. 2022. Focus on the Common Good: Group Distributional Robustness Follows. In International Conference on Learning Representations.

[30]

Yanru Qu, Bohui Fang, Weinan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, Yong Yu, and Xiuqiang He. 2018. Product-based neural networks for user response prediction over multi-field categorical data. ACM Transactions on Information Systems (TOIS), Vol. 37, 1 (2018), 1--35.

Digital Library

[31]

Steffen Rendle. 2010. Factorization machines. In 2010 IEEE International Conference on Data Mining. IEEE, 995--1000.

Digital Library

[32]

Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. 2019. Distributionally Robust Neural Networks. In International Conference on Learning Representations.

[33]

Arindam Sarkar, Dipankar Das, Vivek Sembium, and Prakash Mandayam Comar. 2022. Dual Attentional Higher Order Factorization Machines. In Proceedings of the 16th ACM Conference on Recommender Systems. 378--388.

Digital Library

[34]

Zheyan Shen, Peng Cui, Kun Kuang, Bo Li, and Peixuan Chen. 2018. Causally Regularized Learning with Agnostic Data Selection Bias. In Proceedings of the 26th ACM International Conference on Multimedia. 411--419.

Digital Library

[35]

Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. 2021. Towards Out-of-distribution Generalization: A Survey. arXiv preprint arXiv:2108.13624 (2021).

[36]

Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, and Xia Hu. 2020. Towards automated neural interaction discovery for click-through rate prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 945--955.

Digital Library

[37]

Yixin Su, Yunxiang Zhao, Sarah Erfani, Junhao Gan, and Rui Zhang. 2022. Detecting Arbitrary Order Beneficial Feature Interactions for Recommender Systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1676--1686.

Digital Library

[38]

Wenjie Wang, Xinyu Lin, Fuli Feng, Xiangnan He, Min Lin, and Tat-Seng Chua. 2022b. Causal Representation Learning for Out-of-Distribution Recommendation. In Proceedings of the ACM Web Conference 2022. 3562--3571.

Digital Library

[39]

Yifan Wang, Yifang Qin, Fang Sun, Bo Zhang, Xuyang Hou, Ke Hu, Jia Cheng, Jun Lei, and Ming Zhang. 2022c. DisenCTR: Dynamic Graph-based Disentangled Representation for Click-Through Rate Prediction. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2314--2318.

Digital Library

[40]

Zimu Wang, Yue He, Jiashuo Liu, Wenchao Zou, Philip S Yu, and Peng Cui. 2022a. Invariant Preference Learning for General Debiasing in Recommendation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1969--1978.

Digital Library

[41]

Qitian Wu, Hengrui Zhang, Junchi Yan, and David Wipf. 2022. Handling Distribution Shifts on Graphs: An Invariance Perspective. In International Conference on Learning Representations.

[42]

Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. 3119--3125.

[43]

Weinan Zhang, Jiarui Qin, Wei Guo, Ruiming Tang, and Xiuqiang He. 2021b. Deep Learning for Click-Through Rate Estimation. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. 4695--4703.

[44]

Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021a. Causal intervention for leveraging popularity bias in recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 11--20.

Digital Library

[45]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1059--1068.

Digital Library

[46]

Guanghui Zhu, Feng Cheng, Defu Lian, Chunfeng Yuan, and Yihua Huang. 2022. NAS-CTR: Efficient Neural Architecture Search for Click-Through Rate Prediction. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 332--342.

Digital Library

Cited By

Di YShi HWang XMa RLiu Y(2025)Federated Recommender System Based on Diffusion Augmentation and Guided DenoisingACM Transactions on Information Systems10.1145/368857043:2(1-36)Online publication date: 17-Jan-2025
https://dl.acm.org/doi/10.1145/3688570
Zhang YHu ZBai YWu JWang QFeng F(2024)Recommendation Unlearning via Influence FunctionACM Transactions on Recommender Systems10.1145/37017633:2(1-23)Online publication date: 23-Dec-2024
https://dl.acm.org/doi/10.1145/3701763
Lv ZHe SZhan TZhang SZhang WChen JZhao ZWu FCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Semantic Codebook Learning for Dynamic Recommendation ModelsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680574(9611-9620)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680574
Show More Cited By

Index Terms

Reformulating CTR Prediction: Learning Invariant Feature Interactions for Recommendation
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

A novel graph-based feature interaction model for click-through rate prediction
Abstract
Click-through rate (CTR) prediction is a crucial issue in recommender systems. In addition, data sparsity is a notable challenge for recommender systems compared to other applications. To overcome it, many learning-based models are studied to ...
Highlights
- A novel feature interaction model based on graph and FM is studied.
- Graph and FM make feature interactions flexible and learnable.
- Three existing CTR prediction methods are improved by our feature interaction model.
- Experiments ...
Warming Up Cold-Start CTR Prediction by Learning Item-Specific Feature Interactions
KDD '24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

In recommendation systems, new items are continuously introduced, initially lacking interaction records but gradually accumulating them over time. Accurately predicting the click-through rate (CTR) for these items is crucial for enhancing both revenue ...
Invariant Representation Learning for Multimedia Recommendation
MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Multimedia recommendation forms a personalized ranking task with multimedia content representations which are mostly extracted via generic encoders. However, the generic representations introduce spurious correlations --- the meaningless correlation ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2023

3567 pages

ISBN:9781450394086

DOI:10.1145/3539618

General Chairs:
Hsin-Hsi Chen
National Taiwan University
,
Wei-Jou (Edward) Duh
National Taiwan University
,
Hen-Hsen Huang
Academia Sinica
,
Program Chairs:
Makoto P. Kato
Spotify
,
Josiane Mothe
Universite de Toulouse
,
Barbara Poblete
University of Chile and Amazon Visiting Academic

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key Research and Development Program of China
National Natural Science Foundation of China

Conference

SIGIR '23

Sponsor:

SIGIR

SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 23 - 27, 2023

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
247
Total Downloads

Downloads (Last 12 months)115
Downloads (Last 6 weeks)4

Reflects downloads up to 09 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Di YShi HWang XMa RLiu Y(2025)Federated Recommender System Based on Diffusion Augmentation and Guided DenoisingACM Transactions on Information Systems10.1145/368857043:2(1-36)Online publication date: 17-Jan-2025
https://dl.acm.org/doi/10.1145/3688570
Zhang YHu ZBai YWu JWang QFeng F(2024)Recommendation Unlearning via Influence FunctionACM Transactions on Recommender Systems10.1145/37017633:2(1-23)Online publication date: 23-Dec-2024
https://dl.acm.org/doi/10.1145/3701763
Lv ZHe SZhan TZhang SZhang WChen JZhao ZWu FCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Semantic Codebook Learning for Dynamic Recommendation ModelsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680574(9611-9620)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680574
Bai YZhang YFeng FLu JZang XLei CSong YBaeza-Yates RBonchi F(2024)GradCraft: Elevating Multi-task Recommendations through Holistic Gradient CraftingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671585(4774-4783)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671585
Bai ZWu NCai FZhu XXiong YSerra ESpezzano F(2024)Aligning Large Language Model with Direct Multi-Preference Optimization for RecommendationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679611(76-86)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679611
Ouyang WDong RTao RLiu XHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)FedUD: Exploiting Unaligned Data for Cross-Platform Federated Click-Through Rate PredictionProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657941(2416-2420)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657941
Bao KZhang JZhang YWang WFeng FHe X(2023)TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with RecommendationProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608857(1007-1014)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608857
Zhang YChen HDuh WHuang HKato MMothe JPoblete B(2023)Towards Trustworthy Recommender System: A Faithful and Responsible Recommendation PerspectiveProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591798(3497-3497)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591798
Bao KZhang JWang WZhang YYang ZLuo YChen CFeng FTian Q(undefined)A Bi-Step Grounding Paradigm for Large Language Models in Recommendation SystemsACM Transactions on Recommender Systems10.1145/3716393
https://dl.acm.org/doi/10.1145/3716393

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten