research-article

Equivariant Contrastive Learning for Sequential Recommendation

Authors:

Sunghun KimAuthors Info & Claims

RecSys '23: Proceedings of the 17th ACM Conference on Recommender Systems

Pages 129 - 140

https://doi.org/10.1145/3604915.3608786

Published: 14 September 2023 Publication History

Abstract

Contrastive learning (CL) benefits the training of sequential recommendation models with informative self-supervision signals. Existing solutions apply general sequential data augmentation strategies to generate positive pairs and encourage their representations to be invariant. However, due to the inherent properties of user behavior sequences, some augmentation strategies, such as item substitution, can lead to changes in user intent. Learning indiscriminately invariant representations for all augmentation strategies might be sub-optimal. Therefore, we propose Equivariant Contrastive Learning for Sequential Recommendation (ECL-SR), which endows SR models with great discriminative power, making the learned user behavior representations sensitive to invasive augmentations (e.g., item substitution) and insensitive to mild augmentations (e.g., feature-level dropout masking). In detail, we use the conditional discriminator to capture differences in behavior due to item substitution, which encourages the user behavior encoder to be equivariant to invasive augmentations. Comprehensive experiments on four benchmark datasets show that the proposed ECL-SR framework achieves competitive performance compared to state-of-the-art SR models. The source code is available at https://github.com/Tokkiu/ECL.

References

[1]

Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. 2021. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478 (2021).

[2]

Jianxin Chang, Chen Gao, Yu Zheng, Yiqun Hui, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. 2021. Sequential Recommendation with Graph Neural Networks. In SIGIR. 378–387.

[3]

Xu Chen, Hongteng Xu, Yongfeng Zhang, Jiaxi Tang, Yixin Cao, Zheng Qin, and Hongyuan Zha. 2018. Sequential recommendation with user memory networks. In WSDM. 108–116.

[4]

Yongjun Chen, Jia Li, and Caiming Xiong. 2022. ELECRec: Training Sequential Recommenders as Discriminators. CoRR abs/2204.02011 (2022).

[5]

Yongjun Chen, Zhiwei Liu, Jia Li, Julian J. McAuley, and Caiming Xiong. 2022. Intent Contrastive Learning for Sequential Recommendation. In WWW. ACM, 2172–2182.

[6]

Dading Chong, Helin Wang, Peilin Zhou, and Qingcheng Zeng. 2023. Masked spectrogram prediction for self-supervised audio pre-training. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5.

[7]

Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljacic, Shang-Wen Li, Wen-tau Yih, Yoon Kim, and James R. Glass. 2022. DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings. CoRR abs/2204.10298 (2022).

[8]

Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555 (2020).

[9]

Taco Cohen and Max Welling. 2016. Group equivariant convolutional networks. In International conference on machine learning. PMLR, 2990–2999.

[10]

Alexander Dallmann, Daniel Zoller, and Andreas Hotho. 2021. A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models. In RecSys. 505–514.

[11]

Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, and Marin Soljacic. 2021. Equivariant Contrastive Learning. CoRR abs/2111.00899 (2021).

[12]

James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, Taylor Van Vleet, Ullas Gargi, Sujoy Gupta, Yu He, Mike Lambert, Blake Livingston, 2010. The YouTube video recommendation system. In Proceedings of the fourth ACM conference on Recommender systems. 293–296.

Digital Library

[13]

Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple Contrastive Learning of Sentence Embeddings. In EMNLP (1). ACL, 6894–6910.

[14]

Priyanka Gupta, Diksha Garg, Pankaj Malhotra, Lovekesh Vig, and Gautam Shroff. 2019. NISER: Normalized item and session representations to handle popularity bias. (Sept. 2019). arxiv:1909.04276 [cs.IR]

[15]

Ido Guy, Naama Zwerdling, Inbal Ronen, David Carmel, and Erel Uziel. 2010. Social media recommendation based on people and tags. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. 194–201.

Digital Library

[16]

Ruining He and Julian McAuley. 2016. Fusing similarity models with markov chains for sparse sequential recommendation. In ICDM. IEEE, 191–200.

[17]

Zhankui He, Handong Zhao, Zhe Lin, Zhaowen Wang, Ajinkya Kale, and Julian McAuley. 2021. Locker: Locally Constrained Self-Attentive Sequential Recommendation. In CIKM. 3088–3092.

[18]

Balázs Hidasi and Alexandros Karatzoglou. 2018. Recurrent neural networks with top-k gains for session-based recommendations. In CIKM. 843–852.

[19]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).

[20]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based Recommendations with Recurrent Neural Networks. In ICLR.

[21]

Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y Chang. 2018. Improving sequential recommendation with knowledge-enhanced memory networks. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 505–514.

Digital Library

[22]

Hyunwoo Hwangbo, Yang Sok Kim, and Kyung Jin Cha. 2018. Recommendation system development for fashion retail e-commerce. Electronic Commerce Research and Applications 28 (2018), 94–101.

Digital Library

[23]

Minguk Kang and Jaesik Park. 2020. Contragan: Contrastive learning for conditional image generation. NeurIPS 33 (2020), 21357–21369.

[24]

Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In ICDM. IEEE, 197–206.

[25]

Walid Krichene and Steffen Rendle. 2020. On sampled metrics for item recommendation. In SIGKDD. 1748–1757.

[26]

Jiacheng Li, Yujie Wang, and Julian McAuley. 2020. Time interval aware self-attention for sequential recommendation. In WSDM. 322–330.

[27]

Yicong Li, Hongxu Chen, Xiangguo Sun, Zhenchao Sun, Lin Li, Lizhen Cui, Philip S. Yu, and Guandong Xu. 2021. Hyperbolic Hypergraphs for Sequential Recommendation. In CIKM. ACM, 988–997.

[28]

Yang Li, Tong Chen, Peng-Fei Zhang, and Hongzhi Yin. 2021. Lightweight Self-Attentive Sequential Recommendation. In CIKM. 967–977.

[29]

Junling Liu, Chao Liu, Renjie Lv, Kang Zhou, and Yan Zhang. 2023. Is chatgpt a good recommender? a preliminary study. arXiv preprint arXiv:2304.10149 (2023).

[30]

Zhiwei Liu, Yongjun Chen, Jia Li, Philip S. Yu, Julian J. McAuley, and Caiming Xiong. 2021. Contrastive Self-supervised Sequential Recommendation with Robust Augmentation. CoRR abs/2108.06479 (2021).

[31]

Chen Ma, Liheng Ma, Yingxue Zhang, Jianing Sun, Xue Liu, and Mark Coates. 2020. Memory augmented graph neural networks for sequential recommendation. In AAAI, Vol. 34. 5045–5052.

[32]

Julian J. McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-Based Recommendations on Styles and Substitutes. In SIGIR, Ricardo Baeza-Yates, Mounia Lalmas, Alistair Moffat, and Berthier A. Ribeiro-Neto (Eds.). ACM, 43–52. https://doi.org/10.1145/2766462.2767755

Digital Library

[33]

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).

[34]

Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. 2022. Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation. In WSDM. ACM, 813–823.

[35]

Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, and Paolo Cremonesi. 2017. Personalizing session-based recommendations with hierarchical recurrent neural networks. In RecSys. 130–137.

[36]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (Montreal, Quebec, Canada) (UAI ’09). AUAI Press, Arlington, Virginia, USA, 452–461.

Digital Library

[37]

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In WWW. 811–820.

[38]

Aaqib Saeed, David Grangier, and Neil Zeghidour. 2021. Contrastive learning of general-purpose audio representations. In ICASSP. IEEE, 3875–3879.

[39]

Wenzhuo Song, Shoujin Wang, Yan Wang, and Shengsheng Wang. 2021. Next-item recommendations in short sessions. In Proceedings of the 15th ACM Conference on Recommender Systems. 282–291.

Digital Library

[40]

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM. 1441–1450.

[41]

Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM. 565–573.

[42]

Feng Wang and Huaping Liu. 2021. Understanding the behaviour of contrastive loss. In CVPR. 2495–2504.

[43]

Lei Wang, Ee-Peng Lim, Zhiwei Liu, and Tianxiang Zhao. 2022. Explanation Guided Contrastive Learning for Sequential Recommendation. (Sept. 2022). arxiv:2209.01347 [cs.IR]

[44]

Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z. Sheng, and Mehmet Orgun. 2019. Sequential recommender systems: challenges, progress and prospects. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. AAAI Press, 6332–6338.

[45]

Shoujin Wang, Gabriella Pasi, Liang Hu, and Longbing Cao. 2020. The Era of Intelligent Recommendation: Editorial on Intelligent Recommendation with Advanced AI and Learning. IEEE Intelligent Systems 35, 05 (2020), 3–6.

[46]

Shoujin Wang, Yan Wang, Fikret Sivrikaya, Sahin Albayrak, and Vito Walter Anelli. 2023. Data science for next-generation recommender systems. International Journal of Data Science and Analytics (2023), 1–11.

[47]

Shoujin Wang, Xiaofei Xu, Xiuzhen Zhang, Yan Wang, and Wenzhuo Song. 2022. Veracity-aware and event-driven personalized news recommendation for fake news mitigation. In Proceedings of the ACM Web Conference 2022. 3673–3684.

Digital Library

[48]

Shoujin Wang, Xiuzhen Zhang, Yan Wang, Huan Liu, and Francesco Ricci. 2022. Trustworthy Recommender Systems. arXiv preprint arXiv:2208.06265 (2022).

[49]

Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James Sharpnack. 2020. SSE-PT: Sequential recommendation via personalized transformer. In RecSys. 328–337.

[50]

Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Bolin Ding, and Bin Cui. 2020. Contrastive learning for sequential recommendation. arXiv preprint arXiv:2010.14395 (2020).

[51]

Yueqi Xie, Jingqi Gao, Peilin Zhou, Qichen Ye, Yining Hua, Jaeboum Kim, Fangzhao Wu, and Sunghun Kim. 2023. Rethinking Multi-Interest Learning for Candidate Matching in Recommender Systems. arXiv preprint arXiv:2302.14532 (2023).

[52]

Yueqi Xie, Peilin Zhou, and Sunghun Kim. 2022. Decoupled Side Information Fusion for Sequential Recommendation. In SIGIR.

[53]

Ghim-Eng Yap, Xiao-Li Li, and Philip S Yu. 2012. Effective next-items recommendation via personalized sequential pattern mining. In International conference on database systems for advanced applications. Springer, 48–64.

Digital Library

[54]

Chenyu You, Ruihan Zhao, Lawrence H Staib, and James S Duncan. 2022. Momentum contrastive voxel-wise representation learning for semi-supervised volumetric medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 639–652.

Digital Library

[55]

Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Jundong Li, and Zi Huang. 2022. Self-Supervised Learning for Recommender Systems: A Survey. CoRR abs/2203.15876 (2022).

[56]

Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, and Xiangnan He. 2019. A simple convolutional generative network for next item recommendation. In WSDM. 582–590.

[57]

Jiahao Yuan, Zihan Song, Mingyou Sun, Xiaoling Wang, and Wayne Xin Zhao. 2021. Dual Sparse Attention Network For Session-based Recommendation. In AAAI.

[58]

Xu Yuan, Dongsheng Duan, Lingling Tong, Lei Shi, and Cheng Zhang. 2021. ICAI-SR: Item Categorical Attribute Integrated Sequential Recommendation. In SIGIR. 1687–1691.

[59]

Peiyan Zhang, Jiayan Guo, Chaozhuo Li, Yueqi Xie, Jaeboum Kim, Yan Zhang, Xing Xie, Haohan Wang, and Sunghun Kim. 2022. Efficiently Leveraging Multi-level User Intent for Session-based Recommendation via Atten-Mixer Network. arXiv preprint arXiv:2206.12781 (2022).

[60]

Wayne Xin Zhao, Shanlei Mu, Yupeng Hou, Zihan Lin, Yushuo Chen, Xingyu Pan, Kaiyuan Li, Yujie Lu, Hui Wang, Changxin Tian, 2021. Recbole: Towards a unified, comprehensive and efficient framework for recommendation algorithms. In CIKM. 4653–4664.

[61]

Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In CIKM. 1893–1902.

Cited By

Niu JZhou WLuo FZhang YZeng JWen J(2025)Intent-guided Bilateral Long and Short-Term Information Mining with Contrastive Learning for Sequential RecommendationIEEE Transactions on Services Computing10.1109/TSC.2024.3520868(1-15)Online publication date: 2025
https://doi.org/10.1109/TSC.2024.3520868
Huang ZZhu YDou ZWen J(2025)CAGS: Context-Aware Document Ranking With Contrastive Graph SamplingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.349199637:1(89-101)Online publication date: Jan-2025
https://doi.org/10.1109/TKDE.2024.3491996
Cui ZWu HHe BCheng JMa CSerra ESpezzano F(2024)Context Matters: Enhancing Sequential Recommendation with Context-aware Diffusion-based Contrastive LearningProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679655(404-414)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679655
Show More Cited By

Index Terms

Equivariant Contrastive Learning for Sequential Recommendation
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
    1. Redundancy
  2. Embedded and cyber-physical systems
    1. Embedded systems
    2. Robotics
2. Networks
  1. Network properties
    1. Network reliability

Recommendations

Sequential Recommendation with Diverse Supervised Contrastive Views
Advanced Data Mining and Applications
Abstract
Recently, contrastive learning has been used in sequential recommendation to address data sparsity issue. Among various contrastive learning methods, supervised contrastive learning has shown excellent performance by utilizing supervised ...
Multi-interest sequential recommendation with contrastive learning and temporal analysis
Abstract
Sequential recommendation systems aim to forecast the subsequent item of interest to users by analyzing their historical behaviors. While existing approaches, which employ attention mechanisms, have significantly advanced by capturing users’ ...
Highlights
- Proposing a multi-interest recommendation approach that incorporates user preference features and temporal characteristics at the user level.
- Introducing a time-period module to capture short-term changes in user interest.
- ...
ELECRec: Training Sequential Recommenders as Discriminators
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Sequential recommendation is often considered as a generative task, i.e., training a sequential encoder to generate the next item of a user's interests based on her historical interacted items. Despite their prevalence, these methods usually require ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

RecSys '23: Proceedings of the 17th ACM Conference on Recommender Systems

September 2023

1406 pages

ISBN:9798400702419

DOI:10.1145/3604915

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

RecSys '23

Sponsor:

RecSys '23: Seventeenth ACM Conference on Recommender Systems

September 18 - 22, 2023

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
776
Total Downloads

Downloads (Last 12 months)306
Downloads (Last 6 weeks)18

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Niu JZhou WLuo FZhang YZeng JWen J(2025)Intent-guided Bilateral Long and Short-Term Information Mining with Contrastive Learning for Sequential RecommendationIEEE Transactions on Services Computing10.1109/TSC.2024.3520868(1-15)Online publication date: 2025
https://doi.org/10.1109/TSC.2024.3520868
Huang ZZhu YDou ZWen J(2025)CAGS: Context-Aware Document Ranking With Contrastive Graph SamplingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.349199637:1(89-101)Online publication date: Jan-2025
https://doi.org/10.1109/TKDE.2024.3491996
Cui ZWu HHe BCheng JMa CSerra ESpezzano F(2024)Context Matters: Enhancing Sequential Recommendation with Context-aware Diffusion-based Contrastive LearningProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679655(404-414)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679655
Zhou PHuang YXie YGao JWang SKim JKim SChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Is Contrastive Learning Necessary? A Study of Data Augmentation vs Contrastive Learning in Sequential RecommendationProceedings of the ACM Web Conference 202410.1145/3589334.3645661(3854-3863)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645661
You DLee K(2024)Context-Aware Diffusion-based Sequential Recommendation2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10826051(670-679)Online publication date: 15-Dec-2024
https://doi.org/10.1109/BigData62323.2024.10826051
Zhu RZhu W(2024)Sequential Recommendation Based on Attention Mechanism and Fourier Transform2024 3rd International Conference on Artificial Intelligence, Internet of Things and Cloud Computing Technology (AIoTC)10.1109/AIoTC63215.2024.10748274(219-223)Online publication date: 13-Sep-2024
https://doi.org/10.1109/AIoTC63215.2024.10748274
Huang BYu WXie RLuo JXiao JHuang J(2024)Lorentz equivariant model for knowledge-enhanced hyperbolic collaborative filteringKnowledge-Based Systems10.1016/j.knosys.2024.111590291:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111590
Chen SChen JWan S(2024)Efficient weighted sequential pattern miningExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122703243:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122703
Wang YShi XZhao X(2024)MLLM4Rec : multimodal information enhancing LLM for sequential recommendationJournal of Intelligent Information Systems10.1007/s10844-024-00915-3Online publication date: 30-Dec-2024
https://doi.org/10.1007/s10844-024-00915-3
Sang LHu YZhang YZhang Y(2024)Multi-view denoising contrastive learning for bundle recommendationApplied Intelligence10.1007/s10489-024-05825-z54:23(12332-12346)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s10489-024-05825-z
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten