research-article

Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation

Authors:

Yue DingAuthors Info & Claims

WWW '21: Proceedings of the Web Conference 2021

Pages 449 - 459

https://doi.org/10.1145/3442381.3449873

Published: 03 June 2021 Publication History

Abstract

Sequential recommendation as an emerging topic has attracted increasing attention due to its important practical significance. Models based on deep learning and attention mechanism have achieved good performance in sequential recommendation. Recently, the generative models based on Variational Autoencoder (VAE) have shown the unique advantage in collaborative filtering. In particular, the sequential VAE model as a recurrent version of VAE can effectively capture temporal dependencies among items in user sequence and perform sequential recommendation. However, VAE-based models suffer from a common limitation that the representational ability of the obtained approximate posterior distribution is limited, resulting in lower quality of generated samples. This is especially true for generating sequences. To solve the above problem, in this work, we propose a novel method called Adversarial and Contrastive Variational Autoencoder (ACVAE) for sequential recommendation. Specifically, we first introduce the adversarial training for sequence generation under the Adversarial Variational Bayes (AVB) framework, which enables our model to generate high-quality latent variables. Then, we employ the contrastive loss. The latent variables will be able to learn more personalized and salient characteristics by minimizing the contrastive loss. Besides, when encoding the sequence, we apply a recurrent and convolutional structure to capture global and local relationships in the sequence. Finally, we conduct extensive experiments on four real-world datasets. The experimental results show that our proposed ACVAE model outperforms other state-of-the-art methods.

References

[1]

Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. 2016. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410(2016).

[2]

Dong-Kyu Chae, Jin-Soo Kang, Sang-Wook Kim, and Jung-Tae Lee. 2018. Cfgan: A generic collaborative filtering framework based on generative adversarial networks. In Proceedings of the 27th ACM international conference on information and knowledge management. 137–146.

Digital Library

[3]

Ruining He and Julian McAuley. 2016. Fusing similarity models with markov chains for sparse sequential recommendation. In 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE, 191–200.

[4]

Xiangnan He, Zhankui He, Xiaoyu Du, and Tat-Seng Chua. 2018. Adversarial personalized ranking for recommendation. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 355–364.

Digital Library

[5]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939(2015).

[6]

Balázs Hidasi, Massimo Quadrana, Alexandros Karatzoglou, and Domonkos Tikk. 2016. Parallel recurrent neural network architectures for feature-rich session-based recommendations. In Proceedings of the 10th ACM conference on recommender systems. 241–248.

Digital Library

[7]

Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2016. beta-vae: Learning basic visual concepts with a constrained variational framework. (2016).

[8]

R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. 2018. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670(2018).

[9]

Ferenc Huszár. 2017. Variational inference using implicit distributions. arXiv preprint arXiv:1702.08235(2017).

[10]

Daniel Im Im, Sungjin Ahn, Roland Memisevic, and Yoshua Bengio. 2017. Denoising criterion for variational auto-encoding framework. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31.

[11]

Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 197–206.

[12]

Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings.

[13]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.

Digital Library

[14]

Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 World Wide Web Conference. 689–698.

Digital Library

[15]

Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. 2019. Learning disentangled representations for recommendation. In Advances in Neural Information Processing Systems. 5711–5722.

[16]

Lars M. Mescheder, Sebastian Nowozin, and Andreas Geiger. 2017. Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks. In Proceedings of the 34th International Conference on Machine Learning. 2391–2400.

[17]

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748(2018).

[18]

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World wide web. 811–820.

Digital Library

[19]

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning. PMLR, 1278–1286.

[20]

Noveen Sachdeva, Giuseppe Manco, Ettore Ritacco, and Vikram Pudi. 2019. Sequential variational autoencoders for collaborative filtering. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. 600–608.

Digital Library

[21]

Ilya Shenbin, Anton Alekseev, Elena Tutubalina, Valentin Malykh, and Sergey I Nikolenko. 2020. RecVAE: A New Variational Autoencoder for Top-N Recommendations with Implicit Feedback. In Proceedings of the 13th International Conference on Web Search and Data Mining. 528–536.

Digital Library

[22]

Qingquan Song, Shiyu Chang, and Xia Hu. 2019. Coupled Variational Recurrent Collaborative Filtering. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 335–343.

Digital Library

[23]

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1441–1450.

Digital Library

[24]

Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. 565–573.

Digital Library

[25]

Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang. 2017. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. 515–524.

Digital Library

[26]

Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z. Sheng, and Mehmet A. Orgun. 2019. Sequential Recommender Systems: Challenges, Progress and Prospects. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. 6332–6338.

[27]

Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J Smola, and How Jing. 2017. Recurrent recommender networks. In Proceedings of the tenth ACM international conference on web search and data mining. 495–503.

Digital Library

[28]

Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Jiajie Xu, Victor S Sheng S. Sheng, Zhiming Cui, Xiaofang Zhou, and Hui Xiong. 2019. Recurrent convolutional neural network for sequential recommendation. In The World Wide Web Conference. 3398–3404.

Digital Library

[29]

Haochao Ying, Fuzhen Zhuang, Fuzheng Zhang, Yanchi Liu, Guandong Xu, Xing Xie, Hui Xiong, and Jian Wu. 2018. Sequential recommender system based on hierarchical attention network. In IJCAI International Joint Conference on Artificial Intelligence.

[30]

Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. Seqgan: Sequence generative adversarial nets with policy gradient. In Thirty-first AAAI conference on artificial intelligence.

Digital Library

[31]

Xianwen Yu, Xiaoning Zhang, Yang Cao, and Min Xia. 2019. VAEGAN: A Collaborative Filtering Framework based on Adversarial Variational Autoencoders. In IJCAI. 4206–4212.

Cited By

Niu YXing XJia ZLiu RXin M(2025)Implicit local–global feature extraction for diffusion sequence recommendationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109471139(109471)Online publication date: Jan-2025
https://doi.org/10.1016/j.engappai.2024.109471
Zhang LZhang XLi CZhou ZLiu JHuang FZhang XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Mitigating Social Hazards: Early Detection of Fake News via Diffusion-Guided Propagation Path GenerationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681087(2842-2851)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681087
Liang SPan Zliu wYin Jde Rijke M(2024)A Survey on Variational Autoencoders in Recommender SystemsACM Computing Surveys10.1145/366336456:10(1-40)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3663364
Show More Cited By

Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

Wasserstein Adversarial Variational Autoencoder for Sequential Recommendation
Web and Big Data
Abstract
Variational autoencoders (VAEs) have shown unique advantages as a generative model for sequence recommendation. The core of VAEs is the reconstruction of error targets through similarity metrics to provide a supervised signal for training. However,...
Contrastive Generator Generative Adversarial Networks for Sequential Recommendation
Web and Big Data
Abstract
Generally, sequential recommendation refers to the task of training a sequential generative model that can generate items of interest for users based on their historical interactions. However, these methods have some drawbacks as they require an ...
Generative Adversarial Networks Based on Contrastive Learning for Sequential Recommendation
Web and Big Data
Abstract
Generative Adversarial Networks(GAN) has made key breakthroughs in computer vision and other fields, so some scholars have tried to apply it to sequential recommendation. However, the recommendation performance of GAN-based algorithms is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '21: Proceedings of the Web Conference 2021

April 2021

4054 pages

ISBN:9781450383127

DOI:10.1145/3442381

Editors:
Jure Leskovec
Stanford
,
Marko Grobelnik
Jožef Stefan Institute
,
Marc Najork
Google
,
Jie Tang
Tsinghua University
,
Leila Zia
Wikimedia Foundation

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '21

Sponsor:

SIGWEB

WWW '21: The Web Conference 2021

April 19 - 23, 2021

Ljubljana, Slovenia

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

52
Total Citations
View Citations
1,119
Total Downloads

Downloads (Last 12 months)158
Downloads (Last 6 weeks)10

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Niu YXing XJia ZLiu RXin M(2025)Implicit local–global feature extraction for diffusion sequence recommendationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109471139(109471)Online publication date: Jan-2025
https://doi.org/10.1016/j.engappai.2024.109471
Zhang LZhang XLi CZhou ZLiu JHuang FZhang XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Mitigating Social Hazards: Early Detection of Fake News via Diffusion-Guided Propagation Path GenerationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681087(2842-2851)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681087
Liang SPan Zliu wYin Jde Rijke M(2024)A Survey on Variational Autoencoders in Recommender SystemsACM Computing Surveys10.1145/366336456:10(1-40)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3663364
Xie WZhou RWang HShen TChen ESerra ESpezzano F(2024)Bridging User Dynamics: Transforming Sequential Recommendations with Schrödinger Bridge and Diffusion ModelsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679756(2618-2628)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679756
Niu YXing XJia ZLiu RXin MCui JChua TNgo CKumar RLauw HKa-Wei Lee R(2024)Diffusion Recommendation with Implicit Sequence InfluenceCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651951(1719-1725)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3651951
Tang HZhao GGao JQian X(2024)Personalized Representation With Contrastive Loss for Recommendation SystemsIEEE Transactions on Multimedia10.1109/TMM.2023.329574026(2419-2429)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3295740
Cheng ZZhou FXu XZhang KTrajcevski GZhong TYu P(2024)Information Cascade Popularity Prediction via Probabilistic DiffusionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.346524136:12(8541-8555)Online publication date: Dec-2024
https://doi.org/10.1109/TKDE.2024.3465241
Li YSun XChen HZhang SYang YXu G(2024)Attention Is Not the Only Choice: Counterfactual Reasoning for Path-Based Explainable RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.337360836:9(4458-4471)Online publication date: Sep-2024
https://doi.org/10.1109/TKDE.2024.3373608
Wu YXie RZhu YZhuang FZhang XLin LHe Q(2024)Personalized Prompt for Sequential RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.335749836:7(3376-3389)Online publication date: Jul-2024
https://doi.org/10.1109/TKDE.2024.3357498
Li PZhou GXie ZXie PHuang J(2024)Learning Dynamic and Static Representations for Extrapolation-Based Temporal Knowledge Graph ReasoningIEEE/ACM Transactions on Audio, Speech, and Language Processing10.1109/TASLP.2024.348550032(4741-4754)Online publication date: 2024
https://doi.org/10.1109/TASLP.2024.3485500
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents