research-article

Structured Graph Convolutional Networks with Stochastic Masks for Recommender Systems

Authors:

Chin-Chia Michael Yeh,

Hao YangAuthors Info & Claims

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 614 - 623

https://doi.org/10.1145/3404835.3462868

Published: 11 July 2021 Publication History

Abstract

Graph Convolutional Networks (GCNs) are powerful for collaborative filtering. The key component of GCNs is to explore neighborhood aggregation mechanisms to extract high-level representations of users and items. However, real-world user-item graphs are often incomplete and noisy. Aggregating misleading neighborhood information may lead to sub-optimal performance if GCNs are not regularized properly. Also, the real-world user-item graphs are often sparse and low rank. These two intrinsic graph properties are widely used in shallow matrix completion models, but far less studied in graph neural models. Here we propose Structured Graph Convolutional Networks (SGCNs) to enhance the performance of GCNs by exploiting graph structural properties of sparsity and low rank. To achieve sparsity, we attach each layer of a GCN with a trainable stochastic binary mask to prune noisy and insignificant edges, resulting in a clean and sparsified graph. To preserve its low-rank property, the nuclear norm regularization is applied. We jointly learn the parameters of stochastic binary masks and original GCNs by solving a stochastic binary optimization problem. An unbiased gradient estimator is further proposed to better backpropagate the gradients of binary variables. Experimental results demonstrate that SGCNs achieve better performance compared with the state-of-the-art GCNs.

Supplementary Material

MP4 File (sigir2021.mp4)

Structured Graph Convolutional Networks with Stochastic Masks for Recommender Systems

Download
53.37 MB

References

[1]

Zeyuan Allen-Zhu and Yuanzhi Li. 2016. LazySVD: Even faster SVD decomposition yet without agonizing pain. In NeurIPS . 974--982.

[2]

Yoshua Bengio, Nicholas Léonard, and Aaron Courville. 2013. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432 (2013).

[3]

Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph convolutional matrix completion. In KDD Workshop on Deep Learning Day .

[4]

Huiyuan Chen and Jing Li. 2017. Learning multiple similarities of users and items in recommender systems. In ICDM. IEEE, 811--816.

[5]

Huiyuan Chen and Jing Li. 2020. Neural Tensor Model for Learning Multi-Aspect Factors in Recommender Systems. In IJCAI .

[6]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et almbox. 2016. Wide & deep learning for recommender systems. In DLRS. 7--10.

[7]

Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. 2018. Adversarial Attack on Graph Structured Data. In ICML. 1115--1124.

[8]

Zhe Dong, Andriy Mnih, and George Tucker. 2020. DisARM: An Antithetic Gradient Estimator for Binary Latent Variables. In NeurIPS .

[9]

Negin Entezari, Saba A Al-Sayouri, Amirali Darvishzadeh, and Evangelos E Papalexakis. 2020. All You Need Is Low (Rank) Defending Against Adversarial Attacks on Graphs. In WSDM . 169--177.

[10]

David Eppstein, Zvi Galil, Giuseppe F Italiano, and Amnon Nissenzweig. 1997. Sparsification-a technique for speeding up dynamic graph algorithms. Journal of the ACM (JACM) (1997), 669--696.

[11]

Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. In WWW. 417--426.

[12]

Santo Fortunato. 2010. Community detection in graphs. Physics reports, Vol. 486, 3--5 (2010), 75--174.

[13]

Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He. 2019. Learning Discrete Structures for Graph Neural Networks. In ICML. 1972--1982.

[14]

Ming Gao, Leihui Chen, Xiangnan He, and Aoying Zhou. 2018. Bine: Bipartite network embedding. In SIGIR. 715--724.

Digital Library

[15]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In IJCAI .

[16]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS. 1024--1034.

[17]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, YongDong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In SIGIR. 639--648.

[18]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW. 173--182.

[19]

Catalin Ionescu, Orestis Vantzos, and Cristian Sminchisescu. 2015. Matrix backpropagation for deep networks with structured layers. In CVPR . 2965--2973.

[20]

Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical Reparameterization with Gumbel-Softmax. In ICLR .

[21]

Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. 2020. Graph Structure Learning for Robust Graph Neural Networks. In KDD. 66--74.

[22]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR .

[23]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer (2009), 30--37.

Digital Library

[24]

Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem. 2019. Deepgcns: Can gcns go as deep as cnns?. In CVPR. 9267--9276.

[25]

Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI .

[26]

Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. 2018. xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In KDD . 1754--1763.

[27]

Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In WWW. 689--698.

[28]

Christos Louizos, Max Welling, and Diederik P Kingma. 2019. Learning Sparse Neural Networks through $ L_0 $ Regularization. In ICLR .

[29]

Dongsheng Luo, Wei Cheng, Wenchao Yu, Bo Zong, Jingchao Ni, Haifeng Chen, and Xiang Zhang. 2021. Learning to Drop: Robust Graph Neural Network via Topological Denoising. In WSDM .

[30]

Federico Monti, Michael Bronstein, and Xavier Bresson. 2017. Geometric matrix completion with recurrent multi-graph neural networks. In NeurIPS . 3697--3707.

[31]

Cameron Musco and Christopher Musco. 2015. Randomized block krylov methods for stronger and faster approximate singular value decomposition. NeurIPS (2015), 1396--1404.

[32]

Xia Ning and George Karypis. 2011. Slim: Sparse linear methods for top-n recommender systems. In ICDM. 497--506.

[33]

Hoang NT and Takanori Maehara. 2019. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint arXiv:1905.09550 (2019).

[34]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In UAI. 452--461.

Digital Library

[35]

Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2019. Dropedge: Towards deep graph convolutional networks on node classification. In ICLR .

[36]

Sam T Roweis and Lawrence K Saul. 2000. Nonlinear dimensionality reduction by locally linear embedding. science, Vol. 290, 5500 (2000), 2323--2326.

[37]

Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020. Unbiased recommender learning from missing-not-at-random implicit feedback. In WSDM. 501--509.

[38]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research (2014), 1929--1958.

[39]

Joshua B Tenenbaum, Vin De Silva, and John C Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. science, Vol. 290, 5500 (2000), 2319--2323.

[40]

Maksims Volkovs, Guangwei Yu, and Tomi Poutanen. 2017. Dropoutnet: Addressing cold start in recommender systems. In NeurIPS . 4957--4966.

[41]

Wei Wang, Zheng Dang, Yinlin Hu, Pascal Fua, and Mathieu Salzmann. 2019 a. Backpropagation-friendly eigendecomposition. In NeurIPS. 3162--3170.

[42]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019 b. Neural graph collaborative filtering. In SIGIR. 165--174.

[43]

Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, Vol. 8, 3--4 (1992), 229--256.

[44]

Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying Graph Convolutional Networks. In ICML. 6861--6871.

[45]

Jheng-Hong Yang, Chih-Ming Chen, Chuan-Ju Wang, and Ming-Feng Tsai. 2018. HOP-rec: high-order proximity for implicit recommendation. In RecSys . 140--144.

[46]

Mingzhang Yin and Mingyuan Zhou. 2019. ARM: Augment-REINFORCE-merge gradient for stochastic binary networks. In ICLR .

[47]

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In KDD . 974--983.

[48]

Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR) (2019), 1--38.

Digital Library

[49]

Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, and Wei Wang. 2020. Robust graph representation learning via neural sparsification. In ICML . 11458--11468.

[50]

Lei Zheng, Chun-Ta Lu, Fei Jiang, Jiawei Zhang, and Philip S Yu. 2018. Spectral collaborative filtering. In RecSys. 311--319.

[51]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In KDD. 1059--1068.

[52]

Dingyuan Zhu, Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2019. Robust graph convolutional networks against adversarial attacks. In KDD . 1399--1407.

Cited By

Zeng JWang NLi J(2025)Knowledge-driven hierarchical intents modeling for recommendationExpert Systems with Applications10.1016/j.eswa.2024.125361259(125361)Online publication date: Jan-2025
https://doi.org/10.1016/j.eswa.2024.125361
Yang HYang C(2024)Graph Convolutional Recommendation System Based on Bilateral Attention MechanismJournal of Engineering10.1155/2024/29786802024:1Online publication date: 15-Jul-2024
https://doi.org/10.1155/2024/2978680
Ma JBian KXu YZhu L(2024)ANAGL: A Noise-Resistant and Anti-Sparse Graph Learning for Micro-Video RecommendationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/367040720:9(1-15)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670407
Show More Cited By

Index Terms

Structured Graph Convolutional Networks with Stochastic Masks for Recommender Systems
1. Computer systems organization
  1. Architectures
    1. Other architectures
      1. Neural networks
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

MPIA: Multiple Preferences with Item Attributes for Graph Convolutional Collaborative Filtering
Web Engineering
Abstract
Personalized recommender systems are playing an increasingly critical role in a variety of online applications. In recent years, advancements in graph-structured deep neural networks have attracted considerable interest and achieved state-of-the-...
A Scalable, Accurate Hybrid Recommender System
WKDD '10: Proceedings of the 2010 Third International Conference on Knowledge Discovery and Data Mining

Recommender systems apply machine learning techniques for filtering unseen information and can predict whether a user would like a given resource. There are three main types of recommender systems: collaborative filtering, content-based filtering, and ...
Modelling trust networks using resistive circuits for trust-aware recommender systems

Recommender systems have been widely used for predicting unknown ratings. Collaborative filtering as a recommendation technique uses known ratings for predicting user preferences in the item selection. However, current collaborative filtering methods ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2021

2998 pages

ISBN:9781450380379

DOI:10.1145/3404835

General Chairs:
Fernando Diaz
(Google)
,
Chirag Shah
University of Washington
,
Torsten Suel
New York University
,
Program Chairs:
Pablo Castells
Universidad Autónoma de Madrid, Amazon
,
Rosie Jones
Spotify
,
Tetsuya Sakai
Waseda University

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGIR '21

Sponsor:

SIGIR

SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2021

Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

39
Total Citations
View Citations
1,146
Total Downloads

Downloads (Last 12 months)79
Downloads (Last 6 weeks)4

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zeng JWang NLi J(2025)Knowledge-driven hierarchical intents modeling for recommendationExpert Systems with Applications10.1016/j.eswa.2024.125361259(125361)Online publication date: Jan-2025
https://doi.org/10.1016/j.eswa.2024.125361
Yang HYang C(2024)Graph Convolutional Recommendation System Based on Bilateral Attention MechanismJournal of Engineering10.1155/2024/29786802024:1Online publication date: 15-Jul-2024
https://doi.org/10.1155/2024/2978680
Ma JBian KXu YZhu L(2024)ANAGL: A Noise-Resistant and Anti-Sparse Graph Learning for Micro-Video RecommendationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/367040720:9(1-15)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670407
Luo TLiu YPan S(2024)Collaborative Sequential Recommendations via Multi-view GNN-transformersACM Transactions on Information Systems10.1145/364943642:6(1-27)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3649436
Chua HDu YSun ZWang ZZhang JOng Y(2024)Unified Denoising Training for RecommendationProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688109(612-621)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688109
Yan YHu YZhou QWu SWang DTong HSerra ESpezzano F(2024)Topological Anonymous Walk Embedding: A New Structural Node Embedding ApproachProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679565(2796-2806)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679565
Chen HLai VJin HJiang ZDas MHu XAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Towards Mitigating Dimensional Collapse of Representations in Collaborative FilteringProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635832(106-115)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635832
Zhao YXu MChen HChen YCai YIslam RWang YDerr TChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Can One Embedding Fit All? A Multi-Interest Learning Paradigm Towards Improving User Interest Diversity FairnessProceedings of the ACM Web Conference 202410.1145/3589334.3645662(1237-1248)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645662
Yan YHu YZhou QLiu LZeng ZChen YPan MChen HDas MTong HChua TNgo CKa-Wei Lee RKumar RLauw H(2024)PaCEr: Network Embedding From Positional to StructuralProceedings of the ACM Web Conference 202410.1145/3589334.3645516(2485-2496)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645516
Lai YZhu YFan WZhang XZhou K(2024)Toward Adversarially Robust Recommendation From Adaptive Fraudster DetectionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.332787619(907-919)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIFS.2023.3327876
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents