research-article

Dual Policy Learning for Aggregation Optimization in Graph Neural Network-based Recommender Systems

Authors:

Hogun ParkAuthors Info & Claims

WWW '23: Proceedings of the ACM Web Conference 2023

Pages 1478 - 1488

https://doi.org/10.1145/3543507.3583241

Published: 30 April 2023 Publication History

Abstract

Graph Neural Networks (GNNs) provide effective representations for recommendation tasks. GNN-based recommendation systems (GNN-Rs) capture the complex high-order connectivity between users and items by aggregating information from distant neighbors and can improve the performance of recommender systems. Recently, Knowledge Graphs (KGs) have also been incorporated into the user-item interaction graph to provide more abundant contextual information; they are exploited to address cold-start problems and enable more explainable aggregation in GNN-Rs. However, due to the heterogeneous nature of users and items, developing an effective aggregation strategy that works across multiple GNN-Rs, such as LightGCN and KGAT, remains a challenge. In this paper, we propose a novel reinforcement learning-based message passing framework for recommender systems, which we call DPAO (Dual Policy learning framework for Aggregation Optimization). This framework adaptively determines high-order connectivity to aggregate users and items using dual policy learning. Dual policy learning leverages two Deep-Q-Network models to exploit the user- and item-aware feedback from a GNN-R and boost the performance of the target GNN-R. Our proposed framework was evaluated with both non-KG-based and KG-based GNN-R models on six real-world datasets, and their results show that our proposed framework significantly enhances the recent base model, improving nDCG and Recall by up to 63.7% and 42.9%, respectively. Our implementation code is available at https://github.com/steve30572/DPAO/.

Supplemental Material

PDF File

Appendix

Download
2.12 MB

References

[1]

Sami Abu-El-Haija, Amol Kapoor, Bryan Perozzi, and Joonseok Lee. 2020. N-gcn: Multi-scale graph convolution for semi-supervised node classification. In Proc. of UAI. 841–851.

[2]

Smriti Bhagat, Graham Cormode, and S Muthukrishnan. 2011. Node classification in social networks. In Social network data analytics. 115–148.

[3]

Xinshi Chen, Shuang Li, Hui Li, Shaohua Jiang, Yuan Qi, and Le Song. 2019. Generative adversarial user model for reinforcement learning based recommendation system. In Proc. of ICML.

[4]

William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proc. of NeurIPS. 1025–1035.

[5]

Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proc. of SIGIR. 355–364.

[6]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proc. of SIGIR. 639–648.

[7]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proc. of TheWebConf. 173–182.

[8]

Binbin Hu, Chuan Shi, Wayne Xin Zhao, and Philip S Yu. 2018. Leveraging meta-path based context for top-n recommendation with a neural co-attention model. In Proc. of SIGKDD. 1531–1540.

[9]

Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. Proc. of ICLR (2017).

[10]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.

[11]

Kwei-Herng Lai, Daochen Zha, Kaixiong Zhou, and Xia Hu. 2020. Policy-gnn: Aggregation optimization for graph neural networks. In Proc. of SIGKDD. 461–471.

[12]

Yu Lei, Hongbin Pei, Hanqi Yan, and Wenjie Li. 2020. Reinforcement learning based recommendation with graph convolutional q-network. In Proc. of SIGIR. 1757–1760.

[13]

Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In Proc. of ICML. 1928–1937.

[14]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).

[15]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529–533.

[16]

Shanlei Mu, Yaliang Li, Wayne Xin Zhao, Jingyuan Wang, Bolin Ding, and Ji-Rong Wen. 2022. Alleviating Spurious Correlations in Knowledge-aware Recommendations through Counterfactual Generator. In Proc. of SIGIR. 1401–1411.

[17]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012).

[18]

Steffen Rendle, Zeno Gantner, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2011. Fast context-aware recommendations with factorization machines. In Proc. of SIGIR. 635–644.

[19]

Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 (2016).

[20]

Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. 2000. Policy gradient methods for reinforcement learning with function approximation. In Proc. of NeurIPS. 1057–1063.

[21]

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In Proc. of WWW. 1067–1077.

[22]

Guojia Wan, Bo Du, Shirui Pan, and Gholameza Haffari. 2020. Reinforcement learning based meta-path discovery in large-scale heterogeneous information networks. In Proc. of AAAI. 6094–6101.

[23]

Hongwei Wang, Fuzheng Zhang, Mengdi Zhang, Jure Leskovec, Miao Zhao, Wenjie Li, and Zhongyuan Wang. 2019. Knowledge-aware graph neural networks with label smoothness regularization for recommender systems. In Proc. of SIGKDD. 968–977.

[24]

Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. Kgat: Knowledge graph attention network for recommendation. In Proc. of SIGKDD. 950–958.

[25]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In Proc. of SIGIR. 165–174.

[26]

Xiang Wang, Tinglin Huang, Dingxian Wang, Yancheng Yuan, Zhenguang Liu, Xiangnan He, and Tat-Seng Chua. 2021. Learning Intents behind Interactions with Knowledge Graph for Recommendation. In Proc. of TheWebConf. 878–887.

[27]

Xiang Wang, Yaokun Xu, Xiangnan He, Yixin Cao, Meng Wang, and Tat-Seng Chua. 2020. Reinforced negative sampling over knowledge graph for recommendation. In Proc. of TheWebConf. 99–109.

[28]

Ze Wang, Guangyan Lin, Huobin Tan, Qinghong Chen, and Xiyang Liu. 2020. CKAN: Collaborative Knowledge-aware Attentive Network for Recommender Systems. In Proc. of SIGIR. 219–228.

[29]

Yikun Xian, Zuohui Fu, Shan Muthukrishnan, Gerard De Melo, and Yongfeng Zhang. 2019. Reinforcement knowledge graph reasoning for explainable recommendation. In Proc. of SIGIR. 285–294.

[30]

Teng Xiao, Zhengyu Chen, Donglin Wang, and Suhang Wang. 2021. Learning How to Propagate Messages in Graph Neural Networks. In Proc. of SIGKDD. 1894–1903.

[31]

Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018. Representation learning on graphs with jumping knowledge networks. In Proc. of ICML. 5453–5462.

[32]

Zekun Yin, Xiaoming Xu, Kaichao Fan, Ruilin Li, Weizhong Li, Weiguo Liu, and Beifang Niu. 2019. DGCF: A Distributed Greedy Clustering Framework for Large-scale Genomic Sequences. In Proc. of IEEE BIBM. 2272–2279.

[33]

Yongfeng Zhang, Qingyao Ai, Xu Chen, and Pengfei Wang. 2018. Learning over knowledge-base embeddings for recommendation. arXiv preprint arXiv:1803.06540 (2018).

[34]

Lixin Zou, Long Xia, Yulong Gu, Xiangyu Zhao, Weidong Liu, Jimmy Xiangji Huang, and Dawei Yin. 2020. Neural interactive collaborative filtering. In Proc. of SIGIR. 749–758.

Cited By

Li FChen Z(2024)Cross-Aggregation Based Information Re-Enhancement for Recommendation2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10651506(1-6)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10651506
Tiwary NMohd Noah SFauzi FYee T(2024)A Review of Explainable Recommender Systems Utilizing Knowledge Graphs and Reinforcement LearningIEEE Access10.1109/ACCESS.2024.342241612(91999-92019)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3422416
Li FChen W(2024)Feature Re-enhanced Meta-Contrastive Learning for RecommendationKnowledge Science, Engineering and Management10.1007/978-981-97-5501-1_20(260-271)Online publication date: 27-Jul-2024
https://doi.org/10.1007/978-981-97-5501-1_20
Show More Cited By

Index Terms

Dual Policy Learning for Aggregation Optimization in Graph Neural Network-based Recommender Systems
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Investigating serendipity in recommender systems based on real user feedback
SAC '18: Proceedings of the 33rd Annual ACM Symposium on Applied Computing

Over the past several years, research in recommender systems has emphasized the importance of serendipity, but there is still no consensus on the definition of this concept and whether serendipitous items should be recommended is still not a well-...
Acquiring User Information Needs for Recommender Systems
WI-IAT '13: Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 03

Most recommender systems attempt to use collaborative filtering, content-based filtering or hybrid approach to recommend items to new users. Collaborative filtering recommends items to new users based on their similar neighbours, and content-based ...
A Scalable, Accurate Hybrid Recommender System
WKDD '10: Proceedings of the 2010 Third International Conference on Knowledge Discovery and Data Mining

Recommender systems apply machine learning techniques for filtering unseen information and can predict whether a user would like a given resource. There are three main types of recommender systems: collaborative filtering, content-based filtering, and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '23: Proceedings of the ACM Web Conference 2023

April 2023

4293 pages

ISBN:9781450394161

DOI:10.1145/3543507

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Data Availability

Appendix https://dl.acm.org/doi/10.1145/3543507.3583241#3583241.pdf

Funding Sources

the Institute of Information & com- munications Technology Planning & evaluation (IITP) funded by the Korea government (MSIT)
the National Research Foundation of Korea (NRF)
the Institute of Information & com- munications Technology Planning & evaluation (IITP) funded by the Korea government (MSIT)
the National Research Foundation of Korea (NRF)

Conference

WWW '23

Sponsor:

SIGWEB

WWW '23: The ACM Web Conference 2023

April 30 - May 4, 2023

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
274
Total Downloads

Downloads (Last 12 months)109
Downloads (Last 6 weeks)13

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li FChen Z(2024)Cross-Aggregation Based Information Re-Enhancement for Recommendation2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10651506(1-6)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10651506
Tiwary NMohd Noah SFauzi FYee T(2024)A Review of Explainable Recommender Systems Utilizing Knowledge Graphs and Reinforcement LearningIEEE Access10.1109/ACCESS.2024.342241612(91999-92019)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3422416
Li FChen W(2024)Feature Re-enhanced Meta-Contrastive Learning for RecommendationKnowledge Science, Engineering and Management10.1007/978-981-97-5501-1_20(260-271)Online publication date: 27-Jul-2024
https://doi.org/10.1007/978-981-97-5501-1_20
Zhang XKuang S(2023)A Lightweight Method of Knowledge Graph Convolution Network for Collaborative FilteringInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.32735319:1(1-21)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.4018/IJSWIS.327353
Li FChen W(2023)Feature Re-enhanced Meta-Contrastive Learning for Recommendation2023 International Conference on Computer, Internet of Things and Smart City (CIoTSC)10.1109/CIoTSC60428.2023.00028(133-138)Online publication date: 3-Nov-2023
https://doi.org/10.1109/CIoTSC60428.2023.00028

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents