research-article

Device-cloud Collaborative Recommendation via Meta Controller

Authors:

Hongxia YangAuthors Info & Claims

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Pages 4353 - 4362

https://doi.org/10.1145/3534678.3539181

Published: 14 August 2022 Publication History

Abstract

On-device machine learning enables the lightweight deployment of recommendation models in local clients, which reduces the burden of the cloud-based recommenders and simultaneously incorporates more real-time user features. Nevertheless, the cloud-based recommendation in the industry is still very important considering its powerful model capacity and the efficient candidate generation from the billion-scale item pool. Previous attempts to integrate the merits of both paradigms mainly resort to a sequential mechanism, which builds the on-device recommender on top of the cloud-based recommendation. However, such a design is inflexible when user interests dramatically change: the on-device model is stuck by the limited item cache while the cloud-based recommendation based on the large item pool do not respond without the new re-fresh feedback. To overcome this issue, we propose a meta controller to dynamically manage the collaboration between the on-device recommender and the cloud-based recommender, and introduce a novel efficient sample construction from the causal perspective to solve the dataset absence issue of meta controller. On the basis of the counterfactual samples and the extended training, extensive experiments in the industrial recommendation scenarios show the promise of meta controller in the device-cloud collaboration.

Supplemental Material

MP4 File

Device-Cloud Collaborative Recommendation via Meta Controller

Download
29.30 MB

References

[1]

Amin Banitalebi-Dehkordi, Naveen Vedula, Jian Pei, Fei Xia, Lanjun Wang, and Yong Zhang. 2021. Auto-Split: A General Framework of Collaborative Edge-Cloud AI. In SIGKDD .

[2]

Han Cai, Tianyao Chen, Weinan Zhang, Yong Yu, and Jun Wang. 2018. Efficient architecture search by network transformation. In AAAI .

[3]

Han Cai, Chuang Gan, Ligeng Zhu, and Song Han. 2020. TinyTL: Reduce Memory, Not Parameters for Efficient On-Device Learning. NeurIPS (2020).

[4]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et almbox. 2016. Wide & deep learning for recommender systems. In DLRS .

[5]

Sauptik Dhar, Junyao Guo, Jiayi Liu, Samarth Tripathi, Unmesh Kurup, and Mohak Shah. 2021. A survey of on-device machine learning: An algorithms and learning theory perspective. TIOT (2021).

Digital Library

[6]

Eustache Diemert, Artem Betlei, Christophe Renaudin, and Massih-Reza Amini. 2018. A large scale benchmark for uplift modeling. In SIGKDD .

[7]

Thang Le Duc, Rafael Garc'ia Leiva, Paolo Casari, and Per-Olov Östberg. 2019. Machine learning methods for reliable resource provisioning in edge-cloud computing: A survey. CSUR (2019).

[8]

Yu Gong, Ziwen Jiang, Yufei Feng, Binbin Hu, Kaiqi Zhao, Qingwen Liu, and Wenwu Ou. 2020. EdgeRec: recommender system on edge in Mobile Taobao. In CIKM .

[9]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In IJCAI .

[10]

Dietmar Jannach, Malte Ludewig, and Lukas Lerche. 2017. Session-based item recommendation in e-commerce: on short-term intents, reminders, trends and discounts. UMUAI (2017).

[11]

Fredrik Johansson, Uri Shalit, and David Sontag. 2016. Learning representations for counterfactual inference. In ICML .

[12]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer (2009).

Digital Library

[13]

Kun Kuang, Lian Li, Zhi Geng, Lei Xu, Kun Zhang, Beishui Liao, Huaxin Huang, Peng Ding, Wang Miao, and Zhichao Jiang. 2020. Causal inference. Engineering (2020).

[14]

Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu. 2019. Metalearners for estimating heterogeneous treatment effects using machine learning. PNAS (2019).

[15]

Ji Lin, Wei-Ming Chen, Yujun Lin, Chuang Gan, Song Han, et almbox. 2020. MCUNet: Tiny Deep Learning on IoT Devices. NeurIPS (2020).

[16]

Clara Meister, Elizabeth Salesky, and Ryan Cotterell. 2020. Generalized Entropy Regularization or: There's Nothing Special about Label Smoothing. In ACL .

[17]

Thomas Norrie, Nishant Patil, Doe Hyun Yoon, George Kurian, Sheng Li, James Laudon, Cliff Young, Norman Jouppi, and David Patterson. 2021. The Design Process for Google's Training Chips: TPUv2 and TPUv3. IEEE Micro (2021).

[18]

Diego Olaya, Kristof Coussement, and Wouter Verbeke. 2020. A survey and benchmarking study of multitreatment uplift modeling. DMKD (2020).

[19]

Yujie Pan, Jiangchao Yao, Bo Han, Kunyang Jia, Ya Zhang, and Hongxia Yang. 2021. Click-through Rate Prediction with Auto-Quantized Contrastive Learning. arXiv preprint arXiv:2109.13921 (2021).

[20]

Judea Pearl and Dana Mackenzie. 2018. The book of why: the new science of cause and effect .

[21]

Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on long sequential user behavior modeling for click-through rate prediction. In SIGKDD .

[22]

Qi Pi, Guorui Zhou, Yujing Zhang, Zhe Wang, Lejian Ren, Ying Fan, Xiaoqiang Zhu, and Kun Gai. 2020. Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In CIKM .

[23]

Nicholas Radcliffe. 2007. Using control groups to target on predicted lift: Building and assessing uplift model.

[24]

Paul Resnick and Hal R Varian. 1997. Recommender systems. Commun. ACM (1997).

[25]

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In CVPR .

[26]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In WWW .

[27]

Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. 2021. Toward causal representation learning. IEEE (2021).

[28]

Uri Shalit, Fredrik D Johansson, and David Sontag. 2017. Estimating individual treatment effect: generalization bounds and algorithms. In ICML .

[29]

Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. ICLR (2017).

[30]

Chuan Sun, Hui Li, Xiuhua Li, Junhao Wen, Qingyu Xiong, and Wei Zhou. 2020 a. Convergence of recommender systems and edge computing: A comprehensive survey. IEEE Access (2020).

[31]

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM .

[32]

Yang Sun, Fajie Yuan, Min Yang, Guoao Wei, Zhou Zhao, and Duo Liu. 2020 c. A generic network compression framework for sequential recommender systems. In SIGIR .

[33]

Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou. 2020 b. Mobilebert: a compact task-agnostic bert for resource-limited devices. arXiv preprint arXiv:2004.02984 (2020).

[34]

Qiaoyu Tan, Jianwei Zhang, Jiangchao Yao, Ninghao Liu, Jingren Zhou, Hongxia Yang, and Xia Hu. 2021. Sparse-interest network for sequential recommendation. In WSDM .

[35]

Shoujin Wang, Longbing Cao, Yan Wang, Quan Z Sheng, Mehmet A Orgun, and Defu Lian. 2021. A survey on session-based recommender systems. CSUR (2021).

[36]

Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma. 2020. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768 (2020).

[37]

Xiangfu Shi Yiqiao Dai Philip S. Yu Xiaoqiang Zhu Wenwei Ke, Chuanren Liu. 2021. Addressing Exposure Bias in Uplift Modeling for Large-scale Online Advertising. In ICDM .

[38]

Hong-Jian Xue, Xinyu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. 2017. Deep Matrix Factorization Models for Recommender Systems. In IJCAI .

[39]

Liu Yang, Ben Tan, Vincent W Zheng, Kai Chen, and Qiang Yang. 2020. Federated recommendation systems. In Federated Learning .

[40]

Jiangchao Yao, Feng Wang, Kunyang Jia, Bo Han, Jingren Zhou, and Hongxia Yang. 2021. Device-Cloud Collaborative Learning for Recommendation. In SIGKDD .

[41]

Jiangchao Yao, Yanfeng Wang, Ya Zhang, Jun Sun, and Jun Zhou. 2017a. Joint latent Dirichlet allocation for social tags. TMM, Vol. 20, 1 (2017), 224--237.

[42]

Jiangchao Yao, Shengyu Zhang, Yang Yao, Feng Wang, Jianxin Ma, Jianwei Zhang, Yunfei Chu, Luo Ji, Kunyang Jia, Tao Shen, et almbox. 2022. Edge-Cloud Polarization and Collaboration: A Comprehensive Survey. TKDE (2022).

[43]

Jiangchao Yao, Ya Zhang, Ivor Tsang, and Jun Sun. 2017b. Discovering user interests from social images. In MMM. 160--172.

[44]

Shengyu Zhang, Lingxiao Yang, Dong Yao, Yujie Lu, Fuli Feng, Zhou Zhao, Tat-seng Chua, and Fei Wu. 2022. Re4: Learning to Re-Contrast, Re-Attend, Re-Construct for Multi-Interest Recommendation. In Proceedings of the ACM Web Conference . 2216--2226.

Digital Library

[45]

Shengyu Zhang, Dong Yao, Zhou Zhao, Tat-Seng Chua, and Fei Wu. 2021. Causerec: Counterfactual user sequence synthesis for sequential recommendation. In SIGIR . 367--377.

[46]

Kui Zhao, Junhao Hua, Ling Yan, Qi Zhang, Huan Xu, and Cheng Yang. 2019. A Unified Framework for Marketing Budget Allocation. In SIGKDD .

[47]

Zhenyu Zhao and Totte Harinen. 2019. Uplift modeling for multiple treatments with cost optimization. In DSAA .

[48]

Zhi-Dan Zhao and Ming-Sheng Shang. 2010. User-based collaborative-filtering recommendation algorithms on hadoop. In SIGKDD .

[49]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In SIGKDD .

[50]

Zhi Zhou, Xu Chen, En Li, Liekang Zeng, Ke Luo, and Junshan Zhang. 2019. Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proc. IEEE (2019).

[51]

Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016).

Cited By

Lv ZHe SZhan TZhang SZhang WChen JZhao ZWu FCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Semantic Codebook Learning for Dynamic Recommendation ModelsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680574(9611-9620)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680574
Liu YLi HZheng ZWu FChen GPapavassiliou SSchmid S(2024)On the Analysis of Two-Stage Stochastic BanditProceedings of the Twenty-fifth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing10.1145/3641512.3686360(51-60)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3641512.3686360
Lv ZZhang WChen ZZhang SKuang KChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Intelligent Model Update Strategy for Sequential RecommendationProceedings of the ACM Web Conference 202410.1145/3589334.3645316(3117-3128)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645316
Show More Cited By

Index Terms

Device-cloud Collaborative Recommendation via Meta Controller
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Typicality-Based Collaborative Filtering Recommendation

Collaborative filtering (CF) is an important and popular technology for recommender systems. However, current CF methods suffer from such problems as data sparsity, recommendation inaccuracy, and big-error in predictions. In this paper, we borrow ideas ...
Using a trust network to improve top-N recommendation
RecSys '09: Proceedings of the third ACM conference on Recommender systems

Top-N item recommendation is one of the important tasks of recommenders. Collaborative filtering is the most popular approach to building recommender systems which can predict ratings for a given user and item. Collaborative filtering can be extended ...
Improved recommendation based on collaborative tagging behaviors
IUI '08: Proceedings of the 13th international conference on Intelligent user interfaces

Considering the natural tendency of people to follow direct or indirect cues of other people's activities, collaborative filtering-based recommender systems often predict the utility of an item for a particular user according to previous ratings by ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 2022

5033 pages

ISBN:9781450393850

DOI:10.1145/3534678

General Chairs:
Aidong Zhang
University of Virginia
,
Huzefa Rangwala
Amazon/George Mason University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '22

Sponsor:

KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 14 - 18, 2022

Washington DC, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
388
Total Downloads

Downloads (Last 12 months)92
Downloads (Last 6 weeks)10

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lv ZHe SZhan TZhang SZhang WChen JZhao ZWu FCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Semantic Codebook Learning for Dynamic Recommendation ModelsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680574(9611-9620)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680574
Liu YLi HZheng ZWu FChen GPapavassiliou SSchmid S(2024)On the Analysis of Two-Stage Stochastic BanditProceedings of the Twenty-fifth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing10.1145/3641512.3686360(51-60)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3641512.3686360
Lv ZZhang WChen ZZhang SKuang KChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Intelligent Model Update Strategy for Sequential RecommendationProceedings of the ACM Web Conference 202410.1145/3589334.3645316(3117-3128)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645316
Mai WYao JChen GZhang YCheung YHan B(2023)Server-Client Collaborative Distillation for Federated Reinforcement LearningACM Transactions on Knowledge Discovery from Data10.1145/360493918:1(1-22)Online publication date: 10-Aug-2023
https://dl.acm.org/doi/10.1145/3604939
Lang LHu MPei CChen G(2023)Beyond the Content: Considering the Network for Online Video RecommendationProceedings of the 7th Asia-Pacific Workshop on Networking10.1145/3600061.3600075(150-156)Online publication date: 29-Jun-2023
https://dl.acm.org/doi/10.1145/3600061.3600075
Lv ZZhang WZhang SKuang KWang FWang YChen ZShen TYang HOoi BWu F(2023)DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model GeneralizationProceedings of the ACM Web Conference 202310.1145/3543507.3583451(3077-3085)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583451
Li ZLi QZhou YZhong WZhang GWu CChen HDuh WHuang HKato MMothe JPoblete B(2023)Edge-cloud Collaborative Learning with Federated and Centralized FeaturesProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591976(1949-1953)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591976
Fan ZWang YYao JLyu LZhang YTian Q(2022)FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation2022 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM54844.2022.00023(131-140)Online publication date: Nov-2022
https://doi.org/10.1109/ICDM54844.2022.00023

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents