Article

Zero-Shot Recommendation as Language Modeling

Authors:

Robbe RaymaekersAuthors Info & Claims

Advances in Information Retrieval: 44th European Conference on IR Research, ECIR 2022, Stavanger, Norway, April 10–14, 2022, Proceedings, Part II

Pages 223 - 230

https://doi.org/10.1007/978-3-030-99739-7_26

Published: 10 April 2022 Publication History

Abstract

Recommendation is the task of ranking items (e.g. movies or products) according to individual user needs. Current systems rely on collaborative filtering and content-based techniques, which both require structured training data. We propose a framework for recommendation with off-the-shelf pretrained language models (LM) that only used unstructured text corpora as training data. If a user u liked Matrix and Inception, we construct a textual prompt, e.g. "Movies like Matrix, Inception,

< m >

” to estimate the affinity between u and m with LM likelihood. We motivate our idea with a corpus analysis, evaluate several prompt structures, and we compare LM-based recommendation with standard matrix factorization trained on different data regimes. The code for our experiments is publicly available (https://colab.research.google.com/drive/...?usp=sharing).

References

[1]

Barkan, O., Koenigstein, N.: Item2vec: neural item embedding for collaborative filtering. In: 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6 (2016).

[2]

Brown, T.B., et al.: Language models are few-shot learners (2020)

[3]

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis (2019)., https://aclanthology.org/N19-1423

[4]

Devooght, R., Bersini, H.: Long and short-term recommendations with recurrent neural networks. In: UMAP 2017, pp. 13–21. Association for Computing Machinery, New York (2017).

Digital Library

[5]

Ding, H., Ma, Y., Deoras, A., Wang, Y., Wang, H.: Zero-shot recommender systems (2021)

[6]

Feng, P.J., Pan, P., Zhou, T., Chen, H., Luo, C.: Zero shot on the cold-start problem: Model-agnostic interest learning for recommender systems. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM 2021, pp. 474–483. Association for Computing Machinery, New York (2021).

Digital Library

[7]

Guàrdia-Sebaoun, E., Guigue, V., Gallinari, P.: Latent trajectory modeling: a light and efficient way to introduce time in recommender systems. In: Proceedings of the 9th ACM Conference on Recommender Systems, pp. 281–284 (2015)

[8]

Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4) (2015).

Digital Library

[9]

Koren Y, Bell R, and Volinsky C Matrix factorization techniques for recommender systems Computer 2009 42 8 30-37

Digital Library

[10]

Li, J., Jing, M., Lu, K., Zhu, L., Yang, Y., Huang, Z.: From zero-shot learning to cold-start recommendation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 4189–4196 (2019)., https://ojs.aaai.org/index.php/AAAI/article/view/4324

Digital Library

[11]

Li, Z., Zhao, H., Liu, Q., Huang, Z., Mei, T., Chen, E.: Learning from history and present: next-item recommendation via discriminatively exploiting user behaviors. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1734–1743 (2018)

[12]

Penha, G., Hauff, C.: What does bert know about books, movies and music? probing bert for conversational recommendation. In: Fourteenth ACM Conference on Recommender Systems, RecSys 2020, pp. 388–397. Association for Computing Machinery, New York (2020).

Digital Library

[13]

Petroni, F., et al.: Language models as knowledge bases? In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2463–2473. Association for Computational Linguistics, Hong Kong (2019)., https://aclanthology.org/D19-1250

[14]

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners (2019). https://openai.com/blog/better-language-models/

[15]

Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: bayesian personalized ranking from implicit feedback. In: UAI 2009, pp. 452–461. AUAI Press, Arlington (2009)

[16]

Salah A, Truong QT, and Lauw HW Cornac: a comparative framework for multimodal recommender systems J. Mach. Learn. Res. 2020 21 95 1-5

[17]

Schein, A.I., Popescul, A., Ungar, L.H., Pennock, D.M.: Methods and metrics for cold-start recommendations. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2002, pp. 253–260. Association for Computing Machinery, New York (2002).

Digital Library

[18]

Schröder, G., Thiele, M., Lehner, W.: Setting goals and choosing metrics for recommender system evaluations. In: UCERSTI2 Workshop at the 5th ACM Conference on Recommender Systems, Chicago, USA, vol. 23, p. 53 (2011)

[19]

Sun, F., Let al.: Bert4rec: sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, pp. 1441–1450. Association for Computing Machinery, New York (2019).

Digital Library

[20]

Volkovs, M., Yu, G., Poutanen, T.: Dropoutnet: addressing cold start in recommender systems. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017). https://proceedings.neurips.cc/paper/2017/file/dbd22ba3bd0df8f385bdac3e9f8be207-Paper.pdf

Cited By

Lin JDai XXi YLiu WChen BZhang HLiu YWu CLi XZhu CGuo HYu YTang RZhang W(2025)How Can Recommender Systems Benefit from Large Language Models: A SurveyACM Transactions on Information Systems10.1145/367800443:2(1-47)Online publication date: 18-Jan-2025
https://dl.acm.org/doi/10.1145/3678004
Zhai JLiao LLiu XWang YLi RCao XGao LGong ZGu FHe JLu YShi YSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Actions speak louder than wordsProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694484(58484-58509)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3694484
Ge YLiu SFu ZTan JLi ZXu SLi YXian YZhang Y(2024)A Survey on Trustworthy Recommender SystemsACM Transactions on Recommender Systems10.1145/36528913:2(1-68)Online publication date: 13-Apr-2024
https://dl.acm.org/doi/10.1145/3652891
Show More Cited By

Recommendations

An Industrial Framework for Cold-Start Recommendation in Zero-Shot Scenarios
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

There exists the cold-start problem in the recommendation systems when observed user-item interactions are insufficient. To alleviate this problem, most existing works aim to learn globally shared prior knowledge across all items and be fast adapted to ...
Modeling user exposure with recommendation influence
SAC '20: Proceedings of the 35th Annual ACM Symposium on Applied Computing

Recommender systems learn users' preferences from their explicit feedback, such as ratings, and implicit feedback, such as purchases. Learning a user's preference from implicit feedback is difficult since inaction in such data can be because of either ...
Modeling Location-Based User Rating Profiles for Personalized Recommendation
TKDD Special Issue (SIGKDD'13)

This article proposes LA-LDA, a location-aware probabilistic generative model that exploits location-based ratings to model user profiles and produce recommendations. Most of the existing recommendation models do not consider the spatial information of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

Advances in Information Retrieval: 44th European Conference on IR Research, ECIR 2022, Stavanger, Norway, April 10–14, 2022, Proceedings, Part II

Apr 2022

629 pages

ISBN:978-3-030-99738-0

DOI:10.1007/978-3-030-99739-7

Editors:
Matthias Hagen
Martin Luther University Halle-Wittenberg, Halle, Germany
,
Suzan Verberne
Leiden University, Leiden, The Netherlands
,
Craig Macdonald
University of Glasgow, Glasgow, UK
,
Christin Seifert
University of Duisburg-Essen, Essen, Germany
,
Krisztian Balog
University of Stavanger, Stavanger, Norway
,
Kjetil Nørvåg
Norwegian University of Science and Technology, Trondheim, Norway
,
Vinay Setty
University of Stavanger, Stavanger, Norway

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 10 April 2022

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lin JDai XXi YLiu WChen BZhang HLiu YWu CLi XZhu CGuo HYu YTang RZhang W(2025)How Can Recommender Systems Benefit from Large Language Models: A SurveyACM Transactions on Information Systems10.1145/367800443:2(1-47)Online publication date: 18-Jan-2025
https://dl.acm.org/doi/10.1145/3678004
Zhai JLiao LLiu XWang YLi RCao XGao LGong ZGu FHe JLu YShi YSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Actions speak louder than wordsProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694484(58484-58509)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3694484
Ge YLiu SFu ZTan JLi ZXu SLi YXian YZhang Y(2024)A Survey on Trustworthy Recommender SystemsACM Transactions on Recommender Systems10.1145/36528913:2(1-68)Online publication date: 13-Apr-2024
https://dl.acm.org/doi/10.1145/3652891
Chen JDu KLin JChen BTang RZhang WYu YSerra ESpezzano F(2024)ELCoRec: Enhance Language Understanding with Co-Propagation of Numerical and Categorical Features for RecommendationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679789(259-269)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679789
Yang DChen FFang HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Behavior Alignment: A New Perspective of Evaluating LLM-based Conversational Recommendation SystemsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657924(2286-2290)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657924
Xu SHua WZhang YHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)OpenP5: An Open-Source Platform for Developing, Training, and Evaluating LLM-based Recommender SystemsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657883(386-394)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657883
Hao BYang CGuo LYu JYin HAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Motif-based Prompt Learning for Universal Cross-domain RecommendationProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635754(257-265)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635754
Wu XZhou HShi YYao WHuang XLiu NChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start RecommendationProceedings of the ACM Web Conference 202410.1145/3589334.3645494(3566-3575)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645494
Dai SShao NZhao HYu WSi ZXu CSun ZZhang XXu J(2023)Uncovering ChatGPT’s Capabilities in Recommender SystemsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3610646(1126-1132)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3610646

View Options

View options

Figures

Tables

Media

View Table of Conten