research-article

Open access

Neural Collaborative Filtering vs. Matrix Factorization Revisited

Authors:

Steffen Rendle,

Walid Krichene,

John AndersonAuthors Info & Claims

RecSys '20: Proceedings of the 14th ACM Conference on Recommender Systems

Pages 240 - 248

https://doi.org/10.1145/3383313.3412488

Published: 22 September 2020 Publication History

All formats PDF

Abstract

Embedding based models have been the state of the art in collaborative filtering for over a decade. Traditionally, the dot product or higher order equivalents have been used to combine two or more embeddings, e.g., most notably in matrix factorization. In recent years, it was suggested to replace the dot product with a learned similarity e.g. using a multilayer perceptron (MLP). This approach is often referred to as neural collaborative filtering (NCF). In this work, we revisit the experiments of the NCF paper that popularized learned similarities using MLPs. First, we show that with a proper hyperparameter selection, a simple dot product substantially outperforms the proposed learned similarities. Second, while a MLP can in theory approximate any function, we show that it is non-trivial to learn a dot product with an MLP. Finally, we discuss practical issues that arise when applying MLP based similarities and show that MLPs are too costly to use for item recommendation in production environments while dot products allow to apply very efficient retrieval algorithms. We conclude that MLPs should be used with care as embedding combiner and that dot products might be a better default choice.

References

[1]

Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. 2019. A convergence theory for deep learning via over-parameterization. In Proceedings of the 36th International Conference on Machine Learning. 242–252.

[2]

Alexandr Andoni, Rina Panigrahy, Gregory Valiant, and Li Zhang. 2014. Learning Polynomials with Neural Networks. In Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32(ICML’14). JMLR.org, II–1908–II–1916.

[3]

Andrew R Barron. 1993. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information theory 39, 3 (1993), 930–945.

Digital Library

[4]

Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of machine learning research 3, Feb (2003), 1137–1155.

[5]

Alex Beutel, Paul Covington, Sagar Jain, Can Xu, Jia Li, Vince Gatto, and Ed H. Chi. 2018. Latent Cross: Making Use of Context in Recurrent Recommender Systems. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining(WSDM ’18). Association for Computing Machinery, New York, NY, USA, 46–54. https://doi.org/10.1145/3159652.3159727

Digital Library

[6]

Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep Neural Networks for YouTube Recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems(RecSys ’16). Association for Computing Machinery, New York, NY, USA, 191–198. https://doi.org/10.1145/2959100.2959190

Digital Library

[7]

George Cybenko. 1989. Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems 2, 4 (1989), 303–314.

[8]

Maurizio Ferrari Dacrema, Simone Boglio, Paolo Cremonesi, and Dietmar Jannach. 2019. A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research. arxiv:cs.IR/1911.07698

[9]

Maurizio Ferrari Dacrema, Federico Parroni, Paolo Cremonesi, and Dietmar Jannach. 2020. Critically Examining the Claimed Value of Convolutions over User-Item Embedding Maps for Recommender Systems. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM 2020), October 19–23, 2020, Virtual Event, Ireland. https://doi.org/10.1145/3340531.3411901

[10]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arxiv:cs.CL/1810.04805

[11]

Simon Du, Jason Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai. 2019. Gradient Descent Finds Global Minima of Deep Neural Networks. In Proceedings of the 36th International Conference on Machine Learning. 1675–1685.

[12]

Gintare Karolina Dziugaite and Daniel M. Roy. 2015. Neural Network Matrix Factorization. arxiv:cs.LG/1511.06443

[13]

X. Geng, H. Zhang, J. Bian, and T. Chua. 2015. Learning Image and User Features for Recommendation in Social Networks. In 2015 IEEE International Conference on Computer Vision (ICCV). 4274–4282.

[14]

F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst. 5, 4, Article Article 19 (Dec. 2015), 19 pages. https://doi.org/10.1145/2827872

Digital Library

[15]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Jun 2016). https://doi.org/10.1109/cvpr.2016.90

[16]

Xiangnan He, Xiaoyu Du, Xiang Wang, Feng Tian, Jinhui Tang, and Tat-Seng Chua. 2018. Outer Product-based Neural Collaborative Filtering. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18. International Joint Conferences on Artificial Intelligence Organization, 2227–2233. https://doi.org/10.24963/ijcai.2018/308

[17]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In Proceedings of the 26th International Conference on World Wide Web(WWW ’17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 173–182. https://doi.org/10.1145/3038912.3052569

Digital Library

[18]

Kurt Hornik, Maxwell Stinchcombe, Halbert White, 1989. Multilayer feedforward networks are universal approximators.Neural networks 2, 5 (1989), 359–366.

[19]

Binbin Hu, Chuan Shi, Wayne Xin Zhao, and Philip S. Yu. 2018. Leveraging Meta-Path Based Context for Top- N Recommendation with A Neural Co-Attention Model. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining(KDD ’18). Association for Computing Machinery, New York, NY, USA, 1531–1540. https://doi.org/10.1145/3219819.3219965

Digital Library

[20]

Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative Filtering for Implicit Feedback Datasets. In Proceedings of the 2008 Eighth IEEE International Conference on Data Mining(ICDM ’08). 263–272.

Digital Library

[21]

I. M. A. Jawarneh, P. Bellavista, A. Corradi, L. Foschini, R. Montanari, J. Berrocal, and J. M. Murillo. 2020. A Pre-Filtering Approach for Incorporating Contextual Information Into Deep Learning Based Recommender Systems. IEEE Access 8(2020), 40485–40498.

[22]

Yehuda Koren. 2009. The BellKor Solution to the Netflix Grand Prize.

[23]

Yehuda Koren and Robert Bell. 2011. Advances in Collaborative Filtering. Springer US, Boston, MA, 145–186. https://doi.org/10.1007/978-0-387-85820-3_5

[24]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems. 1097–1105.

[25]

Mark Levy and Kris Jack. 2013. Efficient top-n recommendation by linear regression. In RecSys Large Scale Recommender Systems Workshop.

[26]

Yuanzhi Li and Yang Yuan. 2017. Convergence Analysis of Two-layer Neural Networks with ReLU Activation. In Advances in Neural Information Processing Systems. 597–607.

[27]

Ting Liu, Andrew W. Moore, Alexander Gray, and Ke Yang. 2004. An Investigation of Practical Approximate Nearest Neighbor Algorithms. In Proceedings of the 17th International Conference on Neural Information Processing Systems(NIPS’04). MIT Press, Cambridge, MA, USA, 825–832.

Digital Library

[28]

Peter Mattson, Christine Cheng, Cody Coleman, Greg Diamos, Paulius Micikevicius, David Patterson, Hanlin Tang, Gu-Yeon Wei, Peter Bailis, Victor Bittorf, David Brooks, Dehao Chen, Debojyoti Dutta, Udit Gupta, Kim Hazelwood, Andrew Hock, Xinyuan Huang, Atsushi Ike, Bill Jia, Daniel Kang, David Kanter, Naveen Kumar, Jeffery Liao, Guokai Ma, Deepak Narayanan, Tayo Oguntebi, Gennady Pekhimenko, Lillian Pentecost, Vijay Janapa Reddi, Taylor Robie, Tom St. John, Tsuguchika Tabaru, Carole-Jean Wu, Lingjie Xu, Masafumi Yamazaki, Cliff Young, and Matei Zaharia. 2019. MLPerf Training Benchmark. arxiv:cs.LG/1910.01500

[29]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111–3119.

[30]

Xia Ning and George Karypis. 2011. Slim: Sparse linear methods for top-n recommender systems. In 2011 IEEE 11th International Conference on Data Mining. IEEE, 497–506.

Digital Library

[31]

Wei Niu, James Caverlee, and Haokai Lu. 2018. Neural Personalized Ranking for Image Recommendation. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining(WSDM ’18). Association for Computing Machinery, New York, NY, USA, 423–431. https://doi.org/10.1145/3159652.3159728

Digital Library

[32]

Arkadiusz Paterek. 2007. Improving regularized singular value decomposition for collaborative filtering. In Proceedings of KDD cup and workshop, Vol. 2007. 5–8.

[33]

Jiarui Qin, Kan Ren, Yuchen Fang, Weinan Zhang, and Yong Yu. 2020. Sequential Recommendation with Dual Side Neighbor-Based Collaborative Relation Modeling. In Proceedings of the 13th International Conference on Web Search and Data Mining(WSDM ’20). Association for Computing Machinery, New York, NY, USA, 465–473. https://doi.org/10.1145/3336191.3371842

Digital Library

[34]

Steffen Rendle, Li Zhang, and Yehuda Koren. 2019. On the Difficulty of Evaluating Baselines: A Study on Recommender Systems. CoRR abs/1905.01395(2019). arxiv:1905.01395http://arxiv.org/abs/1905.01395

[35]

Anshumali Shrivastava and Ping Li. 2014. Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS). In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2(NIPS’14). MIT Press, Cambridge, MA, USA, 2321–2329.

[36]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998–6008.

[37]

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144(2016).

[38]

Hamed Zamani and W. Bruce Croft. 2020. Learning a Joint Search and Recommendation Model from User-Item Interactions. In Proceedings of the 13th International Conference on Web Search and Data Mining(WSDM ’20). Association for Computing Machinery, New York, NY, USA, 717–725. https://doi.org/10.1145/3336191.3371818

Digital Library

[39]

Xing Zhao, Ziwei Zhu, Yin Zhang, and James Caverlee. 2020. Improving the Estimation of Tail Ratings in Recommender System with Multi-Latent Representations. In Proceedings of the 13th International Conference on Web Search and Data Mining(WSDM ’20). Association for Computing Machinery, New York, NY, USA, 762–770. https://doi.org/10.1145/3336191.3371810

Digital Library

Cited By

De Waele GMenschaert GWaegeman W(2024)An antimicrobial drug recommender system using MALDI-TOF MS and dual-branch neural networkseLife10.7554/eLife.93242.413Online publication date: 14-Nov-2024
https://doi.org/10.7554/eLife.93242.4
De Waele GMenschaert GWaegeman W(2024)An antimicrobial drug recommender system using MALDI-TOF MS and dual-branch neural networkseLife10.7554/eLife.9324213Online publication date: 14-Nov-2024
https://doi.org/10.7554/eLife.93242
Azri AHaddi AAllali H(2024)IUAutoTimeSVD++: A Hybrid Temporal Recommender System Integrating Item and User Features Using a Contractive AutoencoderInformation10.3390/info1504020415:4(204)Online publication date: 5-Apr-2024
https://doi.org/10.3390/info15040204
Show More Cited By

Neural Collaborative Filtering vs. Matrix Factorization Revisited
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

Reenvisioning the comparison between Neural Collaborative Filtering and Matrix Factorization
RecSys '21: Proceedings of the 15th ACM Conference on Recommender Systems

Collaborative filtering models based on matrix factorization and learned similarities using Artificial Neural Networks (ANNs) have gained significant attention in recent years. This is, in part, because ANNs have demonstrated very good results in a wide ...
Collaborative filtering using non-negative matrix factorisation

Collaborative filtering is a popular strategy in recommender systems area. This approach gathers users' ratings and then predicts what users will rate based on their similarity to other users. However, most of the collaborative filtering methods have ...
A preprocessing matrix factorization on collaborative filtering based library book recommendation system
DSIT '18: Proceedings of the 2018 International Conference on Data Science and Information Technology

Nowadays, recommendation systems are widely used to recommend items to the users that are specific to their individual preferences and most appropriate. For this reason, many academic libraries try to establish an effectiveness and efficiency book ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

RecSys '20: Proceedings of the 14th ACM Conference on Recommender Systems

September 2020

796 pages

ISBN:9781450375832

DOI:10.1145/3383313

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

RecSys '20

Sponsor:

RecSys '20: Fourteenth ACM Conference on Recommender Systems

September 22 - 26, 2020

Virtual Event, Brazil

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

264
Total Citations
View Citations
14,491
Total Downloads

Downloads (Last 12 months)3,511
Downloads (Last 6 weeks)469

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

De Waele GMenschaert GWaegeman W(2024)An antimicrobial drug recommender system using MALDI-TOF MS and dual-branch neural networkseLife10.7554/eLife.93242.413Online publication date: 14-Nov-2024
https://doi.org/10.7554/eLife.93242.4
De Waele GMenschaert GWaegeman W(2024)An antimicrobial drug recommender system using MALDI-TOF MS and dual-branch neural networkseLife10.7554/eLife.9324213Online publication date: 14-Nov-2024
https://doi.org/10.7554/eLife.93242
Azri AHaddi AAllali H(2024)IUAutoTimeSVD++: A Hybrid Temporal Recommender System Integrating Item and User Features Using a Contractive AutoencoderInformation10.3390/info1504020415:4(204)Online publication date: 5-Apr-2024
https://doi.org/10.3390/info15040204
Li YYan SZhao FJiang YChen SWang LMa L(2024)MIMA: Multi-Feature Interaction Meta-Path Aggregation Heterogeneous Graph Neural Network for RecommendationsFuture Internet10.3390/fi1608027016:8(270)Online publication date: 29-Jul-2024
https://doi.org/10.3390/fi16080270
Yuan YZhou YChen XXiong QOkere H(2024)Enhancing Recommendation Diversity and Novelty with Bi-LSTM and Mean Shift ClusteringElectronics10.3390/electronics1319384113:19(3841)Online publication date: 28-Sep-2024
https://doi.org/10.3390/electronics13193841
Yin MBoughanmi KAnsari A(2024)Meta-Learning Customer Preference Dynamics on Digital PlatformsSSRN Electronic Journal10.2139/ssrn.4727171Online publication date: 2024
https://doi.org/10.2139/ssrn.4727171
Iliadis DDe Baets BPahikkala TWaegeman W(2024)A comparison of embedding aggregation strategies in drug–target interaction predictionBMC Bioinformatics10.1186/s12859-024-05684-y25:1Online publication date: 6-Feb-2024
https://doi.org/10.1186/s12859-024-05684-y
Sang LLi HZhang YZhang YYang Y(2024)AdaGIN: Adaptive Graph Interaction Network for Click-Through Rate PredictionACM Transactions on Information Systems10.1145/368178543:1(1-31)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3681785
Liu YKennedy LAmiri HZüfle A(2024)Neural Collaborative Filtering to Detect Anomalies in Human Semantic TrajectoriesProceedings of the 1st ACM SIGSPATIAL International Workshop on Geospatial Anomaly Detection10.1145/3681765.3698463(79-89)Online publication date: 29-Oct-2024
https://dl.acm.org/doi/10.1145/3681765.3698463
Liu WU LLiang SZhu HYu JLiu YYin J(2024)VAE*: A Novel Variational Autoencoder via Revisiting Positive and Negative Samples for Top-N RecommendationACM Transactions on Knowledge Discovery from Data10.1145/368055218:9(1-24)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3680552
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents