research-article

Improved Recurrent Neural Networks for Session-based Recommendations

Authors:

Yong LiuAuthors Info & Claims

DLRS 2016: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems

Pages 17 - 22

https://doi.org/10.1145/2988450.2988452

Published: 15 September 2016 Publication History

Abstract

Recurrent neural networks (RNNs) were recently proposed for the session-based recommendation task. The models showed promising improvements over traditional recommendation approaches. In this work, we further study RNN-based models for session-based recommendations. We propose the application of two techniques to improve model performance, namely, data augmentation, and a method to account for shifts in the input data distribution. We also empirically study the use of generalised distillation, and a novel alternative model that directly predicts item embeddings. Experiments on the RecSys Challenge 2015 dataset demonstrate relative improvements of 12.8% and 14.8% over previously reported results on the Recall@20 and Mean Reciprocal Rank@20 metrics respectively.

References

[1]

Baidu Research. Deep speech 2: End-to-end speech recognition in english and mandarin. CoRR, abs/1512.02595, 2015.

[2]

K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman. Return of the devil in the details: Delving deep into convolutional nets. In British Machine Vision Conference, 2014.

[3]

F. Chollet. Keras. https://github.com/fchollet/keras, 2015.

[4]

J. Chung, Ç. Gülçehre, K. Cho, and Y. Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555, 2014.

[5]

A. de Brébisson, É. Simon, A. Auvolat, P. Vincent, and Y. Bengio. Artificial neural networks applied to taxi destination prediction. CoRR, abs/1508.00021, 2015.

[6]

Y. Gal. A theoretically grounded application of dropout in recurrent neural networks. CoRR, abs/1512.05287, 2016.

[7]

H. Geoffrey, V. Oriol, and D. Jeff. Distilling the knowledge in a neural network. arXiv:1511.03643, 2015.

[8]

A. Graves, A. Mohamed, and G. E. Hinton. Speech recognition with deep recurrent neural networks. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2013, Vancouver, BC, Canada, May 26-31, 2013, pages 6645--6649, 2013.

[9]

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. CoRR, abs/1512.03385, 2015.

[10]

B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk. Session-based recommendations with recurrent neural networks. CoRR, abs/1511.06939, 2015.

[11]

S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735--1780, 1997.

Digital Library

[12]

S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, pages 448--456, 2015.

Digital Library

[13]

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.

[14]

Y. Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, Nevada, USA, August 24-27, 2008, pages 426--434, 2008.

Digital Library

[15]

Y. Koren, R. M. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. IEEE Computer, 42(8):30--37, 2009.

Digital Library

[16]

A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1106--1114, 2012.

Digital Library

[17]

D. Lopez-Paz, B. Schölkopf, L. Bottou, and V. Vapnik. Unifying distillation and privileged information. In International Conference on Learning Representations, 2016.

[18]

T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In 27th Annual Conference on Neural Information Processing Systems, pages 3111--3119, 2013.

Digital Library

[19]

A. Mnih and G. E. Hinton. A scalable hierarchical distributed language model. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21, pages 1081--1088. Curran Associates, Inc., 2009.

Digital Library

[20]

S.-T. Park and W. Chu. Pairwise preference regression for cold-start recommendation. In Proceedings of the Third ACM Conference on Recommender Systems, RecSys '09, pages 21--28, New York, NY, USA, 2009. ACM.

Digital Library

[21]

R. Salakhutdinov, A. Mnih, and G. Hinton. Restricted boltzmann machines for collaborative filtering. In Proceedings of the 24th International Conference on Machine Learning, ICML '07, pages 791--798, New York, NY, USA, 2007. ACM.

Digital Library

[22]

B. M. Sarwar, G. Karypis, J. A. Konstan, and J. Riedl. Item-based collaborative filtering recommendation algorithms. In Proceedings of the Tenth International World Wide Web Conference, WWW 10, Hong Kong, China, May 1-5, 2001, pages 285--295, 2001.

Digital Library

[23]

J. B. Schafer, J. Konstan, and J. Riedl. Recommender systems in e-commerce. In Proceedings of the 1st ACM Conference on Electronic Commerce, EC '99, pages 158--166, New York, NY, USA, 1999. ACM.

Digital Library

[24]

R. Socher, C. C. Lin, A. Y. Ng, and C. D. Manning. Parsing Natural Scenes and Natural Language with Recursive Neural Networks. In Proceedings of the 26th International Conference on Machine Learning (ICML), 2011.

[25]

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929--1958, 2014.

Digital Library

[26]

Theano Development Team. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints, abs/1605.02688, May 2016.

[27]

V. Vapnik and A. Vashist. A new learning paradigm: Learning using privileged information. Neural Networks, 22(5-6):544--557, 2009.

Digital Library

[28]

H. Wang, N. Wang, and D.-Y. Yeung. Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '15, pages 1235--1244, New York, NY, USA, 2015. ACM.

Digital Library

[29]

M. Weimer, A. Karatzoglou, Q. V. Le, and A. J. Smola. Maximum margin matrix factorization for collaborative ranking. In NIPS, 2007.

[30]

X. Xu, J. T. Zhou, I. W. Tsang, Z. Qin, R. S. M. Goh, and Y. Liu. Simple and efficient learning using privileged information. CoRR, abs/1604.01518, 2016.

[31]

Y. Zhang, H. Dai, C. Xu, J. Feng, T. Wang, J. Bian, B. Wang, and T.-Y. Liu. Sequential click prediction for sponsored search with recurrent neural networks. In C. E. Brodley and P. Stone, editors, AAAI, pages 1369--1375. AAAI Press, 2014.

Digital Library

[32]

J. T. Zhou, X. Xu, S. J. Pan, I. W. Tsang, Z. Qin, and R. S. M. Goh. Transfer hashing with privileged information. In IJCAI, 2016.

Cited By

Ding YZhang ZWang B(2025)Category-integrated Dual-Task Graph Neural Networks for session-based recommendationExpert Systems with Applications10.1016/j.eswa.2024.125784263(125784)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125784
Abbasi MBernardo MVáz PSilva JMartins P(2024)Revisiting Database Indexing for Parallel and Accelerated Computing: A Comprehensive Study and Novel ApproachesInformation10.3390/info1508042915:8(429)Online publication date: 24-Jul-2024
https://doi.org/10.3390/info15080429
Mao YLin SCheng Y(2024)A Job Recommendation Model Based on a Two-Layer Attention MechanismElectronics10.3390/electronics1303048513:3(485)Online publication date: 24-Jan-2024
https://doi.org/10.3390/electronics13030485
Show More Cited By

Improved Recurrent Neural Networks for Session-based Recommendations
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals

Recommendations

Parallel Recurrent Neural Network Architectures for Feature-rich Session-based Recommendations
RecSys '16: Proceedings of the 10th ACM Conference on Recommender Systems

Real-life recommender systems often face the daunting task of providing recommendations based only on the clicks of a user session. Methods that rely on user profiles -- such as matrix factorization -- perform very poorly in this setting, thus item-to-...
Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks
RecSys '17: Proceedings of the Eleventh ACM Conference on Recommender Systems

Session-based recommendations are highly relevant in many modern on-line services (e.g. e-commerce, video streaming) and recommendation settings. Recently, Recurrent Neural Networks have been shown to perform very well in session-based settings. While ...
News Session-Based Recommendations using Deep Neural Networks
DLRS 2018: Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems

News recommender systems are aimed to personalize users experiences and help them to discover relevant articles from a large and dynamic search space. Therefore, news domain is a challenging scenario for recommendations, due to its sparse user profiling, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

DLRS 2016: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems

September 2016

47 pages

ISBN:9781450347952

DOI:10.1145/2988450

Conference Chairs:
Alexandros Karatzoglou
Telefonica Research
,
Balázs Hidasi
Gravity Research & Development
,
Domonkos Tikk
Gravity Research Development
,
Oren Sar-Shalom
IBM Research
,
Haggai Roitman
IBM Research
,
Bracha Shapira
Ben-Gurion University
,
Lior Rokach
Ben-Gurion University

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

IBMR: IBM Research

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

DLRS 2016

DLRS 2016: Workshop on Deep Learning for Recommender Systems

September 15, 2016

MA, Boston, USA

Acceptance Rates

Overall Acceptance Rate 11 of 27 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

499
Total Citations
View Citations
2,240
Total Downloads

Downloads (Last 12 months)219
Downloads (Last 6 weeks)23

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ding YZhang ZWang B(2025)Category-integrated Dual-Task Graph Neural Networks for session-based recommendationExpert Systems with Applications10.1016/j.eswa.2024.125784263(125784)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125784
Abbasi MBernardo MVáz PSilva JMartins P(2024)Revisiting Database Indexing for Parallel and Accelerated Computing: A Comprehensive Study and Novel ApproachesInformation10.3390/info1508042915:8(429)Online publication date: 24-Jul-2024
https://doi.org/10.3390/info15080429
Mao YLin SCheng Y(2024)A Job Recommendation Model Based on a Two-Layer Attention MechanismElectronics10.3390/electronics1303048513:3(485)Online publication date: 24-Jan-2024
https://doi.org/10.3390/electronics13030485
Wang LJin D(2024)A Time-Sensitive Graph Neural Network for Session-Based New Item RecommendationElectronics10.3390/electronics1301022313:1(223)Online publication date: 3-Jan-2024
https://doi.org/10.3390/electronics13010223
Mao XLi LHe J(2024)Multi-Granularity and Multi-Interest Contrast-Enhanced Hypergraph Convolutional Networks for Session RecommendationApplied Sciences10.3390/app1418829314:18(8293)Online publication date: 14-Sep-2024
https://doi.org/10.3390/app14188293
Celik EIlhan Omurca S(2024)Skip-Gram and Transformer Model for Session-Based RecommendationApplied Sciences10.3390/app1414635314:14(6353)Online publication date: 21-Jul-2024
https://doi.org/10.3390/app14146353
Vuong Nguyen L(2024)Classifications, evaluation metrics, datasets, and domains in recommendation services: A surveyInternational Journal of Hybrid Intelligent Systems10.3233/HIS-24000320:2(85-100)Online publication date: 11-Jun-2024
https://doi.org/10.3233/HIS-240003
Vaghari HAghdam MEmami H(2024)Diarec: Dynamic Intention-Aware Recommendation with Attention-Based Context-Aware Item Attributes ModelingJournal of Artificial Intelligence and Soft Computing Research10.2478/jaiscr-2024-001014:2(171-189)Online publication date: 19-Mar-2024
https://doi.org/10.2478/jaiscr-2024-0010
Kumar AKwatra CGiri AKumar AKumar SPoudel K(2024)Anime Recommendation SystemSSRN Electronic Journal10.2139/ssrn.4491482Online publication date: 2024
https://doi.org/10.2139/ssrn.4491482
Li ZYang CChen YWang XChen HXu GYao LSheng M(2024)Graph and Sequential Neural Networks in Session-based Recommendation: A SurveyACM Computing Surveys10.1145/369641357:2(1-37)Online publication date: 18-Sep-2024
https://dl.acm.org/doi/10.1145/3696413
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents