Abstract
We put forward an innovative use of probabilistic topic modeling (PTM) intertwined with reinforcement learning (RL), to provide personalized recommendations. Specifically, we model items under recommendation as mixtures of latent topics following a distribution with Dirichlet priors; this can be achieved via the exploitation of crowdsourced information for each item. Similarly, we model the user herself as an “evolving” document represented by its respective mixture of latent topics. The user’s topic distribution is appropriately updated each time she consumes an item. Recommendations are subsequently based on the divergence between the topic distributions of the user and available items. However, to tackle the exploration versus exploitation dilemma, we apply RL to vary the user’s topic distribution update rate. Our method is immune to the notorious “cold start” problem, and it can effectively cope with changing user preferences. Moreover, it is shown to be competitive against state-of-the-art algorithms, outperforming them in terms of sequential performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Datasets: Movielens 1M: 1M ratings, 6,040 users, 3,952 movies. Movielens 10M: 10M ratings, 71,567 users, 10,681 and movies. We found nontrivial data on Wikipedia for 3,137 movies on the 1M dataset and 8,721 movies on the 10M dataset.
- 2.
BYLI had been evaluated on MovieLens 1M only.
References
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Wang, C., Blei, D.M.: Collaborative topic modeling for recommending scientific articles. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 448–456. ACM (2011)
Babas, K., Chalkiadakis, G., Tripolitakis, E.: You are what you consume: a Bayesian method for personalized recommendations. In: Proceedings of the 7th ACM Conference on Recommender Systems, pp. 221–228. ACM (2013)
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009). IEEE
Melville, P., Mooney, R.J., Nagarajan, R.: Content-boosted collaborative filtering for improved recommendations. In: AAAI/IAAI, pp. 187–192 (2002)
Mooney, R.J., Roy, L.: Content-based book recommending using learning for text categorization. In: Proceedings of the Fifth ACM Conference on Digital Libraries, pp. 195–204. ACM (2000)
Bowling, M., Veloso, M.: Rational and convergent learning in stochastic games. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence, vol. 2, pp. 1021–1026 (2001)
Bowling, M., Veloso, M.: Multiagent learning using a variable learning rate. Artif. Intell. 136(2), 215–250 (2002). Elsevier
Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning. MIT Press, Cambridge (1998)
Koren, Y.: The bellkor solution to the netflix grand prize. Netflix Prize Doc. 81, 1–10 (2009)
Piotte, M., Chabbert, M.: The pragmatic theory solution to the netflix grand prize. Netflix Prize Doc. (2009). http://www.netflixprize.com/assets/GrandPrize2009_BPC_PragmaticTheory.pdf
Toscher, A., Jahrer, M., Bell, R.M.: The bigchaos solution to the netflix grand prize. Netflix Prize Doc. (2009). http://www.netflixprize.com/assets/GrandPrize2009_BPC_BigChaos.pdf
Langseth, H., Nielsen, T.D.: A latent model for collaborative filtering. Int. J. Approx. Reason. 53(4), 447–466 (2012)
Bresler, G., Chen, G.H., Shah, D.: A latent source model for online collaborative filtering. In: Advances in Neural Information Processing Systems, pp. 3347–3355 (2014)
Hofmann, T.: Latent semantic models for collaborative filtering. ACM Trans. Inf. Syst. (TOIS) 22(1), 89–115 (2004). ACM
Hu, Y., Koren, Y., Volinsky, C.: Collaborative filtering for implicit feedback datasets. In: Eighth IEEE International Conference on Data Mining, ICDM 2008, pp. 263–272. IEEE (2008)
Blei, D.M.: Probabilistic topic models. Commun. ACM 55(4), 77–84 (2012). ACM
Koren, Y., Bell, R.: Advances in collaborative filtering. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 145–186. Springer, Heidelberg (2011). doi:10.1007/978-0-387-85820-3_5
Agarwal, D., Chen, B.-C.: fLDA: matrix factorization through latent dirichlet allocation. In: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 91–100. ACM (2010)
Ling, G., Lyu, M.R., King, I.: Ratings meet reviews, a combined approach to recommend. In: Proceedings of the 8th ACM Conference on Recommender Systems, pp. 105–112. ACM (2014)
Kurimo, M.: Indexing audio documents by using latent semantic analysis and SOM. Elsevier (1999)
Wallach, H.M., Murray, I., Salakhutdinov, R., Mimno, D.: Evaluation methods for topic models. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1105–1112. ACM (2009)
Hoffman, M., Bach, F.R., Blei, D.M.: Online learning for latent dirichlet allocation. In: Advances in Neural Information Processing Systems, pp. 856–864 (2010)
McCallum, A.K.: MALLET: A Machine Learning for Language Toolkit (2002). http://mallet.cs.umass.edu
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Tripolitakis, E., Chalkiadakis, G. (2017). Probabilistic Topic Modeling, Reinforcement Learning, and Crowdsourcing for Personalized Recommendations. In: Criado Pacheco, N., Carrascosa, C., Osman, N., Julián Inglada, V. (eds) Multi-Agent Systems and Agreement Technologies. EUMAS AT 2016 2016. Lecture Notes in Computer Science(), vol 10207. Springer, Cham. https://doi.org/10.1007/978-3-319-59294-7_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-59294-7_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-59293-0
Online ISBN: 978-3-319-59294-7
eBook Packages: Computer ScienceComputer Science (R0)