Abstract
Recommender systems have evolved in recent years into sophisticated support tools that assist users in dealing with the decisions faced in everyday life. Recommender systems were designed to be invaluable in situations, where a large number of options are available, such as deciding what to watch on television, what information to access online, what to purchase in a supermarket, or what to eat. Recommender system evaluations are carried out typically during the design phase of recommender systems to understand the suitability of approaches to the recommendation process, in the usability phase to gain insight into interfacing and user acceptance, and in live user studies to judge the uptake of recommendations generated and impact of the recommender system. In this chapter, we present a detailed overview of evaluation techniques for recommender systems covering a variety of tried and tested methods and metrics. We illustrate their use by presenting a case study that investigates the applicability of a suite of recommender algorithms in a recipe recommender system aimed to assist individuals in planning their daily food intake. The study details an offline evaluation, which compares algorithms, such as collaborative, content-based, and hybrid methods, using multiple performance metrics, to determine the best candidate algorithm for a recipe recommender application.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
Note that the number of relevant items varies across users, as each profile contains a different number of ratings.
References
Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans knowl Data Eng 17: 734–749
Bambini R, Cremonesi P, Turrin R (2011) A recommender system for an iptv service provider: a real large-scale production environment. In: Recommender systems handbook. Springer, Boston, pp 299–331
Berkovsky S, Freyne J (2010) Group-based recipe recommendations: analysis of data aggregation strategies. In: Amatriain X, Torrens M, Resnick P, Zanker M (eds) RecSys. ACM, New York, pp 111–118
Billsus D, Pazzani M (1998) Learning collaborative information filters. In: Proceedings of the fifteenth international conference on machine learning, vol 54, p 48
Bollen D, Knijnenburg B, Willemsen M, Graus M (2010) Understanding choice overload in recommender systems. In: Proceedings of the fourth ACM conference on recommender systems. ACM, New York, pp 63–70
Burke R (2007) The adaptive web, chap. In: Hybrid web recommender systems. Springer, Berlin/Heidelberg, pp 377–408. URL http://dl.acm.org/citation.cfm?id=1768197.1768211
Burke R, Hammond K, Young B (1996) Knowledge-based navigation of complex information spaces. In: Proceedings of the national conference on artificial intelligence, pp 462–468
Cremonesi P, Garzotto F, Negro S, Papadopoulos A, Turrin R (2011) Comparative evaluation of recommender system quality. In: Proceedings of the 2011 annual conference extended abstracts on human factors in computing systems. ACM, Vancouver, pp 1927–1932
Desrosiers C, Karypis G (2011) A comprehensive survey of neighborhood-based recommendation methods. In: Recommender systems handbook. Springer, Boston, pp 107–144
Farzan R, Coyle M, Freyne J, Brusilovsky P, Smyth B (2007) Assist: adaptive social support for information space traversal. In: Proceedings of the eighteenth conference on hypertext and hypermedia, HT ’07. ACM, New York, pp 199–208. doi:http://doi.acm.org/10.1145/1286240.1286299. URL:http://doi.acm.org/10.1145/1286240.1286299
Farzan R, DiMicco J, Millen D, Geyer W, Brownholtz E (2008) Results from deploying a participation incentive mechanism within the enterprise
Freyne J, Berkovsky S (2010) Intelligent food planning: personalized recipe recommendation. In: Proceedings of the 2010 international conference on intelligent user interfaces (IUI 2010), pp 321–324
Freyne J, Berkovsky S, Baghaei N, Kimani S, Smith G (2011) Personalized techniques for lifestyle change. In: Proceedings artificial intelligence in medicine, AIME, Bled, Slovenia, 2–6 Jul 2011, pp 139–148
Herlocker J, Konstan J, Borchers A, Riedl J (1999) An algorithmic framework for performing collaborative filtering. In: Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval. ACM, New York, pp 230–237
Herlocker JL, Konstan JA, Terveen LG, Riedl JT (2004) Evaluating collaborative filtering recommender systems. ACM Trans Inf Syst 22(1):5–53. doi:http://doi.acm.org/10.1145/963770.963772
Koenigstein N, Dror G, Koren Y (2011) Yahoo! music recommendations: modeling music ratings with temporal dynamics and item taxonomy. In: Mobasher B, Burke RD, Jannach D, Adomavicius G (eds) RecSys. ACM, Chicago, pp 165–172
Kohavi R, Longbotham R, Sommerfield D, Henne R (2009) Controlled experiments on the web: survey and practical guide. Data Min Knowl Dis 18(1):140–181
Konstan J, Miller B, Maltz D, Herlocker J, Gordon L, Riedl J (1997) GroupLens: applying collaborative filtering to Usenet news. Commun ACM 40(3):87
Koren Y, Bell R (2011) Advances in collaborative filtering. In: Recommender systems handbook. Springer, Boston, pp 145–186
Lops P, Gemmis M, Semeraro G (2011) Content-based recommender systems: state of the art and trends. In: Recommender systems handbook. Springer, Boston, pp 73–105
Masthoff J (2004) Group modeling: selecting a sequence of television items to suit a group of viewers. User Model User-Adap Interact 14(1):37–85
McJones P (1997) Eachmovie collaborative filtering dataset, DEC systems research center. http://www.research.compaq.com/src/eachmovie/
McNee S, Albert I, Cosley D, Gopalkrishnan P, Lam S, Rashid A, Konstan J, Riedl J (2002) On the recommending of citations for research papers. In: Proceedings of the 2002 ACM conference on computer supported cooperative work. ACM, New York, pp 116–125
Noakes M, Clifton P (2005) The CSIRO total wellbeing diet book. Penguin Group, Australia
Noakes M, Clifton P (2006) The CSIRO total wellbeing diet book 2. Penguin Group, Australia
van Pinxteren Y, Geleijnse G, Kamsteeg P (2011) Deriving a recipe similarity measure for recommending healthful meals. In: Proceedings of the 2011 international conference on intelligent user interfaces, IUI 2011, pp 105–114
Pu P, Chen L (2006) Trust building with explanation interfaces. In: Proceedings of the 11th international conference on intelligent user interfaces, IUI ’06. ACM, New York, pp 93–100. doi:http://doi.acm.org/10.1145/1111449.1111475. URL:http://doi.acm.org/10.1145/1111449.1111475
Quinlan J (1992) Learning with continuous classes. In: Proceedings of the 5th Australian joint conference on artificial intelligence, Citeseer, pp 343–348
Rashid A, Karypis G, Riedl J (2008) Learning preferences of new users in recommender systems: an information theoretic approach. ACM SIGKDD Explor Newslett 10(2):90–100
Rashid AM, Albert I, Cosley D, Lam SK, McNee SM, Konstan JA, Riedl J (2001) Getting to know you: learning new user preferences in recommender systems. In: Proceedings of the 7th international conference on intelligent user interfaces, IUI ’02. ACM, New York, pp 127–134. doi:http://doi.acm.org/10.1145/502716.502737. URL:http://doi.acm.org/10.1145/502716.502737
Ricci F, Rokach L, Shapira B, Kantor P (2010) Recommender systems handbook. Springer, New York
Said A, Berkovsky S, De Luca EW (2010) Putting things in context: challenge on context-aware movie recommendation. In: Proceedings of the workshop on context-aware movie recommendation, CAMRa ’10. ACM, New York, pp 2–6. doi:http://doi.acm.org/10.1145/1869652.1869665. URL:http://doi.acm.org/10.1145/1869652.1869665
Shani G, Gunawardana A (2009) Evaluating recommender systems. Microsoft research report MSR-TR-2009-159, Nov 2009
Swearingen K, Sinha R (2001) Beyond algorithms: an hci perspective on recommender systems. In: ACM SIGIR 2001 workshop on recommender systems, Citeseer
Wang Y, Witten I (1996) Induction of model trees for predicting continuous classes
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this chapter
Cite this chapter
Freyne, J., Berkovsky, S. (2013). Evaluating Recommender Systems for Supportive Technologies. In: Martín, E., Haya, P., Carro, R. (eds) User Modeling and Adaptation for Daily Routines. Human–Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-4471-4778-7_8
Download citation
DOI: https://doi.org/10.1007/978-1-4471-4778-7_8
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-4777-0
Online ISBN: 978-1-4471-4778-7
eBook Packages: Computer ScienceComputer Science (R0)