Abstract
Evaluating Recommender Systems (RSs) is a challenging issue that is significantly magnified by the multifaceted properties of RSs, which makes it insufficient to use only one metric to evaluate recommenders. This challenge necessitates the need for a unified evaluation model that comprehensively assesses multiple aspects of the recommender. This position paper proposes a cognition-based comprehensive evaluation to evaluate the main activities of RSs. We innovated the proposed model based on the cognitive dimension of Bloom’s taxonomy, a widely used model for classifying learning objectives in the teaching area. We created a phase-wise mapping between RSs and Bloom’s taxonomy to come up with an overall evaluation for recommenders. Based on these connections, we believe that the proposed evaluation model would have the potential to support the decision of selecting the most appropriate recommender systems by giving a benchmarked score for different aspects of RSs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P. (eds.) Recommender Systems Handbook, pp. 257–297. Springer, Boston (2011). https://doi.org/10.1007/978-0-387-85820-3_8
Pu, P., Chen, L., Hu, R.: A user-centric evaluation framework for recommender systems. In: Proceedings of the Fifth ACM Conference on Recommender Systems, pp. 157–164. ACM, October 2011
Seminario, C.E., Wilson, D.C.: Robustness and accuracy tradeoffs for recommender systems under attack. In: FLAIRS Conference (2012)
Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005)
Kowald, D., Lex, E.: Evaluating tag recommender algorithms in real-world folksonomies: a comparative study. In: Proceedings of the 9th ACM Conference on Recommender Systems, pp. 265–268. ACM (2015)
Said, A., Tikk, D., Stumpf, K., Shi, Y., Larson, M., Cremonesi, P.: Recommender systems evaluation: a 3D benchmark. In: RUE@ RecSys, pp. 21–23 (2012)
Karthwohl, D.R., Anderson, W.: A revision of Bloom’s taxonomy: an overview theory into practice. The Ohio State University (2002)
Isinkaye, F.O., Folajimi, Y.O., Ojokoh, B.A.: Recommendation systems: principles, methods and evaluation. Egypt. Inform. J. 16(3), 261–273 (2015)
Avazpour, I., Pitakrat, T., Grunske, L., Grundy, J.: Dimensions and metrics for evaluating recommendation systems. In: Robillard, M., Maalej, W., Walker, R., Zimmermann, T. (eds.) Recommendation Systems in Software Engineering, pp. 245–273. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-45135-5_10
Salfner, F., Lenk, M., Malek, M.: A survey of online failure prediction methods. ACM Comput. Surv. (CSUR) 42(3), 10 (2010)
Smyth, B., McClave, P.: Similarity vs. diversity. In: Aha, D.W., Watson, I. (eds.) ICCBR 2001. LNCS (LNAI), vol. 2080, pp. 347–361. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44593-5_25
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Alslaity, A., Tran, T. (2018). Towards a Comprehensive Evaluation of Recommenders: A Cognition-Based Approach. In: Bagheri, E., Cheung, J. (eds) Advances in Artificial Intelligence. Canadian AI 2018. Lecture Notes in Computer Science(), vol 10832. Springer, Cham. https://doi.org/10.1007/978-3-319-89656-4_32
Download citation
DOI: https://doi.org/10.1007/978-3-319-89656-4_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-89655-7
Online ISBN: 978-3-319-89656-4
eBook Packages: Computer ScienceComputer Science (R0)