Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3460231.3470929acmconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
extended-abstract

Perspectives on the Evaluation of Recommender Systems (PERSPECTIVES)

Published: 13 September 2021 Publication History

Abstract

Evaluation is a cornerstone in the process of developing and deploying recommender systems. The PERSPECTIVES workshop brought together academia and industry to critically reflect on the evaluation of recommender systems. Particularly, the workshop aimed to shed light on the different, and maybe even diverging or contradictory perspectives on the evaluation of recommender systems. Papers reporting a reflection on problems regarding recommender systems evaluation and lessons learned were solicited. The workshop combined flash presentations of accepted papers, a keynote from industry, and an interactive part with discussions in break-out rooms as well as in the plenum. The workshop complemented the program of the main conference as it emphasized problems and lessons learned, fostered exchange integrating various perspectives on evaluation, and sought to move the recommender systems community forward as an outcome of the workshop.

References

[1]
Himan Abdollahpouri, Gediminas Adomavicius, Robin Burke, Ido Guy, Dietmar Jannach, Toshihiro Kamishima, Jan Krasnodebski, and Luiz Pizzato. 2020. Multistakeholder recommendation: Survey and research directions. User Modeling and User-Adapted Interaction(2020). https://doi.org/10.1007/s11257-019-09256-1
[2]
Christine Bauer and Eva Zangerle. 2019. Leveraging Multi-Method Evaluation for Multi-Stakeholder Settings. In Proceedings of the 1st Workshop on the Impact of Recommender Systems (Copenhagen, Denmark, 19 September) (ImpactRS ’19), Oren Sar Shalom, Dietmar Jannach, and Ido Guy (Eds.). 3 pages. arxiv:2001.04348arXiv:2001.04348.
[3]
Joeran Beel, Stefan Langer, Marcel Genzmehr, Bela Gipp, Corinna Breitinger, and Andreas Nürnberger. 2013. Research Paper Recommender System Evaluation: A Quantitative Literature Survey. In Proceedings of the International Workshop on Reproducibility and Replication in Recommender Systems Evaluation (Hong Kong, China) (RepSys ’13). ACM, New York, NY, USA, 15–22. https://doi.org/10.1145/2532508.2532512
[4]
Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl. 2004. Evaluating Collaborative Filtering Recommender Systems. ACM Transaction on Information Systems 22, 1 (Jan. 2004), 5–53. https://doi.org/10.1145/963770.963772
[5]
Dietmar Jannach and Gediminas Adomavicius. 2016. Recommendations with a Purpose. In Proceedings of the 10th ACM Conference on Recommender Systems (Boston, MA, USA) (RecSys ’16). ACM, New York, NY, USA, 7–10. https://doi.org/10.1145/2959100.2959186
[6]
Dietmar Jannach and Christine Bauer. 2020. Escaping the McNamara Fallacy: Toward More Impactful Recommender Systems Research. AI Magazine 41, 4 (2020), 79–95. https://doi.org/10.1609/aimag.v41i4.5312
[7]
Dietmar Jannach, Oren Sar Shalom, and Joseph A Konstan. 2019. Towards More Impactful Recommender Systems Research. In Proceedings of the 1st Workshop on the Impact of Recommender Systems co-located with 13th ACM Conference on Recommender Systems, ImpactRS@RecSys 2019) (Copenhagen, Denmark) (CEUR Workshop Proceedings, Vol. 2462). CEUR-WS.org. http://ceur-ws.org/Vol-2462/short6.pdf
[8]
Alan Said, Domonkos Tikk, Klara Stumpf, Yue Shi, Martha Larson, and Paolo Cremonesi. 2012. Recommender Systems Evaluation: A 3D Benchmark. In Proceedings of the Workshop on Recommendation Utility Evaluation: Beyond RMSE (Dublin, Ireland) (RUE ’12, Vol. 910). CEUR Workshop Proceedings, 21–23. http://ceur-ws.org/Vol-910/

Cited By

View all
  • (2024)Report on the 3rd Workshop on the Perspectives on the Evaluation of Recommender Systems (PERSPECTIVES 2023) at RecSys 2023ACM SIGIR Forum10.1145/3642979.364300057:2(1-4)Online publication date: 22-Jan-2024
  • (2024)Reflections on Recommender Systems: Past, Present, and Future (INTROSPECTIVES)Proceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3687101(1237-1238)Online publication date: 8-Oct-2024
  • (2023)Report on the 2nd Workshop on the Perspectives on the Evaluation of Recommender Systems (PERSPECTIVES 2022) at RecSys 2022ACM SIGIR Forum10.1145/3582900.358291956:2(1-4)Online publication date: 31-Jan-2023

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
RecSys '21: Proceedings of the 15th ACM Conference on Recommender Systems
September 2021
883 pages
ISBN:9781450384582
DOI:10.1145/3460231
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 September 2021

Check for updates

Author Tags

  1. evaluation
  2. methods
  3. recommender systems

Qualifiers

  • Extended-abstract
  • Research
  • Refereed limited

Conference

RecSys '21: Fifteenth ACM Conference on Recommender Systems
September 27 - October 1, 2021
Amsterdam, Netherlands

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)12
  • Downloads (Last 6 weeks)2
Reflects downloads up to 18 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Report on the 3rd Workshop on the Perspectives on the Evaluation of Recommender Systems (PERSPECTIVES 2023) at RecSys 2023ACM SIGIR Forum10.1145/3642979.364300057:2(1-4)Online publication date: 22-Jan-2024
  • (2024)Reflections on Recommender Systems: Past, Present, and Future (INTROSPECTIVES)Proceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3687101(1237-1238)Online publication date: 8-Oct-2024
  • (2023)Report on the 2nd Workshop on the Perspectives on the Evaluation of Recommender Systems (PERSPECTIVES 2022) at RecSys 2022ACM SIGIR Forum10.1145/3582900.358291956:2(1-4)Online publication date: 31-Jan-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media