extended-abstract

How Users Ride the Carousel: Exploring the Design of Multi-List Recommender Interfaces From a User Perspective

Authors:

Benedikt Loepp,

Jürgen ZieglerAuthors Info & Claims

RecSys '23: Proceedings of the 17th ACM Conference on Recommender Systems

Pages 1090 - 1095

https://doi.org/10.1145/3604915.3610638

Published: 14 September 2023 Publication History

Abstract

Multi-list interfaces are widely used in recommender systems, especially in industry, showing collections of recommendations, one below the other, with items that have certain commonalities. The composition and order of these “carousels” are usually optimized by simulating user interaction based on probabilistic models learned from item click data. Research that actually involves users is rare, with only few studies investigating general user experience in comparison to conventional recommendation lists. Hence, it is largely unknown how specific design aspects such as carousel type and length influence the individual perception and usage of carousel-based interfaces. This paper seeks to fill this gap through an exploratory user study. The results confirm previous assumptions about user behavior and provide first insights into the differences in decision making in the presence of multiple recommendation carousels.

Supplemental Material

PDF File

Screenshot of the system and details about the questionnaire.

Download
846.60 KB

References

[1]

Hanna Beißert, Meike Köhler, Marina Rempel, and Constanze Beierlein. 2015. Deutschsprachige Kurzskala zur Messung des Konstrukts Need for Cognition NFC-K. Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS) (2015).

[2]

Walid Bendada, Guillaume Salha, and Théo Bontempelli. 2020. Carousel Personalization in Music Streaming Apps with Contextual Bandits. In RecSys ’20: Proceedings of the 14th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 420–425.

Digital Library

[3]

Dirk Bollen, Bart P. Knijnenburg, Martijn C. Willemsen, and Mark P. Graus. 2010. Understanding Choice Overload in Recommender Systems. In RecSys ’10: Proceedings of the 4th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 63–70.

Digital Library

[4]

Li Chen and Pearl Pu. 2012. Critiquing-Based Recommenders: Survey and Emerging Trends. User Modeling and User-Adapted Interaction 22, 1-2 (2012), 125–150.

Digital Library

[5]

Nicolò Felicioni, Maurizio Ferrari Dacrema, and Paolo Cremonesi. 2021. Measuring the User Satisfaction in a Recommendation Interface with Multiple Carousels. In IMX ’21: Proceedings of the 2nd ACM International Conference on Interactive Media Experiences. ACM, New York, NY, USA, 212–217.

Digital Library

[6]

Maurizio Ferrari Dacrema, Nicolò Felicioni, and Paolo Cremonesi. 2022. Offline Evaluation of Recommender Systems in a User Interface With Multiple Carousels. Frontiers in Big Data 5 (2022), 910030:1–910030:21.

[7]

Katherine Hamilton, Shin-I Shih, and Susan Mohammed. 2016. The Development and Validation of the Rational and Intuitive Decision Styles Scale. Journal of Personality Assessment 98, 5 (2016), 523–535.

[8]

Chen He, Denis Parra, and Katrien Verbert. 2016. Interactive Recommender Systems: A Survey of the State of the Art and Future Research Challenges and Opportunities. Expert Systems with Applications 56 (2016), 9–27.

Digital Library

[9]

Diana C. Hernandez-Bocanegra and Jürgen Ziegler. 2020. Explaining Review-Based Recommendations: Effects of Profile Transparency, Presentation Style and User Characteristics. i-com – Journal of Interactive Media 19, 3 (2020), 181–200.

[10]

Dietmar Jannach, Mathias Jesse, Michael Jugovac, and Christoph Trattner. 2021. Exploring Multi-List User Interfaces for Similar-Item Recommendations. In UMAP ’21: Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization. ACM, New York, NY, USA, 224–228.

Digital Library

[11]

Yucheng Jin, Nava Tintarev, Nyi Nyi Htun, and Katrien Verbert. 2020. Effects of Personal Characteristics in Control-Oriented User Interfaces for Music Recommender Systems. User Modeling and User-Adapted Interaction 30, 2 (2020), 199–249.

Digital Library

[12]

Yucheng Jin, Nava Tintarev, and Katrien Verbert. 2018. Effects of Personal Characteristics on Music Recommender Systems with Different Levels of Controllability. In RecSys ’18: Proceedings of the 12th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 13–21.

Digital Library

[13]

Michael Jugovac and Dietmar Jannach. 2017. Interacting with Recommenders – Overview and Research Directions. ACM Transactions on Interactive Intelligent Systems 7, 3 (2017), 10:1–10:46.

Digital Library

[14]

Timm Kleemann, Magdalena Wagner, Benedikt Loepp, and Jürgen Ziegler. 2021. Modeling User Interaction at the Convergence of Filtering Mechanisms, Recommender Algorithms and Advisory Components. In Mensch & Computer 2021 – Tagungsband. ACM, New York, NY, USA, 531–543.

[15]

Bart P. Knijnenburg, Martijn C. Willemsen, and Alfred Kobsa. 2011. A Pragmatic Procedure to Support the User-Centric Evaluation of Recommender Systems. In RecSys ’11: Proceedings of the 5th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 321–324.

Digital Library

[16]

Pigi Kouki, James Schaffer, Jay Pujara, John O’Donovan, and Lise Getoor. 2019. Personalized Explanations for Hybrid Recommender Systems. In IUI ’19: Proceedings of the 24th International Conference on Intelligent User Interfaces. ACM, New York, NY, USA, 379–390.

Digital Library

[17]

Johannes Kunkel and Jürgen Ziegler. 2023. A Comparative Study of Item Space Visualizations for Recommender Systems. International Journal of Human-Computer Studies 172 (2023).

Digital Library

[18]

Yu Liang and Martijn C. Willemsen. 2021. The Role of Preference Consistency, Defaults and Musical Expertise in Users’ Exploration Behavior in a Genre Exploration Recommender. In RecSys ’21: Proceedings of the 15th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 230–240.

Digital Library

[19]

Chieh Lo, Hongliang Yu, Xin Yin, Krutika Shetty, Changchen He, Kathy Hu, Justin M. Platz, Adam Ilardi, and Sriganesh Madhvanath. 2021. Page-Level Optimization of E-Commerce Item Recommendations. In RecSys ’21: Proceedings of the 15th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 495–504.

Digital Library

[20]

Benedikt Loepp, Tim Donkers, Timm Kleemann, and Jürgen Ziegler. 2019. Interactive Recommending with Tag-Enhanced Matrix Factorization (TagMF). International Journal of Human-Computer Studies 121 (2019), 21–41.

[21]

Benedikt Loepp and Jürgen Ziegler. 2019. Towards Interactive Recommending in Model-Based Collaborative Filtering Systems. In RecSys ’19: Proceedings of the 13th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 546–547.

Digital Library

[22]

James McInerney, Benjamin Lacker, Samantha Hansen, Karl Higley, Hugues Bouchard, Alois Gruson, and Rishabh Mehrotra. 2018. Explore, Exploit, and Explain: Personalizing Explainable Recommendations with Bandits. In RecSys ’18: Proceedings of the 12th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 31–39.

Digital Library

[23]

Martijn Millecamp, Nyi Nyi Htun, Cristina Conati, and Katrien Verbert. 2019. To Explain or Not to Explain: The Effects of Personal Characteristics When Explaining Music Recommendations. In IUI ’19: Proceedings of the 24th International Conference on Intelligent User Interfaces. ACM, New York, NY, USA, 397–407.

Digital Library

[24]

Gergana Y. Nenkov, Maureen Morrin, Andrew Ward, Barry Schwartz, and John Hulland. 2008. A Short Form of the Maximization Scale: Factor Structure, Reliability and Validity Studies. Judgment and Decision Making 3 (2008), 371–388.

[25]

Pearl Pu, Li Chen, and Rong Hu. 2011. A User-Centric Evaluation Framework for Recommender Systems. In RecSys ’11: Proceedings of the 5th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 157–164.

Digital Library

[26]

Behnam Rahdari, Peter Brusilovsky, and Alireza Javadian Sabet. 2021. Controlling Personalized Recommendations in Two Dimensions with a Carousel-Based Interface. In IntRS ’21: Proceedings of the 8th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems. 112–122.

[27]

Behnam Rahdari, Branislav Kveton, and Peter Brusilovsky. 2022. The Magic of Carousels: Single vs. Multi-List Recommender Systems. In HT ’22: Proceedings of the 33rd ACM Conference on Hypertext and Social Media. ACM, New York, NY, USA, 166–174.

Digital Library

[28]

Benjamin Scheibehenne, Rainer Greifeneder, and Peter M. Todd. 2010. Can There Ever Be Too Many Options? A Meta-Analytic Review of Choice Overload. Journal of Consumer Research 37, 3 (2010), 409–425.

[29]

Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. 2017. Design and Evaluation of a Short Version of the User Experience Questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence 4, 6 (2017), 103–108.

[30]

Sanidhya Singal, Piyush Singh, and Manjeet Dahiya. 2021. Automatic Collection Creation and Recommendation. In RecSys ’21: Proceedings of the 15th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 633–638.

Digital Library

[31]

Alain D. Starke, Edis Asotic, and Christoph Trattner. 2021. “Serving Each User”: Supporting Different Eating Goals Through a Multi-List Recommender Interface. In RecSys ’21: Proceedings of the 15th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 124–132.

Digital Library

[32]

Alain D. Starke, Justyna Sedkowska, Mihir Chouhan, and Bruce Ferwerda. 2022. Examining Choice Overload across Single-List and Multi-List User Interfaces. In IntRS ’22: Proceedings of the 8th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems.

[33]

Chao-Yuan Wu, Christopher V. Alvino, Alexander J. Smola, and Justin Basilico. 2016. Using Navigation to Improve Recommendations in Real-Time. In RecSys ’16: Proceedings of the 10th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 341–348.

Digital Library

[34]

Liang Wu, Mihajlo Grbovic, and Jundong Li. 2021. Toward User Engagement Optimization in 2D Presentation. In WSDM ’21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining. ACM, New York, NY, USA, 1047–1055.

Digital Library

[35]

Qian Zhao, Shuo Chang, F. Maxwell Harper, and Joseph A. Konstan. 2016. Gaze Prediction for Recommender Systems. In RecSys ’16: Proceedings of the 10th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 131–138.

Digital Library

Cited By

Rahdari BBrusilovsky P(2024)CARE: An Infrastructure for Evaluation of Carousel-Based Recommender InterfacesCompanion Proceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640544.3645223(41-44)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640544.3645223
Smits ADetweiler CBartels EPott Kvan der Stappen Evan Rossen SCardona KShayan SZiegler Jvan Turhout K(2023)Assessing the Utility of an Interaction Qualities Framework in Systematizing the Evaluation of User ControlDesign for Equality and Justice10.1007/978-3-031-61698-3_17(185-195)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-61698-3_17

Index Terms

How Users Ride the Carousel: Exploring the Design of Multi-List Recommender Interfaces From a User Perspective
1. Human-centered computing
  1. Interaction design
    1. Interaction design process and methods
      1. User interface design
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Exploring Multi-List User Interfaces for Similar-Item Recommendations
UMAP '21: Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization

On many e-commerce and media streaming sites, the user interface (UI) consists of multiple lists of item suggestions. The items in each list are usually chosen based on pre-defined strategies and, e.g., show movies of the same genre or category. Such ...
Examining the User Evaluation of Multi-List Recommender Interfaces in the Context of Healthy Recipe Choices
Multi-list recommender systems have become widespread in entertainment and e-commerce applications. Yet, extensive user evaluation research is missing. Since most content is optimized toward a user’s current preferences, this may be problematic in ...
User Personality and User Satisfaction with Recommender Systems

In this study, we show that individual users' preferences for the level of diversity, popularity, and serendipity in recommendation lists cannot be inferred from their ratings alone. We demonstrate that we can extract strong signals about individual ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

RecSys '23: Proceedings of the 17th ACM Conference on Recommender Systems

September 2023

1406 pages

ISBN:9798400702419

DOI:10.1145/3604915

Copyright © 2023 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2023

Check for updates

Author Tags

Qualifiers

Extended-abstract
Research
Refereed limited

Conference

RecSys '23

Sponsor:

RecSys '23: Seventeenth ACM Conference on Recommender Systems

September 18 - 22, 2023

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
230
Total Downloads

Downloads (Last 12 months)112
Downloads (Last 6 weeks)7

Reflects downloads up to 30 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rahdari BBrusilovsky P(2024)CARE: An Infrastructure for Evaluation of Carousel-Based Recommender InterfacesCompanion Proceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640544.3645223(41-44)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640544.3645223
Smits ADetweiler CBartels EPott Kvan der Stappen Evan Rossen SCardona KShayan SZiegler Jvan Turhout K(2023)Assessing the Utility of an Interaction Qualities Framework in Systematizing the Evaluation of User ControlDesign for Equality and Justice10.1007/978-3-031-61698-3_17(185-195)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-61698-3_17

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents