research-article

Decision making strategies differ in the presence of collaborative explanations: two conjoint studies

Authors:

Panagiotis SymeonidisAuthors Info & Claims

IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

Pages 291 - 302

https://doi.org/10.1145/3301275.3302304

Published: 17 March 2019 Publication History

Abstract

Rating-based summary statistics are ubiquitous in e-commerce, and often are crucial components in personalized recommendation mechanisms. Especially visual rating summarizations have been identified as important means to explain, why an item is presented or proposed to an user. Largely left unexplored, however, is the issue to what extent the descriptives of these rating summary statistics influence decision making of the online consumer. Therefore, we conducted a series of two conjoint experiments to explore how different summarizations of rating distributions (i.e., in the form of number of ratings, mean, variance, skewness, bimodality, or origin of the ratings) impact users' decision making. In a first study with over 200 participants, we identified that users are primarily guided by the mean and the number of ratings, and - to lesser degree - by the variance and origin of a rating. When probing the maximizing behavioral tendencies of our participants, other sensitivities regarding the summary of rating distributions became apparent. We thus instrumented a follow-up eye-tracking study to explore in more detail, how the choices of participants vary in terms of their decision making strategies. This second round with over 40 additional participants supported our hypothesis that users, who usually experience higher decision difficulty, follow compensatory decision strategies, and focus more on the decisions they make. We conclude by outlining how the results of these studies can guide algorithm development, and counterbalance presumable biases in implicit user feedback.

References

[1]

Behnoush Abdollahi and Olfa Nasraoui. 2017. Using Explainability for Constrained Matrix Factorization. In Proceedings of the Eleventh ACM Conference on Recommender Systems - RecSys '17. 79--83.

Digital Library

[2]

James Agarwal, Wayne S DeSarbo, Naresh K Malhotra, and Vithala R Rao. 2015. An interdisciplinary review of research in conjoint analysis: recent developments and directions for future research. Customer Needs and Solutions 2, 1 (2015), 19--40.

[3]

Mustafa Bilgic and Raymond J Mooney. 2005. Explaining Recommendations: Satisfaction vs. Promotion. Proceedings of Beyond Personalization 2005: A Workshop on the Next Stage of Recommender Systems Research at The 2005 International Conference on Intelligent User Interfaces (2005), 13--18.

[4]

Guillermo Carbonell and Matthias Brand. 2018. Choosing a Physician on Social Media: Comments and Ratings of Users are More Important than the Qualification of a Physician. International Journal of Human-Computer Interaction 34, 2 (2 2018), 117--128.

[5]

Nathan N Cheek and Barry Schwartz. 2016. On the meaning and measurement of maximization. Judgment and Decision making 11, 2 (2016), 126.

[6]

Li Chen, Marco de Gemmis, Alexander Felfernig, Pasquale Lops, Francesco Ricci, and Giovanni Semeraro. 2013. Human decision making and recommender systems. ACM Transactions on Interactive Intelligent Systems (TiiS) 3, 3 (2013), 17.

Digital Library

[7]

Minji Cho, Soyoung Kwon, Nooree Na, Hyeon-Jeong Suk, and Kun-Pyo Lee. 2015. The Elders Preference for Skeuomorphism as App Icon Style. Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA'15 (2015), 899--904.

Digital Library

[8]

Jaihak Chung and Vithala R Rao. 2012. A general consumer preference model for experience products: application to internet recommendation services. Journal of marketing research 49, 3 (2012), 289--305.

[9]

Ludovik Coba, Markus Zanker, and Laurens Rook. 2019. Decision Making Based on Bimodal Rating Summary Statistics - An Eye-Tracking Study of Hotels. Information and Communication Technologies in Tourism (ENTER).

[10]

Ludovik Coba, Markus Zanker, Laurens Rook, and Panagiotis Symeonidis. 2018. Exploring Users' Perception of Collaborative Explanation Styles. 2018 IEEE 20th Conference on Business Informatics (CBI) (2018), 70--78.

[11]

Ludovik Coba, Markus Zanker, Laurens Rook, and Panagiotis Symeonidis. 2018. Exploring Users' Perception of Rating Summary Statistics. In Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization - UMAP '18. ACM Press, New York, New York, USA, 353--354.

Digital Library

[12]

Dan Cosley, Shyong K. Lam, Istvan Albert, Joseph a. Konstan, and John Riedl. 2003. Is Seeing Believing? How Recommender System Interfaces Affect Users' Opinions. Proceedings of the Conference on Human Factors in Computing Systems (CHI '03) 5 (2003), 585--592.

Digital Library

[13]

Ilan Dar-Nimrod, Catherine D Rawn, Darrin R Lehman, and Barry Schwartz. 2009. The maximization paradox: The costs of seeking alternatives. Personality and Individual Differences 46, 5-6 (2009), 631--635.

[14]

Bart de Langhe, Philip M. Fernbach, and Donald R. Lichtenstein. 2016. Navigating by the stars: Investigating the actual and perceived validity of online user ratings. Journal of Consumer Research 42, 6 (2016), 817--833.

[15]

Wenjing Duan, Bin Gu, and Andrew B Whinston. 2008. Do online reviews matter?-An empirical investigation of panel data. Decision support systems 45, 4 (2008), 1007--1016.

Digital Library

[16]

Gerhard Friedrich and Markus Zanker. 2011. A Taxonomy for Generating Explanations in Recommender Systems. AI Magazine 32, 3 (2011), 90.

Digital Library

[17]

Matthias Fuchs and Markus Zanker. 2012. Multi-criteria ratings for recommender systems: An empirical analysis in the tourism domain. In Lecture Notes in Business Information Processing, Vol. 123 LNBIP. 100--111.

[18]

William H. Greene. 2012. Econometric analysis. Prentice Hall. 1188 pages.

[19]

A. Brett Hauber, Juan Marcos González, Catharina G.M. M Groothuis-oudshoorn, Thomas Prior, Deborah A. Marshall, Charles Cunningham, Maarten J. Ijzerman, and John F.P. P Bridges. 2016. Statistical Methods for the Analysis of Discrete Choice Experiments: A Report of the ISPOR Conjoint Analysis Good Research Practices Task Force. Value in Health 19, 4 (2016), 300--315.

[20]

John R Hauser and Vithala R Rao. 2004. Conjoint analysis, related modeling, and applications. In Marketing Research and Modeling: Progress and Prospects. Springer, 141--168.

[21]

Jonathan L Herlocker, Joseph A Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM conference on Computer supported cooperative work - CSCW '00. 241--250.

Digital Library

[22]

Kenneth Holmqvist, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost Van De Weijer. 2011. Eye tracking. A comprehensive guide to methods and measures. Oxford University Press. 537 pages.

[23]

Nan Hu, Jie Zhang, and Paul A Pavlou. 2009. Overcoming the J-shaped distribution of product reviews. Commun. ACM 52, 10 (2009), 144--147.

Digital Library

[24]

Sheena S Iyengar, Rachael E Wells, and Barry Schwartz. 2006. Doing better but feeling worse: Looking for the best job undermines satisfaction. Psychological Science 17, 2 (2006), 143--150.

[25]

Dietmar Jannach, Lukas Lerche, and Michael Jugovac. 2015. Item familiarity as a possible confounding factor in user-centric recommender systems evaluation. I-Com 14, 1 (2015), 29--39.

[26]

F. Reed Johnson, Emily Lancsar, Deborah Marshall, Vikram Kilambi, Axel Mühlbacher, Dean A. Regier, Brian W. Bresnahan, Barbara Kanninen, and John F.P. Bridges. 2013. Constructing experimental designs for discrete-choice experiments: Report of the ISPOR conjoint analysis experimental design good research practices task force. Value in Health 16, 1 (1 2013), 3--13.

[27]

Michael Jugovac, Ingrid Nunes, and Dietmar Jannach. 2018. Investigating the Decision-Making Behavior of Maximizers and Satisficers in the Presence of Recommendations. In 26th Conference on User Modeling, Adaptation and Personalization (UMAP'18).

Digital Library

[28]

Bart P Knijnenburg and Martijn C Willemsen. 2011. Each to His Own : How Different Users Call for Different Interaction Methods in Recommender Systems. Proceedings of the 5th ACM conference on Recommender systems - RecSys '11 (2011), 141--148.

Digital Library

[29]

Y. Koren, R. Bell, and C. Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. Computer 42, 8 (2009), 42--49.

Digital Library

[30]

Pigi Kouki, James Schaffer, Jay Pujara, John O'Donovan, and Lise Getoor. 2017. User Preferences for Hybrid Explanations. In Proceedings of the Eleventh ACM Conference on Recommender Systems - RecSys '17. 84--88.

Digital Library

[31]

Warren Kuhfeld. 2005. Experimental design, efficiency, coding, and choice designs. Marketing research methods in sas: Experimental design, choice, conjoint, and graphical techniques (2005), 47--97.

[32]

Jordan J Louviere, Terry N Flynn, and Richard T Carson. 2010. Discrete choice experiments are not conjoint analysis. Journal of Choice Modelling 3, 3 (2010), 57--72.

[33]

Raffaella Misuraca and Ursina Teuscher. 2013. Time flies when you maximize - Maximizers and satisficers perceive time differently when making decisions. Acta psychologica 143, 2 (2013), 176--180.

[34]

Gergana Y Nenkov, Maureen Morrin, Andrew Ward, Barry Schwartz, and John Hulland. 2008. A short form of the Maximization Scale: Factor structure, reliability and validity studies. Judgment and Decision Making 3, 5 (2008), 371--388.

[35]

Ingrid Nunes and Dietmar Jannach. 2017. A systematic review and taxonomy of explanations in decision support and recommender systems. User Modeling and User-Adapted Interaction 27, 3-5 (12 2017), 393--444.

Digital Library

[36]

John W Payne. 1976. Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance 16, 2 (1976), 366--387.

[37]

Fortunato Pesarin, Luigi Salmaso, and Wiley InterScience (Online service). 2010. Permutation tests for complex data : theory, applications, and software. Wiley. 417 pages.

[38]

Roland Pfister, Katharina A Schwarz, Markus Janczyk, Rick Dale, and Jon Freeman. 2013. Good things peak in pairs: a note on the bimodality coefficient. Frontiers in psychology 4 (2013), 700.

[39]

Vithala R Rao. 2008. Developments in conjoint analysis. In Handbook of marketing decision models. Springer, 23--53.

[40]

Jeff Sauro and James R. Lewis. 2010. Average task times in usability tests. In Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. ACM Press, New York, New York, USA, 2347.

Digital Library

[41]

Benjamin Scheibehenne, Rainer Greifeneder, and Peter M Todd. 2010. Can there ever be too many options? A meta-analytic review of choice overload. Journal of Consumer Research 37, 3 (2010), 409--425.

[42]

Barry Schwartz, Andrew Ward, Sonja Lyubomirsky, John Monterosso, Katherine White, and Darrin R Lehman. 2002. Maximizing versus satisficing: Happiness is a matter of choice. Journal of Personality and Social Psychology 83, 5 (2002), 1178--1197.

[43]

Herbert A Simon. 1955. A behavioral model of rational choice. The quarterly journal of economics 69, 1 (1955), 99--118.

[44]

Erin A Sparks, Joyce Ehrlinger, and Richard P Eibach. 2012. Failing to commit: Maximizers avoid commitment in a way that contributes to reduced satisfaction. Personality and Individual Differences 52, 1 (2012), 72--77.

[45]

Tiziano Tempesta, Rosa Arboretti Giancristofaro, Livio Corain, Luigi Salmaso, Diego Tomasi, and Vasco Boatto. 2010. The importance of landscape in wine quality perception: An integrated approach using choice-based conjoint analysis and combination-based permutation tests. Food Quality and Preference 21, 7 (2010), 827--836.

[46]

Erich Christian Teppan and Alexander Felfernig. 2012. Minimization of decoy effects in recommender result sets. Web Intelligence and Agent Systems: An International Journal 10, 4(2012), 385--395.

Digital Library

[47]

Erich Christian Teppan and Markus Zanker. 2015. Decision Biases in Recommender Systems. Journal of Internet Commerce 14, 2 (2015), 255--275.

[48]

Nava Tintarev and Judith Masthof. 2015. Explaining recommendations: design and evaluation. In Recommender Systems Handbook. Springer US, Boston, MA, 217--253.

[49]

Kyung-Hyan Yoo, Ulrike Gretzel, and Markus Zanker. 2012. Persuasive recommender systems: conceptual background and implications. Springer Science & Business Media.

Digital Library

[50]

Markus Zanker and Martin Schoberegger. 2014. An empirical study on the persuasiveness of fact-based explanations for recommender systems. In CEUR Workshop Proceedings, Vol. 1253. 33--36.

[51]

Klaus Zwerina, Joel Huber,and Wf Warren Kuhfeld. 1996. A general method for constructing efficient choice designs. Durham, NC: Fuqua School of Business, Duke Univesrsity September (1996), 39--59.

Cited By

Roubal O(2024)Consumer Culture and the Paradox of Choice in an Ambivalent Era of Emancipation of Individual FreedomsNavigating the Shifting Landscape of Consumer Behavior10.4018/979-8-3693-1594-1.ch003(51-85)Online publication date: 22-Mar-2024
https://doi.org/10.4018/979-8-3693-1594-1.ch003
Cavenaghi EZanga AStella FZanker M(2023)Towards a Causal Decision-Making Framework for Recommender SystemsACM Transactions on Recommender Systems10.1145/36291692:2(1-34)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3629169
Rook LDalen JKetter W(2023)Nudging the Direction of Energy Tariff Selection: Lessons Learned from an Attribute Framing Experiment with Temporal Construal LevelsEnergy Sustainability through Retail Electricity Markets10.1007/978-3-031-39707-3_5(75-96)Online publication date: 26-Jul-2023
https://doi.org/10.1007/978-3-031-39707-3_5
Show More Cited By

Index Terms

Decision making strategies differ in the presence of collaborative explanations: two conjoint studies
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Information systems
  1. Information systems applications

Recommendations

Explainable Matrix Factorization for Collaborative Filtering
WWW '16 Companion: Proceedings of the 25th International Conference Companion on World Wide Web

Explanations have been shown to increase the user's trust in recommendations in addition to providing other benefits such as scrutability, which is the ability to verify the validity of recommendations. Most explanation methods are designed for ...
LLM-generated Explanations for Recommender Systems
UMAP Adjunct '24: Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization

Users are often confronted with situations where they have to decide in favor or against an offered item, like a book, movie, or recipe. Those suggested items are commonly determined by a recommender system, which considers personal preferences to ...
Why-Not Questions & Explanations for Collaborative Filtering
Web Information Systems Engineering – WISE 2020
Abstract
Throughout our digital lives, we are getting recommendations for about almost everything we do, buy or consume. However, it is often the case that recommenders cannot locate the best data items to suggest. To deal with this shortcoming, they ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

March 2019

713 pages

ISBN:9781450362726

DOI:10.1145/3301275

General Chairs:
Wai-Tat Fu,
Shimei Pan,
Program Chairs:
Oliver Brdiczka,
Polo Chau,
Gaelle Calvary

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IUI '19

Sponsor:

IUI '19: 24th International Conference on Intelligent User Interfaces

March 17 - 20, 2019

California, Marina del Ray

Acceptance Rates

IUI '19 Paper Acceptance Rate 71 of 282 submissions, 25%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
451
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)4

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Roubal O(2024)Consumer Culture and the Paradox of Choice in an Ambivalent Era of Emancipation of Individual FreedomsNavigating the Shifting Landscape of Consumer Behavior10.4018/979-8-3693-1594-1.ch003(51-85)Online publication date: 22-Mar-2024
https://doi.org/10.4018/979-8-3693-1594-1.ch003
Cavenaghi EZanga AStella FZanker M(2023)Towards a Causal Decision-Making Framework for Recommender SystemsACM Transactions on Recommender Systems10.1145/36291692:2(1-34)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3629169
Rook LDalen JKetter W(2023)Nudging the Direction of Energy Tariff Selection: Lessons Learned from an Attribute Framing Experiment with Temporal Construal LevelsEnergy Sustainability through Retail Electricity Markets10.1007/978-3-031-39707-3_5(75-96)Online publication date: 26-Jul-2023
https://doi.org/10.1007/978-3-031-39707-3_5
Rook LBhattacharjee SKetter W(2023)Incorporating Social Values for Cooperation in Energy Trading and Balancing ResearchEnergy Sustainability through Retail Electricity Markets10.1007/978-3-031-39707-3_10(179-196)Online publication date: 26-Jul-2023
https://doi.org/10.1007/978-3-031-39707-3_10
Lyons HWijenayake SMiller TVelloso E(2022)What’s the Appeal? Perceptions of Review Processes for Algorithmic DecisionsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517606(1-15)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517606
Ricci F(2022)Recommender Systems in TourismHandbook of e-Tourism10.1007/978-3-030-48652-5_26(457-474)Online publication date: 2-Sep-2022
https://doi.org/10.1007/978-3-030-48652-5_26
McCarty CRedmond PPeel K(2021)Teacher decision-making in the classroom: the influence of cognitive load and teacher affectJournal of Education for Teaching10.1080/02607476.2021.1902748(1-14)Online publication date: 20-Mar-2021
https://doi.org/10.1080/02607476.2021.1902748
Naiseh MJiang NMa JAli R(2020)Personalising Explainable Recommendations: Literature and ConceptualisationTrends and Innovations in Information Systems and Technologies10.1007/978-3-030-45691-7_49(518-533)Online publication date: 8-Jun-2020
https://doi.org/10.1007/978-3-030-45691-7_49
Ricci F(2020)Recommender Systems in TourismHandbook of e-Tourism10.1007/978-3-030-05324-6_26-1(1-18)Online publication date: 31-Jan-2020
https://doi.org/10.1007/978-3-030-05324-6_26-1
Confalonieri RCoba LWagner BBesold T(2020)A historical perspective of explainable Artificial IntelligenceWIREs Data Mining and Knowledge Discovery10.1002/widm.139111:1Online publication date: 19-Oct-2020
https://doi.org/10.1002/widm.1391
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten