research-article

Why and why not explanations improve the intelligibility of context-aware intelligent systems

Authors:

Daniel AvrahamiAuthors Info & Claims

CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 2119 - 2128

https://doi.org/10.1145/1518701.1519023

Published: 04 April 2009 Publication History

Abstract

Context-aware intelligent systems employ implicit inputs, and make decisions based on complex rules and machine learning models that are rarely clear to users. Such lack of system intelligibility can lead to loss of user trust, satisfaction and acceptance of these systems. However, automatically providing explanations about a system's decision process can help mitigate this problem. In this paper we present results from a controlled study with over 200 participants in which the effectiveness of different types of explanations was examined. Participants were shown examples of a system's operation along with various automatically generated explanations, and then tested on their understanding of the system. We show, for example, that explanations describing why the system behaved a certain way resulted in better understanding and stronger feelings of trust. Explanations describing why the system did not behave a certain way, resulted in lower understanding yet adequate performance. We discuss implications for the use of our findings in real-world context-aware applications.

References

[1]

Amazon Mechanical Turk. http://www.mturk.com. Retrieved 16 Sep 2008.

[2]

Anderson, J. R., Corbett, A. T., Koedinger, K. R.,&Pelletier, R. (1995). Cognitive Tutors: Lessons Learned. Journal of the Learning Sciences, 4(2), 167--207.

[3]

Andrews, R., Diederich, J.&Tickle, A. (1995). A Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks. Knowledge Based Systems. 8: 373--389.

Digital Library

[4]

Assad, M., Carmichael, D.J., Kay, J.&Kummerfeld, B. (2007). PersonisAD: Distributed, Active, Scrutable Model Framework for Context-Aware Services. Pervasive 2007, 55--72.

Digital Library

[5]

Avrahami, D. and Hudson, S.E. (2006). Responsiveness in Instant Messaging: Predictive Models Supporting Inter-Personal Communication. CHI'06, 731--740.

Digital Library

[6]

Bellotti, V.&Edwards, W.K. (2001). Intelligibility and Accountability: Human Considerations in Context-Aware Systems, Human-Computer Interaction, 16(2-4): 193--212.

Digital Library

[7]

Bellotti, V., Back, M., Edwards, W.K., Grinter, R.E., Henderson, A.&Lopes, C. (2002). Making sense of sensing: Five questions for designers and researchers. CHI'02, 415--422.

Digital Library

[8]

Cheverst, K., Byun, H.E., Fitton, D., Sas, C., Kray, C. and Villar, N. (2005). Exploring issues of user model transparency and proactive behavior in an office environment control system. User Modeling and User-Adapted Interaction: Journal of Personalization Research, 15(3-4), 235--273.

Digital Library

[9]

Davis, R., Buchanan, B., and Shortliffe, E. (1977). Production rules as a representation for a knowledge-based consultation program. Artificial Intelligence, 8(1): 15--45.

Digital Library

[10]

Dey, A.K.&Newberger, A. (2009). Support for contxt-aware intelligibility and control. To appear in CHI'09.

Digital Library

[11]

Dourish, P., Adler, A.&Smith, B.C. (1996). Organising User Interfaces Around Reflective Accounts. Reflection'96, 235--244.

[12]

Dzindolet, M., Peterson, S., Pomranky, S. Pierce, L.&Beck, H. (2003). The role of trust in automation reliance, Int'l Journal of Human-Computer Studies, 58(6): 697--718.

Digital Library

[13]

Google Web Toolkit. http://code.google.com/webtoolkit/. Retrieved 16 Sep 2008.

[14]

Gregor, S.&Benbasat, I. (1999). Explanations From Intelligent Systems: Theoretical Foundations and Implications for Practice. MIS Quarterly 23(4): 497--530.

Digital Library

[15]

Herlocker, J., Konstan, J.&Riedl, J. (2000). Explaining collaborative filtering recommendations. CSCW 2000, 241--250.

Digital Library

[16]

Ko, A.J.&Myers, B.A. (2004). Designing the WhyLine: A debugging interface for asking questions about programm failures. CHI 2004, 151--158.

Digital Library

[17]

Mozina M., Demsar, J., Kattan, M.,&Zupan, B. (2004). Nomograms for Visualization of Naive Bayesian Classifier. PKDD 2004, 337--348.

Digital Library

[18]

McGuinness, D.L., Glass, A., Wolverton, M.&Pinheiro da Silva, P. (2007). A Categorization of Explanation Questions for Task Processing Systems. AAAI Workshop on Explanation-Aware Computing.

[19]

Muir, B. (1994). Trust in automation: Part i. theoretical issues in the study of trust and human intervention in automated systems. Ergonomics, 37(11): 1905--1922.

[20]

Myers, B.A., Weitzman, D., Ko, A.J.&Chau, D.H. (2006). Answering Why and Why Not Questions in User Interfaces. CHI 2006, 397--406.

Digital Library

[21]

Norman, Donald A. (1988). The Design of Everyday Things. New York, Doubleday.

[22]

Nugent, C.,&Cunningham, P. (2005). A case-based explanation system for black-box systems. Artificial Intelligence Review, 24(2): 163--178.

Digital Library

[23]

Schilit, B.N., Adams, N.I.&Want, R. (1994). Context-aware computing applications. 1st International Workshop on Mobile Computing Systems and Applications, 85--90.

Digital Library

[24]

Tullio, J., Dey, A.K., Fogarty, J.&Chalecki, J. (2007). How it works: A field study of non-technical users interacting with an intelligent system. CHI 2007, 31--40.

Digital Library

[25]

Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers.

Digital Library

[26]

Weiser, M. (1991). The computer for the 21st century. Scientific American, 265(3): 94--104.

[27]

Weiser, M.&Brown, J.S., (1995). Designing Calm Technology. PowerGrid Journal v1.01.

Cited By

Xuan YSmall ESokol KHettiachchi DSanderson M(2025)Comprehension is a double-edged sword: Over-interpreting unspecified information in intelligible machine learning explanationsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103376193(103376)Online publication date: Jan-2025
https://doi.org/10.1016/j.ijhcs.2024.103376
Chari SSeneviratne OGhalwash MShirai SGruen DMeyer PChakraborty PMcGuinness D(2024)Explanation Ontology: A general-purpose, semantic representation for supporting user-centered explanationsSemantic Web10.3233/SW-23328215:4(959-989)Online publication date: 4-Oct-2024
https://doi.org/10.3233/SW-233282
Szczuka JHorstmann ASzymczyk NStrathmann CArtelt AMavrina LKrämer N(2024)Let Me Explain What I Did or What I Would Have Done: An Empirical Study on the Effects of Explanations and Person-Likeness on Trust in and Understanding of AlgorithmsProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685351(1-13)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685351
Show More Cited By

Index Terms

Why and why not explanations improve the intelligibility of context-aware intelligent systems
1. Human-centered computing

Recommendations

Improving trust in context-aware applications with intelligibility
UbiComp '10 Adjunct: Proceedings of the 12th ACM international conference adjunct papers on Ubiquitous computing - Adjunct

Since context-aware applications use implicit sensing and increasingly complex decision making, they may make mistakes or users may misunderstand their actions. This may hinder trust and adoption of context-aware applications. We hypothesize that making ...
Toolkit to support intelligibility in context-aware applications
UbiComp '10: Proceedings of the 12th ACM international conference on Ubiquitous computing

Context-aware applications should be intelligible so users can better understand how they work and improve their trust in them. However, providing intelligibility is non-trivial and requires the developer to understand how to generate explanations from ...
The Impact of Placebic Explanations on Trust in Intelligent Systems
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

Work in social psychology on interpersonal interaction has demonstrated that people are more likely to comply to a request if they are presented with a justification - even if this justification conveys no information. In the light of the many calls for ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2009

2426 pages

ISBN:9781605582467

DOI:10.1145/1518701

General Chairs:
Dan R. Olsen
Brigham Young University
,
Richard B. Arthur
Brigham Young University
,
Program Chairs:
Ken Hinckley
Microsoft Research
,
Meredith Ringel Morris
Microsoft Research
,
Scott Hudson
Carnegie Mellon University
,
Saul Greenberg
University of Calgary

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 April 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '09

Sponsor:

CHI '09: CHI Conference on Human Factors in Computing Systems

April 4 - 9, 2009

MA, Boston, USA

Acceptance Rates

CHI '09 Paper Acceptance Rate 277 of 1,130 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

343
Total Citations
View Citations
3,493
Total Downloads

Downloads (Last 12 months)354
Downloads (Last 6 weeks)48

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xuan YSmall ESokol KHettiachchi DSanderson M(2025)Comprehension is a double-edged sword: Over-interpreting unspecified information in intelligible machine learning explanationsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103376193(103376)Online publication date: Jan-2025
https://doi.org/10.1016/j.ijhcs.2024.103376
Chari SSeneviratne OGhalwash MShirai SGruen DMeyer PChakraborty PMcGuinness D(2024)Explanation Ontology: A general-purpose, semantic representation for supporting user-centered explanationsSemantic Web10.3233/SW-23328215:4(959-989)Online publication date: 4-Oct-2024
https://doi.org/10.3233/SW-233282
Szczuka JHorstmann ASzymczyk NStrathmann CArtelt AMavrina LKrämer N(2024)Let Me Explain What I Did or What I Would Have Done: An Empirical Study on the Effects of Explanations and Person-Likeness on Trust in and Understanding of AlgorithmsProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685351(1-13)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685351
Chatti MGuesmi MMuslim A(2024)Visualization for Recommendation Explainability: A Survey and New PerspectivesACM Transactions on Interactive Intelligent Systems10.1145/367227614:3(1-40)Online publication date: 11-Jun-2024
https://dl.acm.org/doi/10.1145/3672276
Tran PLe HHuynh H(2024)Opinion Mining with Interpretable Random Density ForestsProceedings of the 2024 8th International Conference on Machine Learning and Soft Computing10.1145/3647750.3647761(66-72)Online publication date: 26-Jan-2024
https://dl.acm.org/doi/10.1145/3647750.3647761
Kianpisheh MMariakakis ATruong K(2024)exHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435008:1(1-30)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643500
Lee MMcMillan DTorre IFischer JRuitenburg Y(2024)Between Trust and Identity: Form, Function, and PresentationProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3669999(1-4)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3669999
Lee HJung YShin YPark HChoi WLee U(2024)FamilyScope: Visualizing Affective Aspects of Family Social Interactions using Passive Sensor DataProceedings of the ACM on Human-Computer Interaction10.1145/36373348:CSCW1(1-27)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637334
Bhattacharya AStumpf SVerbert KBoratto LGena CMarras MGermanakos PPopescus E(2024)Representation Debiasing of Generated Data Involving Domain ExpertsAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3664910(516-522)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3631700.3664910
Lee YSon KKim TKim JChung JAdar EKim J(2024)One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI GenerationsProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3662681(2518-2531)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3662681
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents