abstract

Experts in the Shadow of Algorithmic Systems: Exploring Intelligibility in a Decision-Making Context

Authors:

Rhianne JonesAuthors Info & Claims

DIS' 20 Companion: Companion Publication of the 2020 ACM Designing Interactive Systems Conference

Pages 263 - 268

https://doi.org/10.1145/3393914.3395862

Published: 06 July 2020 Publication History

Abstract

Algorithms support decision-making in various contexts, often diminishing human agency in the process. Without meaningful human input, use of predictive systems can result in costly errors, leaving users unable to evaluate accuracy. Intelligibility is one design criterion that may ensure users remain in the decision-making loop. However, guidance is currently diffuse and focused on the lay user, ignoring the role of expertise. We propose a cognitive psychology-based framework that segments decision-making space by users' expertise, risk-environment and motivation. We illustrate this by focusing on expertise, exploring how we might inform usable intelligibility in interface design, enhancing user agency in the decision-making process.

References

[1]

Abdul Ashraf, Jo Vermeulen, Danding Wang, Brian Y. Lim, and Mohan Kankanhalli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An hci research agenda. In Proceedings of the 2018 CHI conference on human factors in computing systems, 1--18.

[2]

Barocas Solon, and Andrew D. Selbst. 2016. Big data's disparate impact. Calif L Rev 104, 671.

[3]

Borst Clark, Vincent A. Bijsterbosch, M. M. Van Paassen, and Max Mulder. 2017. Ecological interface design: supporting fault diagnosis of automated advice in a supervisory air traffic control task. Cognition, Technology & Work 19, 4: 545--560.

Digital Library

[4]

Caruana Rich, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, 1721--1730.

[5]

Cohen S. Marvin, Jared T. Freeman, and Steve Wolf. 1996. Metarecognition in time-stressed decision making: Recognizing, critiquing, and correcting. Human Factors 38, 2: 206--219.

[6]

Dosilovic F. Karlo, Mario Brcic, and Nikica Hlupic. 2018. Explainable artificial intelligence: A survey. In 2018 41st International convention on information and communication technology, electronics and microelectronics (MIPRO), 02100215. IEEE.

[7]

Gilpin H. Leilani, David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2018. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA), 80--89.

[8]

Green Ben, and Yiling Chen. 2019. Disparate interactions: An algorithm-in-the-loop analysis of fairness in risk assessments. In Proceedings of the Conference on Fairness, Accountability, and Transparency, 90--99.

[9]

Hilburn Brian, Carl Westin, and Clark Borst. 2014. Will controllers accept a machine that thinks like they think? The role of strategic conformance in decision aiding automation. Air Traffic Control Quarterly 22, 2: 115--136.

[10]

Kahneman Daniel, and Gary Klein. 2009. Conditions for intuitive expertise: a failure to disagree. American psychologist 64, 6: 515.

[11]

Kehl D. Leah, and Samuel A. Kessler. 2017. Algorithms in the criminal justice system: Assessing the use of risk assessments in sentencing.

[12]

Kim S. Edward, Roy S. Herbst, Ignacio I. Wistuba, Jack Lee, George R. Blumenschein, Anne Tsao, David J. Stewart et al. 2011. The BATTLE trial: personalizing therapy for lung cancer. Cancer discovery 1, 1: 44--53.

[13]

Klein Gary, and Valerie M. Chase. 1998. Sources of power: how people make decisions. Nature 392, 6673: 242--242.

[14]

Klein Gary. 2003. Intuition at work: Why developing your gut instincts will make you better at what you do. Currency/Doubleday.

[15]

Klein Gary, Brian Moon, and Robert R. Hoffman. 2006. Making sense of sensemaking 2: A macrocognitive model. IEEE Intelligent systems 21, 5: 88--92.

[16]

Kleinberg Jon, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807

[17]

Leicht-Deobald Ulrich, Thorsten Busch, Christoph Schank, Antoinette Weibel, Simon Schafheitle, Isabelle Wildhaber, and Gabriel Kasper. 2019. The Challenges of Algorithm-Based HR DecisionMaking for Personal Integrity. Journal of Business Ethics 160, 2: 377--392.

[18]

Orasanu Judith, and Ute Fischer. 1997. Finding decisions in natural environments: The view from the cockpit. Naturalistic decision making: 343--357.

[19]

Payne W. John, James R. Bettman, and Eric J. Johnson. 1998. Adaptive strategy selection in decision making. Journal of experimental psychology: Learning, Memory, and Cognition 14, 3: 534.

[20]

Saunders Jessica, Priscillia Hunt, and John S. Hollywood. 2016. Predictions put into practice: a quasi-experimental evaluation of Chicago's predictive policing pilot. Journal of Experimental Criminology 12, 3: 347--371.

[21]

Veale Michael, Max Van Kleek, and Reuben Binns. 2018. Fairness and accountability design needs for algorithmic support in high-stakes public sector decision-making. In Proceedings of the 2018 chi conference on human factors in computing systems, 1--14.

[22]

Wagner Ben. 2019. Liable, but Not in Control? Ensuring Meaningful Human Agency in Automated Decision-Making Systems. Policy & Internet 11, 1: 104--122.

[23]

Westin Carl, Clark Borst, and Brian Hilburn. 2015. Strategic conformance: Overcoming acceptance issues of decision aiding automation? IEEE Transactions on Human-Machine Systems 46, 1: 41--52.

Cited By

Tankelevitch LKewenig VSimkute AScott ASarkar ASellen ARintel S(2024)The Metacognitive Demands and Opportunities of Generative AIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642902(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642902
Bertrand AEagan JMaxwell W(2023)Questioning the ability of feature-based explanations to empower non-experts in robo-advised financial decision-makingProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594053(943-958)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594053
Ma SLei YWang XZheng CShi CYin MMa X(2023)Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-MakingProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581058(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581058

Index Terms

Experts in the Shadow of Algorithmic Systems: Exploring Intelligibility in a Decision-Making Context
1. Human-centered computing

Recommendations

Comparing humans and automatic speech recognition systems in recognizing dysarthric speech
Canadian AI'11: Proceedings of the 24th Canadian conference on Advances in artificial intelligence

Speech is a complex process that requires control and coordination of articulation, breathing, voicing, and prosody. Dysarthria is a manifestation of an inability to control and coordinate one or more of these aspects, which results in poorly ...
The Lombard intelligibility benefit of native and non-native speech for native and non-native listeners
Highlights
- We compared native English and non-native (Dutch) Lombard and plain speech.
- ...
Abstract
Speech produced in noise (Lombard speech) is more intelligible than speech produced in quiet (plain speech). Previous research on the Lombard intelligibility benefit focused almost entirely on how native speakers produce and perceive ...
Searching for experts in a context-aware recommendation network

We apply expertise to improve the quality of recommendation systems.We consider how context-aware recommender systems can be used in learning context.Our approach save resources since it reduces the number of nodes to query. The huge amount of data in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS' 20 Companion: Companion Publication of the 2020 ACM Designing Interactive Systems Conference

July 2020

605 pages

ISBN:9781450379878

DOI:10.1145/3393914

General Chairs:
Ron Wakkary
Simon Fraser University, CA and Eindhoven University of Technology, NL
,
Kristina Andersen
Eindhoven University of Technology, NL
,
Program Chairs:
Will Odom
Simon Fraser University, CA
,
Audrey Desjardins
University of Washington, USA
,
Marianne Graves Petersen
Aarhus University, DK

Copyright © 2020 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 July 2020

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

Engineering and Physical Sciences Research Council

Conference

DIS '20

Sponsor:

SIGCHI

DIS '20: Designing Interactive Systems Conference 2020

July 6 - 10, 2020

Eindhoven, Netherlands

Acceptance Rates

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
203
Total Downloads

Downloads (Last 12 months)39
Downloads (Last 6 weeks)4

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tankelevitch LKewenig VSimkute AScott ASarkar ASellen ARintel S(2024)The Metacognitive Demands and Opportunities of Generative AIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642902(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642902
Bertrand AEagan JMaxwell W(2023)Questioning the ability of feature-based explanations to empower non-experts in robo-advised financial decision-makingProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594053(943-958)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594053
Ma SLei YWang XZheng CShi CYin MMa X(2023)Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-MakingProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581058(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581058

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents