research-article

Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making

Authors:

Xiaojuan MaAuthors Info & Claims

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 351, Pages 1 - 19

https://doi.org/10.1145/3544548.3581131

Published: 19 April 2023 Publication History

Abstract

Existing research on human-AI collaborative decision-making focuses mainly on the interaction between AI and individual decision-makers. There is a limited understanding of how AI may perform in group decision-making. This paper presents a wizard-of-oz study in which two participants and an AI form a committee to rank three English essays. One novelty of our study is that we adopt a speculative design by endowing AI equal power to humans in group decision-making. We enable the AI to discuss and vote equally with other human members. We find that although the voice of AI is considered valuable, AI still plays a secondary role in the group because it cannot fully follow the dynamics of the discussion and make progressive contributions. Moreover, the divergent opinions of our participants regarding an “equal AI” shed light on the possible future of human-AI relations.

Supplementary Material

Supplemental Materials (3544548.3581131-supplemental-materials.zip)

Download
149.19 KB

MP4 File (3544548.3581131-talk-video.mp4)

Pre-recorded Video Presentation

Download
212.60 MB

MP4 File (3544548.3581131-video-preview.mp4)

Video Preview

Download
6.80 MB

References

[1]

Ali Alkhatib. 2021. To live in their utopia: Why algorithmic systems create absurd outcomes. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–9.

Digital Library

[2]

Ali Alkhatib and Michael Bernstein. 2019. Street-level algorithms: A theory at the gaps between policy and decisions. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[3]

Daniel Balliet, Junhui Wu, and Carsten KW De Dreu. 2014. Ingroup favoritism in cooperation: a meta-analysis.Psychological bulletin 140, 6 (2014), 1556.

[4]

Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, and Daniel S Weld. 2021. Is the most accurate ai the best teammate? optimizing ai for teamwork. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 11405–11414.

[5]

Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16.

Digital Library

[6]

BBC. 2014. Algorithm appointed board director. Retrieved November 21, 2022 from https://www.bbc.com/news/technology-27426942

[7]

Bloomberg. 2016. Tieto the First Nordic Company to Appoint Artificial Intelligence to the Leadership Team of the New Data-Driven Businesses Unit. Retrieved November 21, 2022 from https://www.bloomberg.com/press-releases/2016-10-17/tieto-the-first-nordic-company-to-appoint-artificial-intelligence-to-the-leadership-team-of-the-new-data-driven-businesses-unit

[8]

Rod Bond. 2005. Group size and conformity. Group processes & intergroup relations 8, 4 (2005), 331–354.

[9]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.

[10]

Zana Buçinca, Phoebe Lin, Krzysztof Z Gajos, and Elena L Glassman. 2020. Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. In Proceedings of the 25th international conference on intelligent user interfaces. 454–464.

Digital Library

[11]

Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1(2021), 1–21.

Digital Library

[12]

Carrie J Cai, Emily Reif, Narayan Hegde, Jason Hipp, Been Kim, Daniel Smilkov, Martin Wattenberg, Fernanda Viegas, Greg S Corrado, Martin C Stumpe, 2019. Human-centered tools for coping with imperfect algorithms during medical decision-making. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–14.

Digital Library

[13]

Hao-Fei Cheng, Logan Stapleton, Anna Kawakami, Venkatesh Sivaraman, Yanghuidi Cheng, Diana Qing, Adam Perer, Kenneth Holstein, Zhiwei Steven Wu, and Haiyi Zhu. 2022. How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions. In CHI Conference on Human Factors in Computing Systems. 1–22.

[14]

Jared R Curhan, Jennifer R Overbeck, Yeri Cho, Teng Zhang, and Yu Yang. 2022. Silence is golden: Extended silence, deliberative mindset, and value creation in negotiation.Journal of applied psychology 107, 1 (2022), 78.

[15]

Gerardine DeSanctis and R Brent Gallupe. 1987. A foundation for the study of group decision support systems. Management science 33, 5 (1987), 589–609.

[16]

Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: people erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology: General 144, 1 (2015), 114.

[17]

Upol Ehsan, Q Vera Liao, Michael Muller, Mark O Riedl, and Justin D Weisz. 2021. Expanding explainability: Towards social transparency in ai systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–19.

Digital Library

[18]

Muzaffer Eroğlu and Meltem Karatepe Kaya. 2022. Impact of Artificial Intelligence on Corporate Board Diversity Policies and Regulations. European Business Organization Law Review(2022), 1–32.

[19]

Shelly Farnham, Harry R Chesley, Debbie E McGhee, Reena Kawal, and Jennifer Landau. 2000. Structured online interactions: improving the decision-making of small discussion groups. In Proceedings of the 2000 ACM conference on Computer supported cooperative work. 299–308.

Digital Library

[20]

Lori L. Foster and Michael D. Coovert. 2000. "Intelligent" Team Decision Making. In CHI ’00 Extended Abstracts on Human Factors in Computing Systems (The Hague, The Netherlands) (CHI EA ’00). Association for Computing Machinery, New York, NY, USA, 153–154. https://doi.org/10.1145/633292.633378

Digital Library

[21]

Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing. Ieee, 6645–6649.

[22]

Greg Guest, Kathleen M MacQueen, and Emily E Namey. 2011. Applied thematic analysis. sage publications.

[23]

Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A Survey of Methods for Explaining Black Box Models. ACM Comput. Surv. 51, 5, Article 93 (aug 2018), 42 pages. https://doi.org/10.1145/3236009

Digital Library

[24]

Torben Hagerup, Kurt Mehlhorn, and J. Ian Munro. 1993. Maintaining Discrete Probability Distributions Optimally. In Proceedings of the 20th International Colloquium on Automata, Languages and Programming(Lecture Notes in Computer Science, Vol. 700). Springer-Verlag, Berlin, 253–264.

Digital Library

[25]

Nichole Harvey and Colin A Holmes. 2012. Nominal group technique: an effective method for obtaining group consensus. International journal of nursing practice 18, 2 (2012), 188–194.

[26]

Nicholas Hertz and Eva Wiese. 2018. Under pressure: Examining social conformity with computer and robot groups. Human factors 60, 8 (2018), 1207–1218.

[27]

Michael Hilb. 2020. Toward artificial governance? The role of artificial intelligence in shaping the future of corporate governance. Journal of Management and Governance 24, 4 (2020), 851–870.

[28]

James L Hilton and William Von Hippel. 1996. Stereotypes. Annual review of psychology 47, 1 (1996), 237–271.

[29]

Randy Y Hirokawa and Marshall Scott Poole. 1996. Communication and group decision making. Sage Publications.

[30]

Jess Hohenstein, Lindsay E Larson, Yoyo Tsung-Yu Hou, Alexa M Harris, Aaron Schecter, Leslie Dechurch, Noshir Contractor, and Malte F Jung. 2022. Vero: A Method for Remotely Studying Human-AI Collaboration. Proceedings of the 55th Hawaii International Conference on System Sciences (2022).

[31]

Sungsoo Hong, Minhyang Suh, Nathalie Henry Riche, Jooyoung Lee, Juho Kim, and Mark Zachry. 2018. Collaborative dynamic queries: Supporting distributed small group decision-making. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[32]

Silas Hsu, Tiffany Wenting Li, Zhilin Zhang, Max Fowler, Craig Zilles, and Karrie Karahalios. 2021. Attitudes surrounding an imperfect AI autograder. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.

Digital Library

[33]

Floyd Hunter. 2017. Community power structure: A study of decision makers. UNC Press Books.

[34]

John M Ivancevich, Michael T Matteson, and Robert Konopaske. 1990. Organizational behavior and management. (1990).

[35]

Holtzblatt Karen and Jones Sandra. 2017. Contextual inquiry: A participatory technique for system design. In Participatory design. CRC Press, 177–210.

[36]

Anna Kawakami, Venkatesh Sivaraman, Hao-Fei Cheng, Logan Stapleton, Yanghuidi Cheng, Diana Qing, Adam Perer, Zhiwei Steven Wu, Haiyi Zhu, and Kenneth Holstein. 2022. Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support. In CHI Conference on Human Factors in Computing Systems. 1–18.

Digital Library

[37]

Pranav Khadpe, Chinmay Kulkarni, and Geoff Kaufman. 2022. Empathosphere: Promoting Constructive Communication in Ad-Hoc Virtual Teams through Perspective-Taking Spaces. Proc. ACM Hum.-Comput. Interact. 6, CSCW1, Article 55 (apr 2022), 26 pages. https://doi.org/10.1145/3512902

Digital Library

[38]

Saif Khairat, David Marc, William Crosby, Ali Al Sanousi, 2018. Reasons for physicians not adopting clinical decision support systems: critical analysis. JMIR medical informatics 6, 2 (2018), e8912.

[39]

Eugene C Kim. 2007. YouTube: Testing the safe harbors of digital copyright law. S. Cal. Interdisc. LJ 17(2007), 139.

[40]

Soomin Kim, Jinsu Eun, Changhoon Oh, Bongwon Suh, and Joonhwan Lee. 2020. Bot in the bunch: Facilitating group chat discussion by improving efficiency and participation with a chatbot. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[41]

Soomin Kim, Jinsu Eun, Joseph Seering, and Joonhwan Lee. 2021. Moderator Chatbot for Deliberative Discussion: Effects of Discussion Structure and Discussant Facilitation. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 87 (apr 2021), 26 pages. https://doi.org/10.1145/3449161

Digital Library

[42]

Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2018. Human decisions and machine predictions. The quarterly journal of economics 133, 1 (2018), 237–293.

[43]

Rahul Kumar, Sandeep Mathias, Sriparna Saha, and Pushpak Bhattacharyya. 2022. Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Seattle, United States, 1485–1495. https://doi.org/10.18653/v1/2022.naacl-main.106

[44]

Sandeep Kaur Kuttal, Bali Ong, Kate Kwasny, and Peter Robe. 2021. Trade-offs for substituting a human with an agent in a pair programming context: the good, the bad, and the ugly. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–20.

Digital Library

[45]

Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q Vera Liao, Yunfeng Zhang, and Chenhao Tan. 2022. Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation. In CHI Conference on Human Factors in Computing Systems. 1–18.

Digital Library

[46]

Vivian Lai, Chacha Chen, Q Vera Liao, Alison Smith-Renner, and Chenhao Tan. 2021. Towards a science of human-ai decision making: a survey of empirical studies. arXiv preprint arXiv:2112.11471(2021).

[47]

Vivian Lai, Han Liu, and Chenhao Tan. 2020. " Why is’ Chicago’deceptive?" Towards Building Model-Driven Tutorials for Humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[48]

Vivian Lai and Chenhao Tan. 2019. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the conference on fairness, accountability, and transparency. 29–38.

Digital Library

[49]

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems 30 (2017).

[50]

Markus Langer, Tim Hunsicker, Tina Feldkamp, Cornelius J König, and Nina Grgić-Hlača. 2022. “Look! It’sa Computer Program! It’s an Algorithm! It’s AI!”: Does Terminology Affect Human Perceptions and Evaluations of Algorithmic Decision-Making Systems?. In CHI Conference on Human Factors in Computing Systems. 1–28.

Digital Library

[51]

Ariel Levy, Monica Agrawal, Arvind Satyanarayan, and David Sontag. 2021. Assessing the impact of automated suggestions on decision making: Domain experts mediate model errors but take less initiative. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[52]

Jinyu Li 2022. Recent advances in end-to-end automatic speech recognition. APSIPA Transactions on Signal and Information Processing 11, 1(2022).

[53]

Q Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: informing design practices for explainable AI user experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–15.

Digital Library

[54]

Q Vera Liao and Kush R Varshney. 2021. Human-centered explainable ai (xai): From algorithms to user experiences. arXiv preprint arXiv:2110.10790(2021).

[55]

Robert C Liden, Sandy J Wayne, Renata A Jaworski, and Nathan Bennett. 2004. Social loafing: A field investigation. Journal of management 30, 2 (2004), 285–304.

[56]

Harold A Linstone, Murray Turoff, 1975. The delphi method. Addison-Wesley Reading, MA.

[57]

Duri Long and Brian Magerko. 2020. What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–16.

Digital Library

[58]

Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 4765–4774. http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf

[59]

Scott M Lundberg, Bala Nair, Monica S Vavilala, Mayumi Horibe, Michael J Eisses, Trevor Adams, David E Liston, Daniel King-Wai Low, Shu-Fang Newman, Jerry Kim, 2018. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nature biomedical engineering 2, 10 (2018), 749–760.

[60]

Shuai Ma, Mingfei Sun, and Xiaojuan Ma. 2022. Modeling Adaptive Expression of Robot Learning Engagement and Exploring its Effects on Human Teachers. ACM Transactions on Computer-Human Interaction (2022).

[61]

Shuai Ma, Zijun Wei, Feng Tian, Xiangmin Fan, Jianming Zhang, Xiaohui Shen, Zhe Lin, Jin Huang, Radomír Měch, Dimitris Samaras, 2019. SmartEye: assisting instant photo taking via integrating user preference with deep view proposal network. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–12.

Digital Library

[62]

Shuai Ma, Taichang Zhou, Fei Nie, and Xiaojuan Ma. 2022. Glancee: An adaptable system for instructors to grasp student learning status in synchronous online classes. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–25.

Digital Library

[63]

Karthik Mahadevan, Mauricio Sousa, Anthony Tang, and Tovi Grossman. 2021. “Grip-That-There”: An Investigation of Explicit and Implicit Task Allocation Techniques for Human-Robot Collaboration. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 215, 14 pages. https://doi.org/10.1145/3411764.3445355

Digital Library

[64]

Yaoli Mao, Dakuo Wang, Michael Muller, Kush R Varshney, Ioana Baldini, Casey Dugan, and Aleksandra Mojsilović. 2019. How data scientistswork together with domain experts in scientific collaborations: To find the right answer or to ask the right question?Proceedings of the ACM on Human-Computer Interaction 3, GROUP(2019), 1–23.

[65]

Andrew Monk. 2003. Common ground in electronically mediated communication: Clark’s theory of language use. HCI models, theories, and frameworks: Toward a multidisciplinary science (2003), 265–289.

[66]

Yi Mou and Kun Xu. 2017. The media inequality: Comparing the initial human-human and human-AI social interactions. Computers in Human Behavior 72 (2017), 432–440.

Digital Library

[67]

Dena F Mujtaba and Nihar R Mahapatra. 2019. Ethical considerations in AI-based recruitment. In 2019 IEEE International Symposium on Technology and Society (ISTAS). IEEE, 1–7.

[68]

Michael J Muller and Sarah Kuhn. 1993. Participatory design. Commun. ACM 36, 6 (1993), 24–28.

Digital Library

[69]

Mahsan Nourani, Joanie King, and Eric Ragan. 2020. The role of domain expertise in user trust and the impact of first impressions with intelligent systems. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 8. 112–121.

[70]

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. 2019. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems 32 (2019).

[71]

Zhenhui Peng, Yunhwan Kwon, Jiaan Lu, Ziming Wu, and Xiaojuan Ma. 2019. Design and Evaluation of Service Robot’s Proactivity in Decision-Making Support Process. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300328

Digital Library

[72]

Martin Petrin. 2019. Corporate Management in the Age of AI. Colum. Bus. L. Rev. (2019), 965.

[73]

James Pierce. 2021. In tension with progression: Grasping the frictional tendencies of speculative, critical, and other alternative designs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–19.

Digital Library

[74]

Margaret Potter, Sandy Gordon, and Peter Hamer. 2004. The nominal group technique: a useful consensus methodology in physiotherapy research. NZ Journal of Physiotherapy 32, 3 (2004), 126–130.

[75]

Laurel D Riek. 2012. Wizard of oz studies in hri: a systematic review and new reporting guidelines. Journal of Human-Robot Interaction 1, 1 (2012), 119–136.

Digital Library

[76]

Peter Robe and Sandeep Kaur Kuttal. 2022. Designing PairBuddy—A Conversational Agent for Pair Programming. ACM Transactions on Computer-Human Interaction (TOCHI) 29, 4(2022), 1–44.

Digital Library

[77]

Paul Robinette, Wenchen Li, Robert Allen, Ayanna M. Howard, and Alan R. Wagner. 2016. Overtrust of Robots in Emergency Evacuation Scenarios. In The Eleventh ACM/IEEE International Conference on Human Robot Interaction (Christchurch, New Zealand) (HRI ’16). IEEE Press, 101–108.

[78]

Thomas L Saaty and Kirti Peniwati. 2013. Group decision making: drawing out and reconciling differences. RWS publications.

[79]

Eduardo Salas, Rebecca Grossman, Ashley M Hughes, and Chris W Coultas. 2015. Measuring team cohesion: Observations from the science. Human factors 57, 3 (2015), 365–374.

[80]

William Samuelson and Richard Zeckhauser. 1988. Status quo bias in decision making. Journal of risk and uncertainty 1, 1 (1988), 7–59.

[81]

Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108(2019).

[82]

Beau G Schelble, Christopher Flathmann, Nathan J McNeese, Guo Freeman, and Rohit Mallick. 2022. Let’s Think Together! Assessing Shared Mental Models, Performance, and Trust in Human-Agent Teams. Proceedings of the ACM on Human-Computer Interaction 6, GROUP(2022), 1–29.

Digital Library

[83]

Ameneh Shamekhi, Q Vera Liao, Dakuo Wang, Rachel KE Bellamy, and Thomas Erickson. 2018. Face value? Exploring the effects of embodiment for a group facilitation agent. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–13.

Digital Library

[84]

Chuhan Shi, Fei Nie, Yicheng Hu, Yige Xu, Lei Chen, Xiaojuan Ma, and Qiong Luo. 2022. MedChemLens: An Interactive Visual Tool to Support Direction Selection in Interdisciplinary Experimental Research of Medicinal Chemistry. IEEE Transactions on Visualization and Computer Graphics 29, 1(2022), 63–73.

[85]

Ben Shneiderman. 2022. Human-Centered AI. Oxford University Press.

[86]

Kaveh Taghipour and Hwee Tou Ng. 2016. A neural approach to automated essay scoring. In Proceedings of the 2016 conference on empirical methods in natural language processing. 1882–1891.

[87]

Anna Trunk, Hendrik Birkel, and Evi Hartmann. 2020. On the current state of combining human and artificial intelligence for strategic organizational decision making. Business Research 13, 3 (2020), 875–919.

[88]

Patrick Van Esch, J Stewart Black, and Joseph Ferolie. 2019. Marketing AI recruitment: The next phase in job application and selection. Computers in Human Behavior 90 (2019), 215–222.

[89]

April Yi Wang, Dakuo Wang, Jaimie Drozdal, Michael Muller, Soya Park, Justin D Weisz, Xuye Liu, Lingfei Wu, and Casey Dugan. 2022. Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks. ACM Transactions on Computer-Human Interaction 29, 2(2022), 1–33.

Digital Library

[90]

Dakuo Wang, Josh Andres, Justin D Weisz, Erick Oduor, and Casey Dugan. 2021. Autods: Towards human-centered automation of data science. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[91]

Dakuo Wang, Liuping Wang, Zhan Zhang, Ding Wang, Haiyi Zhu, Yvonne Gao, Xiangmin Fan, and Feng Tian. 2021. “Brilliant AI Doctor” in Rural Clinics: Challenges in AI-Powered Clinical Decision Support System Deployment. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–18.

Digital Library

[92]

Ruotong Wang, F Maxwell Harper, and Haiyi Zhu. 2020. Factors influencing perceived fairness in algorithmic decision-making: Algorithm outcomes, development procedures, and individual differences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[93]

Rina R Wehbe, Edward Lank, and Lennart E Nacke. 2017. Left them 4 dead: Perception of humans versus non-player character teammates in cooperative gameplay. In Proceedings of the 2017 Conference on Designing Interactive Systems. 403–415.

Digital Library

[94]

Qian Yang, Aaron Steinfeld, and John Zimmerman. 2019. Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290605.3300468

Digital Library

[95]

Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295–305.

Digital Library

[96]

Ran Zhao, Alexandros Papangelis, and Justine Cassell. 2014. Towards a dyadic computational model of rapport management for human-virtual agent interaction. In International conference on intelligent virtual agents. Springer, 514–527.

[97]

Chengbo Zheng, Dakuo Wang, April Yi Wang, and Xiaojuan Ma. 2022. Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work. In CHI Conference on Human Factors in Computing Systems. 1–20.

Digital Library

Cited By

Chiang CLu ZLi ZYin M(2024)Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil's AdvocateProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645199(103-119)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645199
Salimzadeh SGadiraju U(2024)When in Doubt! Understanding the Role of Task Characteristics on Peer Decision-Making with AI AssistanceProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659567(89-101)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3627043.3659567
Gao JGebreegziabher SChoo KLi TPerrault SMalone T(2024)A Taxonomy for Human-LLM Interaction Modes: An Initial ExplorationExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650786(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650786
Show More Cited By

Index Terms

Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
  2. Interaction design
    1. Interaction design process and methods
      1. Scenario-based design

Recommendations

Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

While artificial intelligence (AI) is increasingly applied for decision-making processes, ethical decisions pose challenges for AI applications. Given that humans cannot always agree on the right thing to do, how would ethical decision-making by AI ...
The Impact of Imperfect XAI on Human-AI Decision-Making
CSCW

Explainability techniques are rapidly being developed to improve human-AI decision-making across various cooperative work settings. Consequently, previous research has evaluated how decision-makers collaborate with imperfect AI by investigating ...
Intelligent Decision Making: An AI-Based Approach

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

April 2023

14911 pages

ISBN:9781450394215

DOI:10.1145/3544548

Editors:
Albrecht Schmidt
LMU Munich, Germany60028717
,
Kaisa Väänänen
Tampere University, Finland60011170
,
Tesh Goyal
Google Research, USA60006191
,
Per Ola Kristensson
University of Cambridge, UK60031101
,
Anicia Peters
University of Namibia, Namibia60072704
,
Stefanie Mueller
Massachusetts Institute of Technology, USA60022195
,
Julie R. Williamson
University of Glasgow, UK60001490
,
Max L. Wilson
University of Nottingham, UK60015138

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI '23

Sponsor:

SIGCHI

CHI '23: CHI Conference on Human Factors in Computing Systems

April 23 - 28, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
1,136
Total Downloads

Downloads (Last 12 months)772
Downloads (Last 6 weeks)125

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chiang CLu ZLi ZYin M(2024)Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil's AdvocateProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645199(103-119)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645199
Salimzadeh SGadiraju U(2024)When in Doubt! Understanding the Role of Task Characteristics on Peer Decision-Making with AI AssistanceProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659567(89-101)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3627043.3659567
Gao JGebreegziabher SChoo KLi TPerrault SMalone T(2024)A Taxonomy for Human-LLM Interaction Modes: An Initial ExplorationExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650786(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650786
Zheng CYuan KGuo BHadi Mogavi RPeng ZMa SMa X(2024)Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with StudentsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642807(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642807
Wu GQian JCastelo Quispe SChen SRulff JSilva C(2024)ARTiST: Automated Text Simplification for Task Guidance in Augmented RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642772(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642772
Ma SWang XLei YShi CYin MMa X(2024)“Are You Really Sure?” Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision MakingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642671(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642671
Salimzadeh SHe GGadiraju U(2024)Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision MakingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641905(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641905
Anuyah OWan RAdejoro CYeh TMetoyer RBadillo-Urquiola K(2023)Cultural Considerations in AI Systems for the Global South: A Systematic ReviewProceedings of the 4th African Human Computer Interaction Conference10.1145/3628096.3629046(125-134)Online publication date: 27-Nov-2023
https://dl.acm.org/doi/10.1145/3628096.3629046

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Table of Contents