research-article

Open access

Investigating AI Teammate Communication Strategies and Their Impact in Human-AI Teams for Effective Teamwork

Authors:

Christopher Flathmann,

Nathan McNeese,

Alyssa WilliamsAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 7, Issue CSCW2

Article No.: 281, Pages 1 - 31

https://doi.org/10.1145/3610072

Published: 04 October 2023 Publication History

Abstract

Recently, AI is integrating into teams to collaborate with humans as a teammate with the goal of achieving unprecedented team outcomes. Much of the coordination between humans and AI teammates relies on human-AI communication, which is challenging due to AI's limitations on natural language communication. Thus, it is essential to identify and develop effective communication strategies for AI teammates in human-AI teams to facilitate the coordination process. Through interviews with 60 participants who collaborated with an AI teammate in a multiplayer online game, in this paper, we explore communication strategies that humans expect AI teammates to apply to support human-AI coordination and collaboration in dyadic teaming environments, and how the AI teammate's communication can impact teaming processes. Our findings highlight four communication strategies AI teammates should apply to support their coordination with humans in dyadic teaming environments. We also find that AI teammates' proactive communication with humans could facilitate the development of human trust and situation awareness, whereas AI lacking such proactive communication is often not perceived as a teammate. Our study extends the current CSCW/HCI research on human-AI communication in teaming environments by shedding light on how communication should be structured in dyadic human-AI teams for effective and smooth collaboration.

References

[1]

Jennie A. Abrahamson and Victoria L. Rubin. 2012. Discourse structure differences in lay and professional health communication. Journal of Documentation, Vol. 68, 6 (Jan. 2012), 826--851. https://doi.org/10.1108/00220411211277064 Publisher: Emerald Group Publishing Limited.

[2]

Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems. 1--13.

Digital Library

[3]

Zahra Ashktorab, Casey Dugan, James Johnson, Qian Pan, Wei Zhang, Sadhana Kumaravel, and Murray Campbell. 2021. Effects of communication directionality and AI agent differences in human-AI interaction. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--15.

Digital Library

[4]

Anthony L Baker, Sean M Fitzhugh, Lixiao Huang, Daniel E Forster, Angelique Scharine, Catherine Neubauer, Glenn Lematta, Shawaiz Bhatti, Craig J Johnson, Andrea Krausman, et al. 2021. Approaches for assessing communication in human-autonomy teams. Human-Intelligent Systems Integration, Vol. 3, 2 (2021), 99--128.

[5]

Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S Lasecki, Daniel S Weld, and Eric Horvitz. 2019. Beyond accuracy: The role of mental models in human-AI team performance. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 2--11.

[6]

Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--16.

Digital Library

[7]

Carrie J Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. " Hello AI": uncovering the onboarding needs of medical practitioners for human-AI collaborative decision-making. Proceedings of the ACM on Human-computer Interaction, Vol. 3, CSCW (2019), 1--24.

Digital Library

[8]

Sabrina Caldwell, Penny Sweetser, Nicholas O'Donnell, Matthew J. Knight, Matthew Aitchison, Tom Gedeon, Daniel Johnson, Margot Brereton, Marcus Gallagher, and David Conroy. 2022. An Agile New Research Framework for Hybrid Human-AI Teaming: Trust, Transparency, and Transferability. ACM Trans. Interact. Intell. Syst. (jan 2022). https://doi.org/10.1145/3514257 Just Accepted.

Digital Library

[9]

Lorenzo Barberis Canonico, Christopher Flathmann, and Nathan McNeese. 2019. Collectively intelligent teams: Integrating team cognition, collective intelligence, and ai for future teaming. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 63. SAGE Publications Sage CA: Los Angeles, CA, 1466--1470.

[10]

Kathy Charmaz. 2014. Constructing grounded theory. sage.

[11]

Yuhang Che, Heather Culbertson, Chih-Wei Tang, Sudipto Aich, and Allison M Okamura. 2018. Facilitating human-mobile robot communication via haptic feedback and gesture teleoperation. ACM Transactions on Human-Robot Interaction (THRI), Vol. 7, 3 (2018), 1--23.

Digital Library

[12]

Erin K Chiou, Mustafa Demir, Verica Buchanan, Christopher C Corral, Mica R Endsley, Glenn J Lematta, Nancy J Cooke, and Nathan J McNeese. 2022. Towards human--robot teaming: tradeoffs of explanation-based communication strategies in a virtual search and rescue task. International Journal of Social Robotics, Vol. 14, 5 (2022), 1117--1136.

[13]

Gregorio Convertino, Helena M. Mentis, Mary Beth Rosson, Aleksandra Slavkovic, and John M. Carroll. 2009. Supporting Content and Process Common Ground in Computer-Supported Teamwork. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI '09). Association for Computing Machinery, New York, NY, USA, 2339--2348. https://doi.org/10.1145/1518701.1519059

Digital Library

[14]

Nancy J Cooke, Mustafa Demir, and Nathan McNeese. 2016. Synthetic teammates as team players: Coordination of human and synthetic teammates. Technical Report. Cognitive Engineering Research Institute Mesa United States.

[15]

Nancy J Cooke, Jamie C Gorman, Christopher W Myers, and Jasmine L Duran. 2013. Interactive team cognition. Cognitive science, Vol. 37, 2 (2013), 255--285.

[16]

Nancy J Cooke, Preston A Kiekel, Eduardo Salas, Renée Stout, Clint Bowers, and Janis Cannon-Bowers. 2003. Measuring team knowledge: A window to the cognitive underpinnings of team performance. Group Dynamics: Theory, Research, and Practice, Vol. 7, 3 (2003), 179.

[17]

Ana Cristina Costa, C Ashley Fulmer, and Neil R Anderson. 2018. Trust in work teams: An integrative review, multilevel model, and future directions. Journal of Organizational Behavior, Vol. 39, 2 (2018), 169--184.

[18]

Nils Dahlb"ack, Arne Jönsson, and Lars Ahrenberg. 1993. Wizard of Oz studies-why and how. Knowledge-based systems, Vol. 6, 4 (1993), 258--266.

[19]

Ewart J De Visser, Richard Pak, and Tyler H Shaw. 2018. From "automation'to ?autonomy': the importance of trust repair in human--machine interaction. Ergonomics, Vol. 61, 10 (2018), 1409--1427.

[20]

Mustafa Demir, Aaron D Likens, Nancy J Cooke, Polemnia G Amazeen, and Nathan J McNeese. 2018. Team coordination and effectiveness in human-autonomy teaming. IEEE Transactions on Human-Machine Systems, Vol. 49, 2 (2018), 150--159.

[21]

Mustafa Demir, Nathan J McNeese, and Nancy J Cooke. 2016. Team communication behaviors of the human-automation teaming. In 2016 IEEE international multi-disciplinary conference on cognitive methods in situation awareness and decision support (CogSIMA). IEEE, 28--34.

[22]

Mustafa Demir, Nathan J McNeese, and Nancy J Cooke. 2017. Team situation awareness within the context of human-autonomy teaming. Cognitive Systems Research, Vol. 46 (2017), 3--12.

Digital Library

[23]

Mustafa Demir, Nathan J McNeese, and Nancy J Cooke. 2020. Understanding human-robot teams in light of all-human teams: Aspects of team interaction and shared cognition. International Journal of Human-Computer Studies, Vol. 140 (2020), 102436.

[24]

Wen Duan, Nathan J McNeese, and Rui Zhang. 2023. Communication in Human-AI Teaming. In Group Communication: An Advanced Introduction, Torsten Reimer, Ernest Park, and Joseph Bonito (Eds.). Routledge/Taylor and Francis.

[25]

Wen Duan, Naomi Yamashita, and Susan R Fussell. 2019. Increasing native speakers' awareness of the need to slow down in multilingual conversations using a real-time speech speedometer. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--25.

Digital Library

[26]

Wen Duan, Naomi Yamashita, Yoshinari Shirai, and Susan R Fussell. 2021. Bridging Fluency Disparity between Native and Nonnative Speakers in Multilingual Multiparty Collaboration Using a Clarification Agent. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW2 (2021), 1--31.

Digital Library

[27]

Mica R Endsley. 1995. Measurement of situation awareness in dynamic systems. Human factors, Vol. 37, 1 (1995), 65--84.

[28]

Eileen B Entin and Elliot E Entin. 2000. Assessing team situation awareness in simulated military missions. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 44. SAGE Publications Sage CA: Los Angeles, CA, 73--76.

[29]

Alexander Erlei, Richeek Das, Lukas Meub, Avishek Anand, and Ujwal Gadiraju. 2022. For What It's Worth: Humans Overwrite Their Economic Self-interest to Avoid Bargaining With AI Systems. In CHI Conference on Human Factors in Computing Systems. 1--18.

Digital Library

[30]

Alexander Erlei, Franck Nekdem, Lukas Meub, Avishek Anand, and Ujwal Gadiraju. 2020. Impact of algorithmic decision making on human behavior: Evidence from ultimatum bargaining. In Proceedings of the AAAI conference on human computation and crowdsourcing, Vol. 8. 43--52.

[31]

Stephen M Fiore, Michael A Rosen, Kimberly A Smith-Jentsch, Eduardo Salas, Michael Letsky, and Norman Warner. 2010. Toward an understanding of macrocognition in teams: Predicting processes in complex collaborative contexts. Human Factors, Vol. 52, 2 (2010), 203--224.

[32]

Thomas Franke, Christiane Attig, and Daniel Wessel. 2019. A personal resource for technology interaction: development and validation of the affinity for technology interaction (ATI) scale. International Journal of Human-Computer Interaction, Vol. 35, 6 (2019), 456--467.

[33]

Carolyn Fu, Kritika Dhanda, Marc Exposito Gomez, Haeyoung Kim, and Yan Zhang. 2017. TurnTable: Towards more equivalent turn-taking. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction. 609--615.

Digital Library

[34]

Susan R. Fussell, Robert E. Kraut, F. Javier Lerch, William L. Scherlis, Matthew M. McNally, and Jonathan J. Cadiz. 1998. Coordination, Overload and Team Performance: Effects of Team Communication Strategies. In Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work (Seattle, Washington, USA) (CSCW '98). Association for Computing Machinery, New York, NY, USA, 275--284. https://doi.org/10.1145/289444.289502

Digital Library

[35]

Susan R Fussell, Robert E Kraut, and Jane Siegel. 2000. Coordination of communication: Effects of shared visual context on collaborative work. In Proceedings of the 2000 ACM conference on Computer supported cooperative work. 21--30.

Digital Library

[36]

Ge Gao, Naomi Yamashita, Ari MJ Hautasaari, and Susan R Fussell. 2015. Improving multilingual collaboration by displaying how non-native speakers use automated transcripts and bilingual dictionaries. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 3463--3472.

Digital Library

[37]

Cristina B Gibson and Jennifer A Manuel. 2003. Building trust. Virtual teams that work (2003), 59--86.

[38]

Luis R Gomez-Mejia, David B Balkin, Robert L Cardy, and Kenneth P Carson. 2007. Managing human resources. Pearson/Prentice Hall Upper Saddle River, NJ.

[39]

Amy L Gonzales, Jeffrey T Hancock, and James W Pennebaker. 2010. Language style matching as a predictor of social dynamics in small groups. Communication Research, Vol. 37, 1 (2010), 3--19.

[40]

Camilla Göras, Karolina Olin, Maria Unbeck, Karin Pukk-H"arenstam, Anna Ehrenberg, Mesfin Kassaye Tessma, Ulrica Nilsson, and Mirjam Ekstedt. 2019. Tasks, multitasking and interruptions among the surgical team in an operating room: a prospective observational study. BMJ open, Vol. 9, 5 (2019), e026410.

[41]

H. P. Grice. 1975. Logic and Conversation. Brill. https://doi.org/10.1163/9789004368811_003 Pages: 41--58 Section: Speech Acts.

[42]

David A Grimm, Mustafa Demir, Jamie C Gorman, Nancy J Cooke, and Nathan J McNeese. 2019. Layered dynamics and system effectiveness of human-autonomy teams under degraded conditions. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 63. SAGE Publications Sage CA: Los Angeles, CA, 165--165.

[43]

Jonathan Grudin. 2017. From tool to partner: The evolution of human-computer interaction. Synthesis Lectures on Human-Centered Interaction, Vol. 10, 1 (2017), i--183.

[44]

Andrea L Guzman and Seth C Lewis. 2020. Artificial intelligence and communication: A Human--Machine Communication research agenda. New Media & Society, Vol. 22, 1 (2020), 70--86.

[45]

Thomas F. Hack, Lesley F. Degner, and Patricia A. Parker. 2005. The communication goals and needs of cancer patients: a review. Psycho-Oncology, Vol. 14, 10 (2005), 831--845. https://doi.org/10.1002/pon.949 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/pon.949.

[46]

Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139--183.

[47]

Allyson I Hauptman, Wen Duan, and Nathan J Mcneese. 2022. The Components of Trust for Collaborating With AI Colleagues. In Companion Publication of the 2022 Conference on Computer Supported Cooperative Work and Social Computing. 72--75.

[48]

Daniel R. Ilgen, John R. Hollenbeck, Michael Johnson, and Dustin Jundt. 2005. Teams in Organizations: From Input-Process-Output Models to IMOI Models. Annual Review of Psychology, Vol. 56, 1 (2005), 517--543. https://doi.org/10.1146/annurev.psych.56.091103.070250 15709945.

[49]

Alon Jacovi, Ana Marasović, Tim Miller, and Yoav Goldberg. 2021. Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in ai. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 624--635.

Digital Library

[50]

Sirkka L Jarvenpaa, Kathleen Knoll, and Dorothy E Leidner. 1998. Is anybody out there? Antecedents of trust in global virtual teams. Journal of management information systems, Vol. 14, 4 (1998), 29--64.

Digital Library

[51]

Sirkka L Jarvenpaa and Dorothy E Leidner. 1999. Communication and trust in global virtual teams. Organization science, Vol. 10, 6 (1999), 791--815.

Digital Library

[52]

J. David Johnson, William A. Donohue, Charles K. Atkin, and Sally Johnson. 1994. Differences Between Formal and Informal Communication Channels. The Journal of Business Communication (1973), Vol. 31, 2 (April 1994), 111--122. https://doi.org/10.1177/002194369403100202 Publisher: SAGE Publications.

[53]

Ji-Youn Jung, Sihang Qiu, Alessandro Bozzon, and Ujwal Gadiraju. 2022. Great Chain of Agents: The Role of Metaphorical Representation of Agents in Conversational Crowdsourcing. In CHI Conference on Human Factors in Computing Systems. 1--22.

[54]

Pranav Khadpe, Ranjay Krishna, Li Fei-Fei, Jeffrey T Hancock, and Michael S Bernstein. 2020. Conceptual metaphors impact perceptions of human-ai collaboration. Proceedings of the ACM on Human-Computer Interaction, Vol. 4, CSCW2 (2020), 1--26.

Digital Library

[55]

Alaa Khamis, Ahmed Hussein, and Ahmed Elmogy. 2015. Multi-robot task allocation: A review of the state-of-the-art. Cooperative robots and sensor networks 2015 (2015), 31--51.

[56]

Rafal Kocielnik, Saleema Amershi, and Paul N Bennett. 2019. Will you accept an imperfect ai? exploring designs for adjusting end-user expectations of ai systems. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--14.

Digital Library

[57]

Steve WJ Kozlowski. 2018. Enhancing the effectiveness of work groups and teams: A reflection. Perspectives on Psychological Science, Vol. 13, 2 (2018), 205--212.

[58]

Steve WJ Kozlowski and Daniel R Ilgen. 2006. Enhancing the effectiveness of work groups and teams. Psychological science in the public interest, Vol. 7, 3 (2006), 77--124.

[59]

Kurt Kraiger and Lucy H Wenzel. 1997. Conceptual development and empirical evaluation of measures of shared mental models as indicators of team effectiveness. In Team performance assessment and measurement. Psychology Press, 75--96.

[60]

Diana S Kusunoki and Aleksandra Sarcevic. 2015. Designing for temporal awareness: The role of temporality in time-critical medical teamwork. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. 1465--1476.

Digital Library

[61]

Lindsay Larson and Leslie A DeChurch. 2020. Leading teams in the digital age: Four perspectives on technology and what they mean for leading teams. The Leadership Quarterly, Vol. 31, 1 (2020), 101377.

[62]

John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors, Vol. 46, 1 (2004), 50--80.

[63]

Soyoung Lee, Charlotte Tang, Sun Young Park, and Yunan Chen. 2012. Loosely formed patient care teams: communication challenges and technology design. In Proceedings of the ACM 2012 conference on computer supported cooperative work. 867--876.

Digital Library

[64]

Gilly Leshed, Jeffrey T Hancock, Dan Cosley, Poppy L McLeod, and Geri Gay. 2007. Feedback for guiding reflection on teamwork practices. In Proceedings of the 2007 international ACM conference on Supporting group work. 217--220.

Digital Library

[65]

Roy J Lewicki, Daniel J McAllister, and Robert J Bies. 1998. Trust and distrust: New relationships and realities. Academy of management Review, Vol. 23, 3 (1998), 438--458.

[66]

J David Lewis and Andrew Weigert. 1985. Trust as a social reality. Social forces, Vol. 63, 4 (1985), 967--985.

[67]

Claire Liang, Julia Proft, Erik Andersen, and Ross A Knepper. 2019. Implicit communication of actionable information in human-ai teams. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--13.

Digital Library

[68]

Q Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: informing design practices for explainable AI user experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--15.

Digital Library

[69]

Joseph B Lyons, Kevin T Wynne, Sean Mahoney, and Mark A Roebke. 2019. Trust and human-machine teaming: A qualitative study. In Artificial intelligence for the internet of everything. Elsevier, 101--116.

[70]

Jean Macmillan, Elliot E Entin, and Daniel Serfaty. 2002. A framework for understanding the relationship between team structure and the communication necessary for effective team cognition. Team cognition: Process and performance at the inter-and intra-individual level (2002), 112--139.

[71]

Jean MacMillan, Elliot E Entin, and Daniel Serfaty. 2004. Communication overhead: The hidden cost of team cognition. (2004).

[72]

Michelle A Marks, John E Mathieu, and Stephen J Zaccaro. 2001. A temporally based framework and taxonomy of team processes. Academy of management review, Vol. 26, 3 (2001), 356--376.

[73]

Shannon L Marlow, Christina N Lacerenza, Jensine Paoletti, C Shawn Burke, and Eduardo Salas. 2018. Does team communication represent a one-size-fits-all approach?: A meta-analysis of team communication and performance. Organizational behavior and human decision processes, Vol. 144 (2018), 145--170.

[74]

Roger C Mayer, James H Davis, and F David Schoorman. 1995. An integrative model of organizational trust. Academy of management review, Vol. 20, 3 (1995), 709--734.

[75]

Daniel J McAllister. 1995. Affect-and cognition-based trust as foundations for interpersonal cooperation in organizations. Academy of management journal, Vol. 38, 1 (1995), 24--59.

[76]

Nathan J McNeese, Mustafa Demir, Nancy J Cooke, and Christopher Myers. 2018. Teaming with a synthetic teammate: Insights into human-autonomy teaming. Human factors, Vol. 60, 2 (2018), 262--273.

[77]

Nathan J McNeese, Mustafa Demir, Nancy J Cooke, and Manrong She. 2021. Team Situation Awareness and Conflict: A Study of Human--Machine Teaming. Journal of Cognitive Engineering and Decision Making, Vol. 15, 2--3 (2021), 83--96.

[78]

Nathan J McNeese and Madhu C Reddy. 2015. Articulating and understanding the development of a team mental model in a distributed medium. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 59. SAGE Publications Sage CA: Los Angeles, CA, 240--244.

[79]

Tim R Merritt, Kian Boon Tan, Christopher Ong, Aswin Thomas, Teong Leong Chuah, and Kevin McGee. 2011. Are artificial team-mates scapegoats in computer games. In Proceedings of the ACM 2011 conference on Computer supported cooperative work. 685--688.

Digital Library

[80]

Geoff Musick, Thomas A. O'Neill, Beau G. Schelble, Nathan J. McNeese, and Jonn B. Henke. 2021a. What Happens When Humans Believe Their Teammate is an AI? An Investigation into Humans Teaming with Autonomy. Computers in Human Behavior, Vol. 122 (Sept. 2021), 106852. https://doi.org/10.1016/j.chb.2021.106852

[81]

Geoff Musick, Rui Zhang, Nathan J McNeese, Guo Freeman, and Anurata Prabha Hridi. 2021b. Leveling Up Teamwork in Esports: Understanding Team Cognition in a Dynamic Virtual Environment. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW1 (2021), 1--30.

Digital Library

[82]

Christopher Myers, Jerry Ball, Nancy Cooke, Mary Freiman, Michelle Caisse, Stuart Rodgers, Mustafa Demir, and Nathan McNeese. 2018. Autonomous intelligent agents for team training. IEEE Intelligent Systems, Vol. 34, 2 (2018), 3--14.

Digital Library

[83]

Clifford Nass, BJ Fogg, and Youngme Moon. 1996. Can computers be teammates? International Journal of Human-Computer Studies, Vol. 45, 6 (1996), 669--678.

Digital Library

[84]

Clifford Nass, Jonathan Steuer, and Ellen R Tauber. 1994. Computers are social actors. In Proceedings of the SIGCHI conference on Human factors in computing systems. 72--78.

Digital Library

[85]

Tatsuya Nomura, Tomohiro Suzuki, Takayuki Kanda, and Kensuke Kato. 2006. Measurement of negative attitudes toward robots. Interaction Studies, Vol. 7, 3 (2006), 437--454.

[86]

Mahsan Nourani, Donald R Honeycutt, Jeremy E Block, Chiradeep Roy, Tahrima Rahman, Eric D Ragan, and Vibhav Gogate. 2020. Investigating the importance of first impressions and explainable ai with interactive video analysis. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1--8.

Digital Library

[87]

Thomas O'Neill, Nathan McNeese, Amy Barron, and Beau Schelble. 2020. Human--autonomy teaming: A review and analysis of the empirical literature. Human Factors (2020), 0018720820960865.

[88]

Atanu Pal, Roshan Lal, and Frank Frizelle. 2018. Aviation-based teamwork skills work for surgeons: time for an ?aviation bundle'? ANZ Journal of Surgery, Vol. 88, 12 (2018), 1231--1235. https://doi.org/10.1111/ans.14892 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/ans.14892.

[89]

Rebecca Randell, Stephanie Honey, Natasha Alvarado, Alan Pearman, Joanne Greenhalgh, Andrew Long, Peter Gardner, Arron Gill, David Jayne, and Dawn Dowding. 2016. Embedding robotic surgery into routine practice and impacts on communication and decision making: a review of the experience of surgical teams. Cognition, Technology & Work, Vol. 18, 2 (2016), 423--437.

Digital Library

[90]

PL Patrick Rau, Ye Li, and Dingjun Li. 2009. Effects of communication style and culture on ability to accept recommendations from robots. Computers in Human Behavior, Vol. 25, 2 (2009), 587--595.

Digital Library

[91]

Amy Rechkemmer and Ming Yin. 2022. When Confidence Meets Accuracy: Exploring the Effects of Multiple Performance Indicators on Trust in Machine Learning Models. In CHI Conference on Human Factors in Computing Systems. 1--14.

[92]

Karina A Roundtree, Michael A Goodrich, and Julie A Adams. 2019. Transparency: Transitioning from human--machine systems to human-swarm systems. Journal of Cognitive Engineering and Decision Making, Vol. 13, 3 (2019), 171--195.

[93]

Denise M Rousseau, Sim B Sitkin, Ronald S Burt, and Colin Camerer. 1998. Not so different after all: A cross-discipline view of trust. Academy of management review, Vol. 23, 3 (1998), 393--404.

[94]

Katherine E. Rowan. 1991. Goals, obstacles, and strategies in risk communication: A problem'solving approach to improving communication about risks. Journal of Applied Communication Research, Vol. 19, 4 (Nov. 1991), 300--329. https://doi.org/10.1080/00909889109365311 Publisher: Routledge _eprint: https://doi.org/10.1080/00909889109365311.

[95]

Garrett Sadler, Henri Battiste, Nhut Ho, Lauren Hoffmann, Walter Johnson, Robert Shively, Joseph Lyons, and David Smith. 2016. Effects of transparency on pilot trust and agreement in the autonomous constrained flight planner. In 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC). IEEE, 1--9.

[96]

Eduardo Salas, Carolyn Prince, David P Baker, and Lisa Shrestha. 2017. Situation awareness in team performance: Implications for measurement and training. Situational Awareness (2017), 63--76.

[97]

Eduardo Salas, Dana E Sims, and C Shawn Burke. 2005. Is there a "big five" in teamwork? Small group research, Vol. 36, 5 (2005), 555--599.

[98]

Paul M Salmon, Neville A Stanton, Guy H Walker, Chris Baber, Daniel P Jenkins, Richard McMaster, and Mark S Young. 2008. What really is going on? Review of situation awareness models for individuals and teams. Theoretical Issues in Ergonomics Science, Vol. 9, 4 (2008), 297--323.

[99]

Beau G Schelble, Christopher Flathmann, Nathan J McNeese, Guo Freeman, and Rohit Mallick. 2022a. Let's Think Together! Assessing Shared Mental Models, Performance, and Trust in Human-Agent Teams. Proceedings of the ACM on Human-Computer Interaction, Vol. 6, GROUP (2022), 1--29.

Digital Library

[100]

Beau G Schelble, Christopher Flathmann, Nathan J McNeese, Thomas O'Neill, Richard Pak, and Moses Namara. 2022b. Investigating the Effects of Perceived Teammate Artificiality on Human Performance and Cognition. International Journal of Human-Computer Interaction (2022), 1--16.

[101]

Isabella Seeber, Eva Bittner, Robert O Briggs, Triparna De Vreede, Gert-Jan De Vreede, Aaron Elkins, Ronald Maier, Alexander B Merz, Sarah Oeste-Reiß, Nils Randrup, et al. 2020. Machines as teammates: A research agenda on AI in team collaboration. Information & management, Vol. 57, 2 (2020), 103174.

[102]

Nick Sevdalis, Shabnam Undre, James McDermott, Jasdeep Giddie, Lila Diner, and Gillian Smith. 2014. Impact of intraoperative distractions on patient safety: a prospective descriptive study using validated instruments. World journal of surgery, Vol. 38, 4 (2014), 751--758.

[103]

Paul Stone, Sarah A Jessup, Subhashini Ganapathy, and Assaf Harel. 2022. Design Thinking Framework for Integration of Transparency Measures in Time-Critical Decision Support. International Journal of Human-Computer Interaction (2022), 1--17.

[104]

Michael J. L. Sullivan, Pascal Thibault, André Savard, Richard Catchlove, John Kozey, and William D. Stanish. 2006. The influence of communication goals and physical demands on different dimensions of pain behavior. Pain, Vol. 125, 3 (Dec. 2006), 270--277. https://doi.org/10.1016/j.pain.2006.06.019

[105]

Dag Sverre Syrdal, Kerstin Dautenhahn, Kheng Lee Koay, and Michael L Walters. 2009. The negative attitudes towards robots scale and reactions to robot behaviour in a live human-robot interaction study. Adaptive and emergent behaviour and complex systems (2009).

[106]

Fangcheng Tang, Jifeng Mu, and Ellen Thomas. 2015. Who knows what in NPD teams: Communication context, mode, and task contingencies. Journal of Product Innovation Management, Vol. 32, 3 (2015), 404--423.

[107]

Claire Textor, Rui Zhang, Jeremy Lopez, Beau G Schelble, Nathan J McNeese, Guo Freeman, Richard Pak, Chad Tossell, and Ewart J de Visser. 2022. Exploring the Relationship Between Ethics and Trust in Human--Artificial Intelligence Teaming: A Mixed Methods Approach. Journal of Cognitive Engineering and Decision Making, Vol. 16, 4 (2022), 252--281.

[108]

Judith Tiferes and Ann M Bisantz. 2018. The impact of team characteristics and context on team communication: An integrative literature review. Applied ergonomics, Vol. 68 (2018), 146--159.

[109]

Suzanne Tolmeijer, Ujwal Gadiraju, Ramya Ghantasala, Akshit Gupta, and Abraham Bernstein. 2021. Second chance for a first impression? Trust development in intelligent system interaction. In Proceedings of the 29th ACM Conference on user modeling, adaptation and personalization. 77--87.

Digital Library

[110]

James C Walliser, Ewart J de Visser, Eva Wiese, and Tyler H Shaw. 2019. Team structure and team building improve human--machine teaming with autonomous agents. Journal of Cognitive Engineering and Decision Making, Vol. 13, 4 (2019), 258--278.

[111]

James C Walliser, Patrick R Mead, and Tyler H Shaw. 2017. The perception of teamwork with an autonomous agent enhances affect and performance outcomes. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 61. SAGE Publications Sage CA: Los Angeles, CA, 231--235.

[112]

Dakuo Wang, Justin D Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019. Human-AI collaboration in data science: Exploring data scientists' perceptions of automated AI. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--24.

Digital Library

[113]

Peter H. Welch, Jo R. Aldous, and Jon Foster. 2002. CSP Networking for Java (JCSP. net). In Computational Science - ICCS 2002 (Lecture Notes in Computer Science ), Peter M. A. Sloot, Alfons G. Hoekstra, C. J. Kenneth Tan, and Jack J. Dongarra (Eds.). Springer, Berlin, Heidelberg, 695--708. https://doi.org/10.1007/3--540--46080--2_74

[114]

H James Wilson and Paul R Daugherty. 2018. Collaborative intelligence: Humans and AI are joining forces. Harvard Business Review, Vol. 96, 4 (2018), 114--123.

[115]

Julia L Wright, Jessie YC Chen, Michael J Barnes, and Peter A Hancock. 2016. Agent reasoning transparency's effect on operator workload. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 60. SAGE Publications Sage CA: Los Angeles, CA, 249--253.

[116]

Julia L Wright, Shan G Lakhmani, and Jessie YC Chen. 2022. Bidirectional Communications in Human-Agent Teaming: The Effects of Communication Style and Feedback. International Journal of Human-Computer Interaction (2022), 1--14.

[117]

Melanie C Wright and David B Kaber. 2003. Team coordination and strategies under automation. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 47. SAGE Publications Sage CA: Los Angeles, CA, 553--557.

[118]

Chengke Wu, Xiao Li, Yuanjun Guo, Jun Wang, Zengle Ren, Meng Wang, and Zhile Yang. 2022. Natural language processing for smart construction: Current status and future directions. Automation in Construction, Vol. 134 (2022), 104059.

[119]

Kevin T Wynne and Joseph B Lyons. 2018. An integrative model of autonomous agent teammate-likeness. Theoretical Issues in Ergonomics Science, Vol. 19, 3 (2018), 353--374.

[120]

Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1--12.

Digital Library

[121]

Dale E Zand. 1972. Trust and managerial problem solving. Administrative science quarterly (1972), 229--239.

[122]

Qiaoning Zhang, Matthew L Lee, and Scott Carter. 2022. You Complete Me: Human-AI Teams and Complementary Expertise. In CHI Conference on Human Factors in Computing Systems. 1--28.

[123]

Rui Zhang, Nathan J McNeese, Guo Freeman, and Geoff Musick. 2021. " An Ideal Human" Expectations of AI Teammates in Human-AI Teaming. Proceedings of the ACM on Human-Computer Interaction, Vol. 4, CSCW3 (2021), 1--25.

[124]

Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295--305.

Digital Library

[125]

Zhan Zhang, Aleksandra Sarcevic, Maria Yala, and Randall S Burd. 2014. Informing digital cognitive aids design for emergency medical work by understanding paper checklist use. In Proceedings of the 18th international conference on supporting group work. 204--214.

Digital Library

[126]

Emma M van Zoelen, Anita Cremers, Frank PM Dignum, Jurriaan van Diggelen, and Marieke M Peeters. 2020. Learning to communicate proactively in human-agent teaming. In International Conference on Practical Applications of Agents and Multi-Agent Systems. Springer, 238--249.

Cited By

Duan WWeng NScalia MZhang RTuttle JYin XZhou SFreeman GGorman JFunke GTolston MMcNeese N(2024)Getting Along With Autonomous Teammates: Understanding the Socio-Emotional and Teaming Aspects of Trust in Human-Autonomy TeamsProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/10711813241272123Online publication date: 21-Oct-2024
https://doi.org/10.1177/10711813241272123
Farah YDorneich M(2024)Human-Autonomy Teaming in a Cooperative Gamified Testbed: How Can AI Teammates Support Teamwork Processes?Proceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/10711813241269252Online publication date: 24-Oct-2024
https://doi.org/10.1177/10711813241269252
Chang ZZhang FMa XWang PChen WZhang DJouaber BZhang D(2024)MmECare: Enabling Fine-grained Vital Sign Monitoring for Emergency Care with Handheld MmWave RadarsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997668:4(1-24)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699766
Show More Cited By

Index Terms

Investigating AI Teammate Communication Strategies and Their Impact in Human-AI Teams for Effective Teamwork
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Verbal vs. Visual: How Humans Perceive and Collaborate with AI Teammates Using Different Communication Modalities in Various Human-AI Team Compositions
CSCW

As AI becomes more advanced in natural language processing, the research on AI's communication in teaming environments is getting more attention in CSCW/HCI. Even though AI's communication serves as an essential part of facilitating team coordination and ...
Examining the impact of varying levels of AI teammate influence on human-AI teams
Abstract
The implementation of AI teammates is creating a wealth of research that examines how AI teammates impact human-AI teams. However, AI teammates themselves are not static, and their roles and responsibilities in human-AI teams are ...
Highlights
- Foundational exploration on the influence AI teammates have on shared work.
- AI ...
I Know This Looks Bad, But I Can Explain: Understanding When AI Should Explain Actions In Human-AI Teams
Explanation of artificial intelligence (AI) decision-making has become an important research area in human–computer interaction (HCI) and computer-supported teamwork research. While plenty of research has investigated AI explanations with an intent to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 7, Issue CSCW2

CSCW

October 2023

4055 pages

EISSN:2573-0142

DOI:10.1145/3626953

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 October 2023

Published in PACMHCI Volume 7, Issue CSCW2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
2,257
Total Downloads

Downloads (Last 12 months)1,958
Downloads (Last 6 weeks)275

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Duan WWeng NScalia MZhang RTuttle JYin XZhou SFreeman GGorman JFunke GTolston MMcNeese N(2024)Getting Along With Autonomous Teammates: Understanding the Socio-Emotional and Teaming Aspects of Trust in Human-Autonomy TeamsProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/10711813241272123Online publication date: 21-Oct-2024
https://doi.org/10.1177/10711813241272123
Farah YDorneich M(2024)Human-Autonomy Teaming in a Cooperative Gamified Testbed: How Can AI Teammates Support Teamwork Processes?Proceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/10711813241269252Online publication date: 24-Oct-2024
https://doi.org/10.1177/10711813241269252
Chang ZZhang FMa XWang PChen WZhang DJouaber BZhang D(2024)MmECare: Enabling Fine-grained Vital Sign Monitoring for Emergency Care with Handheld MmWave RadarsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997668:4(1-24)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699766
Duan WZhou SScalia MYin XWeng NZhang RFreeman GMcNeese NGorman JTolston M(2024)Understanding the Evolvement of Trust Over Time within Human-AI TeamsProceedings of the ACM on Human-Computer Interaction10.1145/36870608:CSCW2(1-31)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687060
Leong JTang JCutrell EJunuzovic SBaribault GInkpen K(2024)Dittos: Personalized, Embodied Agents That Participate in Meetings When You Are UnavailableProceedings of the ACM on Human-Computer Interaction10.1145/36870338:CSCW2(1-28)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687033
Zhang RDuan WFlathmann CMcNeese NKnijnenburg BFreeman G(2024)Verbal vs. Visual: How Humans Perceive and Collaborate with AI Teammates Using Different Communication Modalities in Various Human-AI Team CompositionsProceedings of the ACM on Human-Computer Interaction10.1145/36869768:CSCW2(1-34)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686976
Huber SPapenfuß NWeppert LWohlfart VBasch JHappel OGrundgeiger T(2024)Hiring an AI: Incorporating Personnel Selection Methods in User-Centered Design to Design AI Agents for Safety-Critical DomainsAdjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction10.1145/3677045.3685418(1-9)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3677045.3685418
He JHoude SGonzalez GSilva Moran DRoss SMuller MWeisz J(2024)AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual CanvasProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663398(1-14)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3663384.3663398
Xu ZZhang JGreenberg JFrumkin MJaveed SZhang JBenedict BBotterbush KRodebaugh TRay WLu C(2024)Predicting Multi-dimensional Surgical Outcomes with Multi-modal Mobile SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596288:2(1-30)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659628
Jeong DHan K(2024)PRECYSE: Predicting Cybersickness using Transformer for Multimodal Time-Series Sensor DataProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595948:2(1-24)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659594
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents