Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

CoCo: Collaboration Coach for Understanding Team Dynamics during Video Conferencing

Published: 08 January 2018 Publication History

Abstract

We present and discuss a fully-automated collaboration system, CoCo, that allows multiple participants to video chat and receive feedback through custom video conferencing software. After a conferencing session, a virtual feedback assistant provides insights on the conversation to participants. CoCo automatically pulls audial and visual data during conversations and analyzes the extracted streams for affective features, including smiles, engagement, attention, as well as speech overlap and turn-taking. We validated CoCo with 39 participants split into 10 groups. Participants played two back-to-back team-building games, Lost at Sea and Survival on the Moon, with the system providing feedback between the two. With feedback, we found a statistically significant change in balanced participation---that is, everyone spoke for an equal amount of time. There was also statistically significant improvement in participants' self-evaluations of conversational skills awareness, including how often they let others speak, as well as of teammates' conversational skills. The entire framework is available at https://github.com/ROC-HCI/CollaborationCoach_PostFeedback.

References

[1]
1996. Lost at Sea. (1996). http://insight.typepad.co.uk/lost_at_sea.pdf
[2]
2011. Siri. (2011). https://en.wikipedia.org/wiki/Siri
[3]
2015. Video Conferencing Trends 2016. http://resources.idgenterprise.com/original/AST-0162982_Video_Conferencing_Trends_of_2016.pdf
[4]
2016. Affectiva. http://www.affectiva.com/. (2016). Accessed: 2016-10-30.
[5]
Brandon Amos, Bartosz Ludwiczuk, and Mahadev Satyanarayanan. 2016. OpenFace: A general-purpose face recognition library with mobile applications. Technical Report. Technical report, CMU-CS-16-118, CMU School of Computer Science.
[6]
Patricia B. Barger and Alicia A. Grandey. 2006. Service with a Smile and Encounter Satisfaction: Emotional Contagion and Appraisal Mechanisms. The Academy of Management Journal 49, 6 (2006), 1229--1238. http://www.jstor.org/stable/20159829
[7]
Sigal G. Barsade. 2002. The Ripple Effect: Emotional Contagion and Its Influence on Group Behavior. Administrative Science Quarterly 47, 4 (2002), 644--675. http://www.jstor.org/stable/3094912
[8]
JÃľrÃľmy Bauchet, HÃľlÃÍne Pigot, Sylvain Giroux, Dany Lussier-Desrochers, Yves Lachapelle, and Mounir Mokhtari. [n. d.]. Designing judicious interactions for cognitive assistance: the acts of assistance approach. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility. ACM, 11--18. https://doi.org/10.1145/1639642.1639647
[9]
Paul Boersma and David Weenink. 2016. Praat: doing phonetics by computer. http://www.praat.org/. (2016). Accessed: 2016-06-01.
[10]
LouAnne E. Boyd, Alejandro Rangel, Helen Tomimbang, Andrea Conejo-Toledo, Kanika Patel, Monica Tentori, and Gillian R. Hayes. 2016. SayWAT: Augmenting Face-to-Face Conversations for Adults with Autism. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ‘16). ACM, New York, NY, USA, 4872--4883. https://doi.org/10.1145/2858036.2858215
[11]
Erin Bradner, Gloria Mark, and Tammie D. Hertel. 2005. Team size and technology fit: Participation, awareness, and rapport in distributed teams. IEEE Transactions on Professional Communication 48, 1 (2005), 68--77. https://doi.org/10.1109/TPC.2005.843299
[12]
T. Prevett. C. Wickens. 1995. Exploring the dimensions of egocentricity in aircraft navigation displays: influences on local guidance and global situation awareness. Journal of Experimental Psychology: Applied, Neuilly-Sur-Seine, France.
[13]
Dan Calacci, Oren Lederman, David Shrier, and Alex Sandy Pentland. 2016. Breakout: An Open Measurement and Intervention Tool for Distributed Peer Learning Groups. CoRR abs/1607.01443 (2016). http://arxiv.org/abs/1607.01443
[14]
Jean Carletta, Simone Ashby, Sebastien Bourban, Mike Flynn, Mael Guillemot, Thomas Hain, Jaroslav Kadlec, Vasilis Karaiskos, Wessel Kraaij, Melissa Kronenthal, Guillaume Lathoud, Mike Lincoln, Agnes Lisowska, Iain McCowan, Wilfried Post, Dennis Reidsma, and Pierre Wellner. 2006. The AMI Meeting Corpus: A Pre-announcement. In Proceedings of the Second International Conference on Machine Learning for Multimodal Interaction (MLMI‘05). Springer-Verlag, Berlin, Heidelberg, 28--39. https://doi.org/10.1007/11677482_3
[15]
Samuele Centorrino, Elodie Djemai, Astrid Hopfensitz, Manfred Milinski, and Paul Seabright. 2015. Honest signaling in trust interactions: smiles rated as genuine induce trust and signal higher earning opportunities. Evolution and Human Behavior 36, 1 (2015), 8--16. https://doi.org/10.1016/j.evolhumbehav.2014.08.001
[16]
K. M. Cohen. 1982. Speaker interaction: Video teleconference versus face-to-face meetings (Teleconferencing and Electronic Communications). ACM, University of Wisconsin.
[17]
RALPH D'AGOSTINO and Egon S Pearson. 1973. Tests for departure from normality. Empirical results for the distributions of b 2 andâĹŽ b. Biometrika 60, 3 (1973), 613--622.
[18]
Ralph B d'Agostino. 1971. An omnibus test of normality for moderate and large size samples. Biometrika 58, 2 (1971), 341--348.
[19]
Ionut Damian, Chiew Seng (Sean) Tan, Tobias Baur, Johannes SchÃűning, Kris Luyten, and Elisabeth AndrÃľ. [n. d.]. Augmenting Social Interactions: Realtime Behavioural Feedback using Social Signal Processing Techniques. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 565--574. https://doi.org/10.1145/2702123.2702314
[20]
Richard W. DeVaul, Alex “Sandy” Pentland, and Vicka R. Corey. 2003. The Memory Glasses: Subliminal vs. Overt Memory Support with Imperfect Information. In Proceedings of the 7th IEEE International Symposium on Wearable Computers (ISWC ‘03). IEEE Computer Society, Washington, DC, USA, 146--. http://dl.acm.org/citation.cfm?id=946249.946907
[21]
J. J. Diehl and R. Paul. 2009. The assessment and treatment of prosodic disorders and neurological theories of prosody. Int J Speech Lang Pathol 11, 4 (2009), 287--92. https://doi.org/10.1080/17549500902971887
[22]
Joan Morris DiMicco, Anna Pandolfo, and Walter Bender. 2004. Influencing group participation with a shared display. Proceedings of the 2004 ACM conference on Computer supported cooperative work - CSCW ‘04 (2004), 614--623. https://doi.org/10.1145/1031607.1031713
[23]
Norah E. Dunbar and Judee K. Burgoon. 2005. Perceptions of power and interactional dominance in interpersonal relationships. Journal of Social and Personal Relationships 22, 2 (2005), 207--233. https://doi.org/10.1177/0265407505050944 arXiv:http://dx.doi.org/10.1177/0265407505050944
[24]
Jay Hall and Wilfred Harvey Watson. 1970. Survival on the moon. Human relations (1970). https://www.psychologicalscience.org/observer/nasa-exercise
[25]
Laura A. Hambley, Thomas A. O'Neill, and Theresa J B Kline. 2007. Virtual team leadership: The effects of leadership style and communication medium on team interaction styles and outcomes. Organizational Behavior and Human Decision Processes 103, 1 (2007), 1--20. https://doi.org/10.1016/j.obhdp.2006.09.004
[26]
Crystal L. Hoyt and Jim Blascovich. 2003. Transformational and Transactional Leadership in Virtual and Physical Environments. Small Group Research 34, 6 (2003), 678--715. https://doi.org/10.1177/1046496403257527 arXiv:http://dx.doi.org/10.1177/1046496403257527
[27]
H. Hung, Yan Huang, G. Friedland, and D. Gatica-Perez. 2011. Estimating Dominance in Multi-Party Meetings Using Speaker Diarization. Trans. Audio, Speech and Lang. Proc. 19, 4 (May 2011), 847--860. https://doi.org/10.1109/TASL.2010.2066267
[28]
Inseok HWANG, Chungkuk YOO, Chanyou HWANG, Dongsun YIM, Youngki LEE, Singapore Management University, Chulhong MIN, Hjohn KIM, and Junehwa SONG. 2014. TalkBetter: Family-driven Mobile Intervention Care for Children with Language Delay. (2014), 1283. https://doi.org/10.1145/2531602.2531668
[29]
Jeroen Janssen, Gijsbert Erkens, and Gellof Kanselaar. 2007. Visualization of Agreement and Discussion Processes During Computer-supported Collaborative Learning. Comput. Hum. Behav. 23, 3 (May 2007), 1105--1125. https://doi.org/10.1016/j.chb.2006.10.005
[30]
Jeroen Janssen, Gijsbert Erkens, Gellof Kanselaar, and Jos Jaspers. 2007. Visualization of Participation: Does It Contribute to Successful Computer-supported Collaborative Learning? Comput. Educ. 49, 4 (Dec. 2007), 1037--1065. https://doi.org/10.1016/j.compedu.2006.01.004
[31]
Dinesh Babu Jayagopi, Hayley Hung, Chuohao Yeo, and Daniel Gatica-Perez. 2009. Modeling Dominance in Group Conversations Using Nonverbal Activity Cues. Trans. Audio, Speech and Lang. Proc. 17, 3 (March 2009), 501--513. https://doi.org/10.1109/TASL.2008.2008238
[32]
Vaiva Kalnikaité, Patrick Ehlen, and Steve Whittaker. 2012. Markup As You Talk: Establishing Effective Memory Cues While Still Contributing to a Meeting. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW ‘12). ACM, New York, NY, USA, 349--358. https://doi.org/10.1145/2145204.2145260
[33]
Taemie Kim, Pamela Hinds, and Alex Pentland. 2012. Awareness As an Antidote to Distance: Making Distributed Groups Cooperative and Consistent. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW ‘12). ACM, New York, NY, USA, 1237--1246. https://doi.org/10.1145/2145204.2145391
[34]
G. Ross. Lawford. 2003. Beyond success: Achieving synergy in teamwork. The Journal for Quality and Participation 26, 3 (Fall 2003), 23--27. https://search.proquest.com/docview/219147502?accountid=13567
[35]
Yale Child Jane L. Mcsweeny Ami Klin Donald J. Cohen Fred R. Volkmar Lawrence D. Shriberg, Rhea Paul. [n. d.]. Speech and prosody characteristics of adolescents and adults with high functioning autism and Asperger syndrome. In Journal of Speech, Language, and Hearing Research. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.385.7116
[36]
Harold J. Leavitt. 1951. Some effects of certain communication patterns on group performance. Journal of abnormal psychology (1951), 38--50.
[37]
Gilly Leshed. 2009. Automated language-based feedback for teamwork behaviors. Ph.D. Dissertation. https://search.proquest.com/docview/304872230?accountid=13567 Copyright - Database copyright ProQuest LLC; ProQuest does not claim copyright in the individual underlying works; Last updated - 2016-06-03.
[38]
Gilly Leshed, Diego Perez, Jeffrey T. Hancock, Dan Cosley, Jeremy Birnholtz, Soyoung Lee, Poppy L. McLeod, and Geri Gay. 2009. Visualizing Real-time Language-based Feedback on Teamwork Behavior in Computer-mediated Groups. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2009), 537--546. https://doi.org/10.1145/1518701.1518784
[39]
Bill Mark and R Perrault. 2005. Calo: Cognitive assistant that learns and organizes. (2005).
[40]
Poppy Lauretta McLeod, Jeffrey K. Liker, and Sharon A. Lobel. 1992. Process Feedback in Task Groups: An Application of Goal Setting. The Journal of Applied Behavioral Science 28, 1 (1992), 15--41. https://doi.org/10.1177/0021886392281003 arXiv:http://dx.doi.org/10.1177/0021886392281003
[41]
Janice Nadler. 2003. Rapport in negotiation and conflict resolution. Marq. L. Rev. 285, 1990 (2003), 875--882. https://doi.org/10.1525/sp.2007.54.1.23.
[42]
Eyal Ofek, Shamsi T. Iqbal, and Karin Strauss. 2013. Reducing Disruption from Subtle Information Delivery During a Conversation: Mode and Bandwidth Investigation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘13). ACM, New York, NY, USA, 3111--3120. https://doi.org/10.1145/2470654.2466425
[43]
Jeffery R. Ringer. 1985. Pardon Me, Can I Talk Now?: A Look at the Roles of Interruptions in Conversation. Annual Meeting of the Organization for the Study of Communication, Language, and Gneder (1985), 1--29.
[44]
Marcello Russo. 2012. Diversity in goal orientation, team performance, and internal team environment. Equality, Diversity and Inclusion: An International Journal 31, 2 (2012), 124--143. https://doi.org/10.1108/02610151211202781 arXiv:http://dx.doi.org/10.1108/02610151211202781
[45]
Harvey Sacks, Emanuel a Schegloff, and Gail Jefferson. 1974. A simplest systematics for the organization of turn taking for conversation. (1974), 696--735 pages. https://doi.org/10.2307/412243
[46]
M. Satyanarayanan. 2004. From the Editor in Chief: Augmenting Cognition. IEEE Pervasive Computing 3, 2 (2004), 4--5. https://doi.org/10.1109/MPRV.2004.1316809
[47]
Charles R. Scherl and Jay Haley. 2000. Computer Monitor Supervision: A Clinical Note. The American Journal of Family Therapy 28, 3 (2000), 275--282. https://doi.org/10.1080/01926180050081702 arXiv:http://dx.doi.org/10.1080/01926180050081702
[48]
Mark Schittekatte and Alain Van Hiel. 1996. Effects of Partially Shared Information and Awareness of Unshared Information on Information Sampling. Small Group Research 27, 3 (1996), 431--449. https://doi.org/10.1177/1046496496273006 arXiv:http://dx.doi.org/10.1177/1046496496273006
[49]
Abigail J. Sellen. 1992. Speech Patterns in Video-mediated Conversations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘92). ACM, New York, NY, USA, 49--59. https://doi.org/10.1145/142750.142756
[50]
D. Tanen. 1995. The Power of Talk: Who Gets Heard and Why. Harvard Business Review (1995).
[51]
Deborah Tannen. 1997. Gender and Discourse. Discourse (1997), 548--567. https://doi.org/10.2307/2655317
[52]
M. Iftekhar Tanveer, Emy Lin, and Mohammed (Ehsan) Hoque. 2015. Rhema: A Real-Time In-Situ Intelligent Interface to Help People with Public Speaking. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI ‘15). ACM, New York, NY, USA, 286--295. https://doi.org/10.1145/2678025.2701386
[53]
Yla R. Tausczik and James W. Pennebaker. 2013. Improving teamwork using real-time language feedback. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ‘13 (2013), 459--468. https://doi.org/10.1145/2470654.2470720
[54]
Monica Tentori and Gillian R. Hayes. [n. d.]. Designing for interaction immediacy to enhance social skills of children with autism. In Proceedings of the 12th ACM international conference on Ubiquitous computing. ACM, 51--60. https://doi.org/10.1145/1864349.1864359
[55]
Linda Tickle-Degnen and Robert Rosenthal. 1990. The Nature of Rapport and Its Nonverbal Correlates. Psychological Inquiry 1, 4 (1990), 285--293. http://www.jstor.org/stable/1449345
[56]
Janet van der Linden, Rose Johnson, Jon Bird, Yvonne Rogers, and Erwin Schoonderwaldt. 2011. Buzzing to Play: Lessons Learned from an in the Wild Study of Real-time Vibrotactile Feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘11). ACM, New York, NY, USA, 533--542. https://doi.org/10.1145/1978942.1979017
[57]
Joseph B. Walther and Lisa C. Tidwell. 1995. Nonverbal cues in computer-mediated communication, and the effect of chronemics on relational communication. J. Org. Computing 5, 4 (1995), 355--378. http://dblp.uni-trier.de/db/journals/jocec/jocec5.html
[58]
C. Wickens. 1996. Situation awareness: impact of automation and display technology. In NATO AGARD Aerospace Medical Panel Symposium on Situation Awareness.
[59]
Anita Williams Woolley, Christopher F. Chabris, Alex Pentland, Nada Hashmi, and Thomas W. Malone. 2010. Evidence for a Collective Intelligence Factor in the Performance of Human Groups. Science 330, 6004 (2010), 686--688. https://doi.org/10.1126/science.1193147 arXiv:http://science.sciencemag.org/content/330/6004/686.full.pdf
[60]
Qianli Xu, Michal Mukawa, Liyuan Li, Joo Hwee Lim, Cheston Tan, Shue Ching Chia, Tian Gan, and Bappaditya Mandal. [n. d.]. Exploring users' attitudes towards social interaction assistance on Google Glass. In Proceedings of the 6th Augmented Human International Conference. ACM, 9--12. https://doi.org/10.1145/2735711.2735831

Cited By

View all
  • (2024)The CoExplorer Technology Probe: A Generative AI-Powered Adaptive Interface to Support Intentionality in Planning and Running Video MeetingsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661507(1638-1657)Online publication date: 1-Jul-2024
  • (2024)PeerGPT: Probing the Roles of LLM-based Peer Agents as Team Moderators and Participants in Children's Collaborative LearningExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651008(1-6)Online publication date: 11-May-2024
  • (2024)CoExplorer: Generative AI Powered 2D and 3D Adaptive Interfaces to Support Intentionality in Video MeetingsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650797(1-10)Online publication date: 11-May-2024
  • Show More Cited By

Index Terms

  1. CoCo: Collaboration Coach for Understanding Team Dynamics during Video Conferencing

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 1, Issue 4
    December 2017
    1298 pages
    EISSN:2474-9567
    DOI:10.1145/3178157
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 January 2018
    Accepted: 01 October 2017
    Revised: 01 August 2017
    Received: 01 May 2017
    Published in IMWUT Volume 1, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Group discussion
    2. Team dynamics
    3. Video conferencing
    4. Virtual feedback system

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)164
    • Downloads (Last 6 weeks)35
    Reflects downloads up to 16 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)The CoExplorer Technology Probe: A Generative AI-Powered Adaptive Interface to Support Intentionality in Planning and Running Video MeetingsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661507(1638-1657)Online publication date: 1-Jul-2024
    • (2024)PeerGPT: Probing the Roles of LLM-based Peer Agents as Team Moderators and Participants in Children's Collaborative LearningExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651008(1-6)Online publication date: 11-May-2024
    • (2024)CoExplorer: Generative AI Powered 2D and 3D Adaptive Interfaces to Support Intentionality in Video MeetingsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650797(1-10)Online publication date: 11-May-2024
    • (2024)GazePrompt: Enhancing Low Vision People's Reading Experience with Gaze-Aware AugmentationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642878(1-17)Online publication date: 11-May-2024
    • (2024)Talking Face Generation for Impression Conversion Considering Speech SemanticsICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10446947(8411-8415)Online publication date: 14-Apr-2024
    • (2024)Digitally supported coaching and its application to developing self-organizing teamsHuman-Intelligent Systems Integration10.1007/s42454-024-00056-6Online publication date: 8-Nov-2024
    • (2023)Developing Teams by Visualizing Their Communication Structures in Online MeetingsMultimodal Technologies and Interaction10.3390/mti71001007:10(100)Online publication date: 19-Oct-2023
    • (2023)Understanding the effect of a virtual moderator on people’s perception in remote discussion using social VRFrontiers in Virtual Reality10.3389/frvir.2023.11980244Online publication date: 18-Jul-2023
    • (2023)MeetScript: Designing Transcript-based Interactions to Support Active Participation in Group Video MeetingsProceedings of the ACM on Human-Computer Interaction10.1145/36101967:CSCW2(1-32)Online publication date: 4-Oct-2023
    • (2023)HyWayProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962357:2(1-33)Online publication date: 12-Jun-2023
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media