research-article

Public Access

How Trainees Use the Information from Telepointers in Remote Instruction

Authors:

Hannah McGowan,

H. Reza Zahiri,

Andrea Kleinsmith,

Helena MentisAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 3, Issue CSCW

Article No.: 93, Pages 1 - 20

https://doi.org/10.1145/3359195

Published: 07 November 2019 Publication History

Abstract

Researchers have shown both performance drawbacks and benefits of using telepointers or similar display overlay-technologies in remote instruction; however, there is not a clear understanding of why there are these performance effects. This poses a challenge in knowing how and when to successfully use or design telepointing technologies in remote instruction. A better understanding is needed with the rise of remote workers in a wide array of industries from oil rig repair to surgery, and the proliferation of heads-up displays or telecommunications devices to support these future work practices. In this study, we explore how the information conveyed through a telepointer is taken up and acted upon by surgical trainees in a laparoscopic surgical telementoring setting. We collected audio and video data of 12 surgical trainees who performed standard laparoscopic surgical tasks on a physical model under the guidance of a surgical trainer. We investigated both action and talk to determine how the telepointer-based information was used. Our findings reveal three main challenges in using the instructional information conveyed through the telepointer including the trainees' tendency of attending to the telepointer instruction as the primary source of information. We argue that the found challenges are socio-technical in nature and require a redesign of the mentoring context as well as the technological tools.

References

[1]

Judith Amores, Xavier Benavides, and Pattie Maes. 2015. ShowMe. Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '15. https://doi.org/10.1145/2702613.2732927

[2]

K. M. Augestad, H. Han, J. Paige, T. Ponsky, C. M. Schlachta, B. Dunkin, and J. Mellinger. 2017. Educational implications for surgical telementoring: a current review with recommendations for future practice, policy, and research. Surgical Endoscopy 31, 10: 3836--3846.

[3]

David Benyon, Phil Turner, and Susan Turner. 2005. Designing Interactive Systems: People, Activities, Contexts, Technologies. Pearson Education.

[4]

Eric A. Bier and Steven Freeman. 1991. MMM. Proceedings of the 4th Annual ACM Symposium on User interface Software and Ttechnology - UIST '91. https://doi.org/10.1145/120782.120791

[5]

S. A. Bly and S. L. Minneman. 1990. Commune: a shared drawing surface. Proceedings of the Conference on Office Information Systems -. https://doi.org/10.1145/91474.91514

[6]

Rose Challenger and Chris W. Clegg. 2011. Crowd disasters: a socio-technical systems perspective. Contemporary Social Science 6, 343--360. https://doi.org/10.1080/21582041.2011.619862

[7]

Yuan-Chia Chang, Hao-Chuan Wang, Hung-Kuo Chu, Shung-Ying Lin, and Shuo-Ping Wang. 2017. AlphaRead. Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing - CSCW '17. https://doi.org/10.1145/2998181.2998258

[8]

Nicklas H. Christensen, Oliver G. Hjermitslev, Frederik Falk, Marco B. Madsen, Frederik H. Østergaard, Martin Kibsgaard, Martin Kraus, Johan Poulsen, and Jane Petersson. 2017. Depth cues in augmented reality for training of robot-assisted minimally invasive surgery. Proceedings of the 21st International Academic Mindtrek Conference on - AcademicMindtrek '17. https://doi.org/10.1145/3131085.3131123

Digital Library

[9]

Dragos Datcu, Stephan G. Lukosch, and Heide K. Lukosch. 2016. Handheld Augmented Reality for Distributed Collaborative Crime Scene Investigation. Proceedings of the 19th International Conference on Supporting Group Work - GROUP '16. https://doi.org/10.1145/2957276.2957302

[10]

Jeff Dyck, Carl Gutwin, Sriram Subramanian, and Christopher Fedak. 2004. High-performance telepointers. Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work - CSCW '04. https://doi.org/10.1145/1031607.1031636

Digital Library

[11]

Yuanyuan Feng, Katie Li, Azin Semsar, Hannah McGowan, Jacqueline Mun, H. Reza Zahiri, Ivan George, Adrian Park, Andrea Kleinsmith, and Helena M. Mentis. 2019. Communication Cost of Single-user Gesturing Tool in Laparoscopic Surgical Training. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19. https://doi.org/10.1145/3290605.3300841

[12]

Yuanyuan Feng, Hannah McGowan, Azin Semsar, Hamid R. Zahiri, Ivan M. George, Timothy Turner, Adrian Park, Andrea Kleinsmith, and Helena M. Mentis. 2018. A Virtual Pointer to Support the Adoption of Professional Vision in Laparoscopic Training. International Journal of Computer Assisted Radiology and Surgery. 13.9 (2018): 1463--1472.

[13]

Susan R. Fussell, Robert E. Kraut, and Jane Siegel. 2000. Coordination of Communication: Effects of Shared Visual Context on Collaborative Work. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (CSCW '00), 21--30.

Digital Library

[14]

Susan R. Fussell, Leslie D. Setlock, Elizabeth M. Parker, and Jie Yang. 2003. Assessing the value of a cursor pointing device for remote collaboration on physical tasks. CHI '03 Extended Abstracts on Human Factors in Computing Systems - CHI '03. https://doi.org/10.1145/765891.765992

Digital Library

[15]

Susan R. Fussell, Leslie D. Setlock, Jie Yang, Jiazhi Ou, Elizabeth Mauer, and Adam D. I. Kramer. 2004. Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks. Human--Computer Interaction 19, 273--309. https://doi.org/10.1207/s15327051hci1903_3

Digital Library

[16]

Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014. In touch with the remote world. Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology - VRST '14. https://doi.org/10.1145/2671015.2671016

Digital Library

[17]

Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014. World-stabilized annotations and virtual scene navigation for remote collaboration. Proceedings of the 27th annual ACM Symposium on User Interface Software and Technology - UIST '14. https://doi.org/10.1145/2642918.2647372

Digital Library

[18]

Pavel Gurevich, Joel Lanir, and Benjamin Cohen. 2015. Design and Implementation of TeleAdvisor: a Projection-Based Augmented Reality System for Remote Collaboration. Computer Supported Cooperative Work (CSCW) 24, 527--562. https://doi.org/10.1007/s10606-015--9232--7

Digital Library

[19]

Carl Gutwin and Reagan Penner. 2002. Improving interpretation of remote gestures with telepointer traces. Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work - CSCW '02. https://doi.org/10.1145/587078.587086

Digital Library

[20]

Christian Heath, Jon Hindmarsh, and Paul Luff. 2010. Video in Qualitative Research: Analysing Social Interaction in Everyday Life. https://doi.org/10.4135/9781526435385

[21]

Christian Heath, Paul Luff, Hideaki Kuzuoka, Keiichi Yamazaki, and Shinya Oyama. Creating Coherent Environments for Collaboration. ECSCW 2001, 119--138. https://doi.org/10.1007/0--306--48019-0_7

[22]

Alexa Hepburn and Galina B. Bolden. 2012. The Conversation Analytic Approach to Transcription. The Handbook of Conversation Analysis, 57--76. https://doi.org/10.1002/9781118325001.ch4

[23]

Keita Higuch, Ryo Yonetani, and Yoichi Sato. 2016. Can Eye Help You? Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16. https://doi.org/10.1145/2858036.2858438

Digital Library

[24]

Jon Hindmarsh and Christian Heath. 2000. Embodied reference: A study of deixis in workplace interaction. Journal of Pragmatics 32, 1855--1878. https://doi.org/10.1016/s0378--2166(99)00122--8

[25]

Weidong Huang, Leila Alem, Franco Tecchia, and Henry Been-Lirn Duh. 2018. Augmented 3D hands: a gesture-based mixed reality system for distributed collaboration. Journal on Multimodal User Interfaces 12, 77--89. https://doi.org/10.1007/s12193-017-0250--2

[26]

Hiroshi Ishii and Minoru Kobayashi. 1992. ClearBoard. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '92. https://doi.org/10.1145/142750.142977

[27]

Rohana Abdul Karim, Nor Farizan Zakaria, Mohd Asyraf Zulkifley, Mohd Marzuki Mustafa, Ismail Sagap, and Nani Harlina Md Latar. 2013. Telepointer technology in telemedicine : a review. BioMedical Engineering OnLine 12, 21. https://doi.org/10.1186/1475--925x-12--21

[28]

Laurent Karsenty. 1999. Cooperative Work and Shared Visual Context: An Empirical Study of Comprehension Problems in Side-by-Side and Remote Help Dialogues. Human--Computer Interaction 14, 3: 283--315.

Digital Library

[29]

Seungwon Kim, Gun A. Lee, and Nobuchika Sakata. 2013. Comparing pointing and drawing for remote collaboration. 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). https://doi.org/10.1109/ismar.2013.6671833

[30]

David Kirk, Tom Rodden, and Danaë Stanton Fraser. 2007. Turn it this way. Proceedings of the SIGCHI conference on Human Factors in Computing Systems - CHI '07. https://doi.org/10.1145/1240624.1240782

Digital Library

[31]

David S. Kirk and Danaë Stanton Fraser. 2005. The effects of remote gesturing on distance instruction. Proceedings of the 2005 Conference on Computer Support for Collaborative Learning 2005: the next 10 years! - CSCL '05. https://doi.org/10.3115/1149293.1149332

[32]

Sotaro Kita. 2003. Pointing: Where Language, Culture, and Cognition Meet. Psychology Press.

[33]

Hideaki Kuzuoka, Toshio Kosuge, and Masatomo Tanaka. 1994. GestureCam. Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work - CSCW '94. https://doi.org/10.1145/192844.192866

[34]

Hideaki Kuzuoka, Jun Yamashita, Keiichi Yamazaki, and Akiko Yamazaki. 1999. Agora. CHI '99 extended abstracts on Human Factors in Computer Systems - CHI '99. https://doi.org/10.1145/632835.632836

[35]

Jiann-Shu Lee, Ching-Tsorng Tsai, Chen-Hsing Pen, and Hui-Chieh Lu. 2003. A real time collaboration system for teleradiology consultation. International Journal of Medical Informatics 72, 1--3: 73--79.

[36]

Paul Luff, Christian Heath, Hideaki Kuzuoka, Keiichi Yamazaki, and Jun Yamashita. 2006. Handling documents and discriminating objects in hybrid spaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '06. https://doi.org/10.1145/1124772.1124858

Digital Library

[37]

Helena M. Mentis, Amine Chellali, and Steven Schwaitzberg. 2014. Learning to see the body. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems - CHI '14. https://doi.org/10.1145/2556288.2557387

Digital Library

[38]

Helena M. Mentis, Kenton O'Hara, Gerardo Gonzalez, Abigail Sellen, Robert Corish, Antonio Criminisi, Rikin Trivedi, and Pierre Theodore. 2015. Voice or Gesture in the Operating Room. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '15. https://doi.org/10.1145/2702613.2702963

Digital Library

[39]

Helena M. Mentis, Ahmed Rahim, and Pierre Theodore. 2016. Crafting the Image in Surgical Telemedicine. Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing - CSCW '16. https://doi.org/10.1145/2818048.2819978

Digital Library

[40]

Microsoft Corporation. Remote Assist | Microsoft Dynamics 365. Retrieved March 31, 2019 from https://dynamics.microsoft.com/en-us/mixed-reality/remote-assist/

[41]

Ohan Oda, Carmine Elvezio, Mengu Sukan, Steven Feiner, and Barbara Tversky. 2015. Virtual Replicas for Remote Assistance in Virtual and Augmented Reality. Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology - UIST '15. https://doi.org/10.1145/2807442.2807497

Digital Library

[42]

Kenton O'Hara, Gerardo Gonzalez, Graeme Penney, Abigail Sellen, Robert Corish, Helena Mentis, Andreas Varnavas, Antonio Criminisi, Mark Rouncefield, Neville Dastur, and Tom Carrell. 2014. Interactional Order and Constructed Ways of Seeing with Touchless Imaging Systems in Surgery. Computer Supported Cooperative Work: CSCW: an International Journal 23, 3: 299--337.

[43]

Shoichi Ohta, Hideaki Kuzuoka, Mariko Noda, Hirokazu Sasaki, Shiro Mishima, Tadashi Fujikawa, and Tetsuo Yukioka. 2006. Remote support for emergency medicine using a remote-control laser pointer. Journal of Telemedicine and Telecare 12, 1: 44--48.

[44]

Noritaka Osawa. 2006. Aggregate pointers to support large group collaboration using telepointers. CHI '06 Extended Abstracts on Human Factors in Computing Systems - CHI EA '06. https://doi.org/10.1145/1125451.1125671

Digital Library

[45]

Jiazhi Ou, Susan R. Fussell, Xilin Chen, Leslie D. Setlock, and Jie Yang. 2003. Gestural communication over video stream. Proceedings of the 5th International Conference on Multimodal Iinterfaces - ICMI '03. https://doi.org/10.1145/958468.958477

Digital Library

[46]

Todd A. Ponsky, Iuliana D. Bobanga, Marc Schwachter, Theodore H. Stathos, Michael Rosen, Robert Parry, Margaret Nalugo, and Steven S. Rothenberg. 2014. Transcontinental telementoring with pediatric surgeons: proof of concept and technical considerations. Journal of Laparoendoscopic & Advanced Surgical Techniques. Part A 24, 12: 892--896.

[47]

Steven S. Rothenberg, Suzanne Yoder, Saundra Kay, and Todd Ponsky. 2009. Initial experience with surgical telementoring in pediatric laparoscopic surgery using remote presence technology. Journal of Laparoendoscopic & Advanced Surgical Techniques. Part A 19 Suppl 1: S219--22.

[48]

Christopher M. Schlachta, Ninh T. Nguyen, Todd Ponsky, and Brian Dunkin. 2016. Project 6 Summit: SAGES telementoring initiative. Surgical endoscopy 30, 9: 3665--3672.

[49]

Azin Semsar, Hannah McGowan, Yuanyuan Feng, Hamid R. Zahiri, Ivan M. George, Timothy Turner, Adrian Park, Helena M. Mentis, and Andrea Kleinsmith. 2019. Effects of a Virtual Pointer on Trainees' Cognitive Load and Communication Efficiency in Surgical Training. AMIA Annual Symposium Proceedings 2019. American Medical Informatics Association.

[50]

Duncan Roderick Stevenson. 2011. Tertiary-Level Telehealth: A Media Space Application. Computer Supported Cooperative Work (CSCW) 20, 61--92. https://doi.org/10.1007/s10606-010--9125--8

Digital Library

[51]

Barry A. Turner. 1978. Man-made Disasters. Crane Russak, Incorporated.

[52]

Nirit Yuviler-Gavish, Eldad Yechiam, and Arava Kallai. 2011. Learning in multimodal training: Visual guidance can be both appealing and disadvantageous in spatial tasks. International Journal of Human-Computer Studies 69, 113--122. https://doi.org/10.1016/j.ijhcs.2010.11.005

Digital Library

[53]

2019. New AI, mixed reality business solutions lead the way for Microsoft Dynamics 365 - Microsoft Dynamics 365. Microsoft Dynamics 365. Retrieved March 31, 2019 from https://cloudblogs.microsoft.com/dynamics365/2019/02/21/new-ai-mixed-reality-business-solutions-lead-the-way-for-microsoft-dynamics-365/

Cited By

Jiménez-Ruescas JCeldrán FSalazar LSánchez-Margallo JSánchez-Margallo FGonzález P(2024)Assessing the Use of Virtual Pointers in Laparoscopic Surgery Training ActivitiesSurgical Innovation10.1177/15533506241292854Online publication date: 5-Nov-2024
https://doi.org/10.1177/15533506241292854
Lambert SVoros STroccaz JCanlorbe GAvellino I(2024)Understanding The Needs of Mentoring in Surgery to Guide the Design of Surgical Telementoring SystemsProceedings of the ACM on Human-Computer Interaction10.1145/36374208:CSCW1(1-28)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637420
Wang JTang HKantor TSoltani TPopov VWang X(2024)Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery LearningProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642587(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642587
Show More Cited By

Index Terms

How Trainees Use the Information from Telepointers in Remote Instruction
1. Human-centered computing
  1. Collaborative and social computing
    1. Empirical studies in collaborative and social computing

Recommendations

ARTEMIS: A Collaborative Mixed-Reality System for Immersive Surgical Telementoring
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Traumatic injuries require timely intervention, but medical expertise is not always available at the patient’s location. Despite recent advances in telecommunications, surgeons still have limited tools to remotely help inexperienced surgeons. Mixed ...
Miniature in vivo Robots for Remote and Harsh Environments

Long-term human space exploration will require contingencies for emergency medical procedures including some capability to perform surgery. The ability to perform minimally invasive surgery (MIS) would be an important capability. The use of small ...
Depth cues in augmented reality for training of robot-assisted minimally invasive surgery
AcademicMindtrek '17: Proceedings of the 21st International Academic Mindtrek Conference

Training of robot-assisted minimally invasive surgery often includes supervised practice with a robotic surgical system. In this case, augmented reality can improve the communication between instructor and trainee, for example, by allowing the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 3, Issue CSCW

November 2019

5026 pages

EISSN:2573-0142

DOI:10.1145/3371885

Editors:
Airi Lampinen
Stockholm University, Sweden
,
Darren Gergle
Northwestern University, USA
,
David A. Shamma
FXPAL, USA

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2019

Published in PACMHCI Volume 3, Issue CSCW

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
461
Total Downloads

Downloads (Last 12 months)96
Downloads (Last 6 weeks)7

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jiménez-Ruescas JCeldrán FSalazar LSánchez-Margallo JSánchez-Margallo FGonzález P(2024)Assessing the Use of Virtual Pointers in Laparoscopic Surgery Training ActivitiesSurgical Innovation10.1177/15533506241292854Online publication date: 5-Nov-2024
https://doi.org/10.1177/15533506241292854
Lambert SVoros STroccaz JCanlorbe GAvellino I(2024)Understanding The Needs of Mentoring in Surgery to Guide the Design of Surgical Telementoring SystemsProceedings of the ACM on Human-Computer Interaction10.1145/36374208:CSCW1(1-28)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637420
Wang JTang HKantor TSoltani TPopov VWang X(2024)Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery LearningProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642587(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642587
Lambert SVoros SCanlorbe GTroccaz JAvellino I(2024)Understanding Takeovers and Telestration in Laparoscopic Surgery to Inform Telementoring System DesignProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641978(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641978
Popov VChen XWang JKemp MSandhu GKantor TMateju NWang X(2024)Looking Together ≠ Seeing the Same Thing: Understanding Surgeons' Visual Needs During Intra-operative Coordination and InstructionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641929(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641929
Yang SPoonawala AJiang TSchneider B(2023)Can Synchronous Code Editing and Awareness Tools Support Remote Tutoring? Effects on Learning and TeachingProceedings of the ACM on Human-Computer Interaction10.1145/36101777:CSCW2(1-30)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610177
Maria SMentis HCanlorbe GAvellino I(2023)Supporting Collaborative Discussions In Surgical Teleconsulting Through Augmented Reality Head Mounted DisplaysProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580714(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580714
Choi HKwon JNam S(2023)Research on the application of gaze visualization interface on virtual reality training systemsJournal on Multimodal User Interfaces10.1007/s12193-023-00409-617:3(203-211)Online publication date: 18-Aug-2023
https://doi.org/10.1007/s12193-023-00409-6
Gasques DJohnson JSharkey TFeng YWang RXu ZZavala EZhang YXie WZhang XDavis KYip MWeibel NKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)ARTEMIS: A Collaborative Mixed-Reality System for Immersive Surgical TelementoringProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445576(1-14)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445576

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents