poster

Embodied Notes: A Cognitive Support Tool For Remote Scientific Collaboration in VR

Authors:

Monsurat Olaosebikan,

Claudia Aranda Barrios,

Orit ShaerAuthors Info & Claims

CHI EA '22: CHI Conference on Human Factors in Computing Systems Extended Abstracts

Article No.: 279, Pages 1 - 6

https://doi.org/10.1145/3491101.3519664

Published: 28 April 2022 Publication History

Abstract

Scientists collaborate remotely across institutions, countries and continents. However, collaborating remotely is challenging. Video-conferencing tools used for meetings limit the cognitive practices that collaborators can partake in. In virtual reality (VR) users can gain back some of the spatial and social affordances present in collocated collaboration, as well as benefit from interactions that would not be possible in the real world. We introduce Embodied Notes: a cognitive support tool designed to be used in a collaborative virtual environment.

Supplementary Material

MP4 File (3491101.3519664-video-figure.mp4)

Video Figure

Download
270.70 MB

MP4 File (3491101.3519664-video-preview.mp4)

Video Preview

Download
68.08 MB

References

[1]

Mark S Ackerman, Brian Starr, Debby Hindus, and Scott D Mainwaring. 1997. Hanging on the ‘wire. ACM Trans. Comput. Hum. Interact. 4, 1 (March 1997), 39–66.

Digital Library

[2]

Jiban Adhikary and Keith Vertanen. 2021. Text Entry in Virtual Environments using Speech and a Midair Keyboard. IEEE Transactions on Visualization and Computer Graphics 27, 5(2021), 2648–2658.

[3]

Jeremy N. Bailenson. 2021. Nonverbal Overload: A Theoretical Argument for the Causes of Zoom Fatigue. Technology, Mind, and Behavior 2, 1 (2021).

[4]

Andrea Batch, Andrew Cunningham, Maxime Cordeil, Niklas Elmqvist, Tim Dwyer, Bruce H Thomas, and Kim Marriott. 2020. There Is No Spoon: Evaluating Performance, Space Use, and Presence with Expert Domain Users in Immersive Analytics. IEEE Trans. Vis. Comput. Graph. 26, 1 (Jan. 2020), 536–546.

[5]

L Michelle Bennett and Howard Gadlin. 2012. Collaboration and team science: from theory to practice. Journal of investigative medicine 60, 5 (2012), 768–775.

[6]

Jennifer Boyett, Darlla Duniphin, Frederick Miller, Mary Moon, and Bruna Varalli-Claypool. 2021. The Code Breaker: Jennifer Doudna, Gene Editing, and the Future of the Human Race.

[7]

Virginia Braun and Victoria Clarke. 2012. Thematic analysis. American Psychological Association, Washington, DC, 57––71.

[8]

Tom Chandler, Maxime Cordeil, Tobias Czauderna, Tim Dwyer, Jaroslaw Glowacki, Cagatay Goncu, Matthias Klapperstueck, Karsten Klein, Kim Marriott, Falk Schreiber, and Elliot Wilson. 2015. Immersive Analytics. In 2015 Big Data Visual Analytics (BDVA). IEEE, Hobart, TAS, Australia, 1–8.

[9]

Yi-Ting Chen, Chi-Hsuan Hsu, Chih-Han Chung, Yu-Shuen Wang, and Sabarish V. Babu. 2019. iVRNote: Design, Creation and Evaluation of an Interactive Note-Taking Interface for Study and Reflection in VR Learning Environments. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Osaka, Japan, 172–180. https://doi.org/10.1109/VR.2019.8798338

[10]

Andy Clark and David Chalmers. 1998. The Extended Mind. Analysis 58, 1 (Jan. 1998), 7–19.

[11]

Maxime Cordeil, Andrew Cunningham, Tim Dwyer, Bruce H Thomas, and Kim Marriott. 2017. ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST ’17). Association for Computing Machinery, New York, NY, USA, 71–83.

Digital Library

[12]

Maxime Cordeil, Tim Dwyer, Karsten Klein, Bireswar Laha, Kim Marriott, and Bruce H Thomas. 2017. Immersive Collaborative Analysis of Network Connectivity: CAVE-style or Head-Mounted Display?IEEE Trans. Vis. Comput. Graph. 23, 1 (Jan. 2017), 441–450.

Digital Library

[13]

Gillian Crampton Smith. 1995. The hand that rocks the cradle. ID magazine (1995), 60–65.

[14]

Ciro Donalek, S George Djorgovski, Alex Cioc, Anwell Wang, Jerry Zhang, Elizabeth Lawler, Stacy Yeh, Ashish Mahabal, Matthew Graham, Andrew Drake, 2014. Immersive and collaborative data visualization using virtual reality platforms. In 2014 IEEE International Conference on Big Data (Big Data). IEEE, Washington, DC, 609–614.

[15]

Rasmus Elmgren. 2017. Handwriting in VR as a Text Input Method.

[16]

J A W Filho, W Stuerzlinger, and L Nedel. 2020. Evaluating an Immersive Space-Time Cube Geovisualization for Intuitive Trajectory Data Exploration. IEEE Trans. Vis. Comput. Graph. 26, 1 (Jan. 2020), 514–524.

[17]

Arturo S García, Terrence Fernando, David J Roberts, Christian Bar, Michele Cencetti, Wito Engelke, and Andreas Gerndt. 2019. Collaborative virtual reality platform for visualizing space data and mission planning. Multimed. Tools Appl. 78, 23 (Dec. 2019), 33191–33220.

[18]

Keita Higuchi, Yinpeng Chen, Philip A Chou, Zhengyou Zhang, and Zicheng Liu. 2015. ImmerseBoard: Immersive Telepresence Experience using a Digital Whiteboard. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 2383–2392.

Digital Library

[19]

Robert J.K. Jacob, Audrey Girouard, Leanne M. Hirshfield, Michael S. Horn, Orit Shaer, Erin Treacy Solovey, and Jamie Zigelbaum. 2008. Reality-Based Interaction: A Framework for Post-WIMP Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI ’08). Association for Computing Machinery, New York, NY, USA, 201––210. https://doi.org/10.1145/1357054.1357089

Digital Library

[20]

Shunichi Kasahara and Jun Rekimoto. 2015. JackIn head: immersive visual telepresence system with omnidirectional wearable camera for remote collaboration. In Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology (Beijing, China) (VRST ’15). Association for Computing Machinery, New York, NY, USA, 217–225.

Digital Library

[21]

Florian Kern, Peter Kullmann, Elisabeth Ganal, Kristof Korwisi, Rene Stingl, Florian Niebling, and Marc Erich Latoschik. 2021. Off-The-Shelf Stylus: Using XR Devices for Handwriting and Sketching on Physically Aligned Virtual Surfaces. Frontiers in Virtual Reality 2 (2021), 69.

[22]

Andreas Knote, Sabine C. Fischer, Sylvain Cussat-Blanc, Florian Niebling, David Bernard, Florian Cogoni, and Sebastian von Mammen. 2019. Immersive Analysis of 3D Multi-cellular In-Vitro and In-Silico Cell Cultures. In 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). IEEE, San Diego, CA, 82–827. https://doi.org/10.1109/AIVR46125.2019.00021

[23]

Miriam K Konkel, Brygg Ullmer, Orit Shaer, Ali Mazalek, and Chris Branton. 2020. Toward tangibles and display-rich interfaces for co-located and distributed genomics collaborations.

[24]

Martin Kuechler and Andreas M Kunz. 2010. Collaboard: a remote collaboration groupware device featuring an embodiment-enriched shared workspace. In Proceedings of the 16th ACM international conference on Supporting group work (Sanibel Island, Florida, USA) (GROUP ’10). Association for Computing Machinery, New York, NY, USA, 211–214.

Digital Library

[25]

Joon Hyub Lee, Donghyeok Ma, Haena Cho, and Seok-Hyung Bae. 2021. Post-Post-it: A Spatial Ideation System in VR for Overcoming Limitations of Physical Post-it Notes. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–7.

[26]

Lee Lisle, Xiaoyu Chen, J K Edward Gitre, Chris North, and Doug A Bowman. 2020. Evaluating the Benefits of the Immersive Space to Think.

[27]

Kim Marriott, Falk Schreiber, Tim Dwyer, Karsten Klein, Nathalie Henry Riche, Takayuki Itoh, Wolfgang Stuerzlinger, and Bruce H Thomas. 2018. Immersive Analytics. Springer.

[28]

Sarah Morrison-Smith and Jaime Ruiz. 2020. Challenges and barriers in virtual teams: a literature review. SN Applied Sciences 2, 6 (May 2020), 1096.

[29]

James D Myers, Cedrus Fox-Dobbs, Justin Laird, Dai Le, Dan Reich, and Thad Curtz. 1996. Electronic laboratory notebooks for collaborative research. In Proceedings of WET ICE’96. IEEE 5th Workshop on Enabling Technologies; Infrastucture for Collaborative Enterprises. IEEE, Stanford, CA, 47–51.

[30]

Robby Nadler. 2020. Understanding “Zoom fatigue”: Theorizing spatial dynamics as third skins in computer-mediated communication. Computers and Composition 58 (2020), 102613.

[31]

Nanome. 2020. Nanome. https://nanome.ai/nanome/. Accessed: 2020-11-23.

[32]

David T Nguyen and John Canny. 2007. Multiview: improving trust in group video conferencing through spatial faithfulness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 1465–1474.

Digital Library

[33]

Normcore. 2021. Normcore: Seamless multiplayer for unity.https://normcore.io/

[34]

Judith S Olson and Gary M Olson. 2014. Bridging Distance: Empirical studies of distributed teams. In Human-Computer Interaction and Management Information Systems: Applications. Advances in Management Information Systems. Routledge, New York, NY, 117–134.

[35]

Walter Pauk and Ross JQ Owens. 2013. How to study in college. Cengage Learning, Boston, MA.

[36]

Annie Murphy Paul. 2021. The Extended Mind: The Power of Thinking Outside the Brain. Houghton Mifflin Harcourt, New York, NY.

[37]

Duc-Minh Pham and Wolfgang Stuerzlinger. 2019. HawKEY: Efficient and Versatile Text Entry for Virtual Reality. In 25th ACM Symposium on Virtual Reality Software and Technology (Parramatta, NSW, Australia) (VRST ’19). Association for Computing Machinery, New York, NY, USA, Article 21, 11 pages. https://doi.org/10.1145/3359996.3364265

Digital Library

[38]

Ivan Poupyrev, Numada Tomokazu, and Suzanne Weghorst. 1998. Virtual Notepad: handwriting in immersive VR. In Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No. 98CB36180). IEEE, Atlanta, GA, 126–132.

[39]

Christine Ripken, Sebastian Tusk, and Christian Tominski. 2021. Immersive analytics of heterogeneous biological data informed through need-finding interviews.

[40]

Evan F Risko and Sam J Gilbert. 2016. Cognitive offloading. Trends in cognitive sciences 20, 9 (2016), 676–688.

[41]

David Rose. 2021. Supersight: What augmented reality means for our lives, our work, and the way we imagine the future. BenBella Books, Inc., Dallas, TX.

[42]

Kimiko Ryokai, Hayes Raffle, and Andy Brooks. 2009. Tangible Message Bubbles for Children’s Communication and Play. In CHI ’09 Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA) (CHI EA ’09). Association for Computing Machinery, New York, NY, USA, 4597––4602. https://doi.org/10.1145/1520340.1520706

[43]

Orit Shaer, Consuelo Valdes, Sirui Liu, Kara Lu, Kimberly Chang, Wendy Xu, Traci L Haddock, Swapnil Bhatia, Douglas Densmore, and Robert Kincaid. 2014. Designing reality-based interfaces for experiential bio-design. Personal and Ubiquitous Computing 18, 6 (2014), 1515–1532.

Digital Library

[44]

Spatial Systems. 2021. Spatial - metaverse spaces that bring us together. http://www.spatial.io. Accessed: 2021-12-15.

[45]

Facebook Technologies. 2022. Horizon Workrooms: VR for business meetings. https://www.oculus.com/workrooms Accessed: 2021-12-15.

Cited By

Luo W(2024)Exploring Spatial Organization Strategies for Virtual Content in Mixed Reality EnvironmentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638181(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3638181
Lisle LDavidson KGitre ENorth CBowman D(2023)Different realities: a comparison of augmented and virtual reality for the sensemaking processFrontiers in Virtual Reality10.3389/frvir.2023.11778554Online publication date: 9-Aug-2023
https://doi.org/10.3389/frvir.2023.1177855
Tahmid ILisle LDavidson KNorth CBowman D(2022)Evaluating the Benefits of Explicit and Semi-Automated Clusters for Immersive Sensemaking2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR55827.2022.00064(479-488)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR55827.2022.00064

Recommendations

Identifying Cognitive and Creative Support Needs for Remote Scientific Collaboration using VR: Practices, Affordances, and Design Implications
C&C '22: Proceedings of the 14th Conference on Creativity and Cognition

Remote scientific collaborations have been pivotal in generating scientific discoveries and breakthroughs that accelerate research in many fields. Emerging VR applications for remote work, which utilize commercially available head-mounted displays (...
BeHere: a VR/SAR remote collaboration system based on virtual replicas sharing gesture and avatar in a procedural task
Abstract
In this paper, we focus on the significance of remote collaboration using virtual replicas, avatar, and gesture on a procedural task in industry; thus, we present a Virtual Reality (VR)/Spatial Augmented Reality (SAR) remote collaboration system, ...
Mixed Reality Remote Collaboration Combining 360 Video and 3D Reconstruction
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Remote Collaboration using Virtual Reality (VR) and Augmented Reality (AR) has recently become a popular way for people from different places to work together. Local workers can collaborate with remote helpers by sharing 360-degree live video or 3D ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

3066 pages

ISBN:9781450391566

DOI:10.1145/3491101

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster
Research
Refereed limited

Funding Sources

NSF

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
448
Total Downloads

Downloads (Last 12 months)108
Downloads (Last 6 weeks)2

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Luo W(2024)Exploring Spatial Organization Strategies for Virtual Content in Mixed Reality EnvironmentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638181(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3638181
Lisle LDavidson KGitre ENorth CBowman D(2023)Different realities: a comparison of augmented and virtual reality for the sensemaking processFrontiers in Virtual Reality10.3389/frvir.2023.11778554Online publication date: 9-Aug-2023
https://doi.org/10.3389/frvir.2023.1177855
Tahmid ILisle LDavidson KNorth CBowman D(2022)Evaluating the Benefits of Explicit and Semi-Automated Clusters for Immersive Sensemaking2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR55827.2022.00064(479-488)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR55827.2022.00064

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents