Reality and Beyond: Proxemics as a Lens for Designing Handheld Collaborative Augmented Reality
Authors: 
Mille Skovhus Lunding, 
Jens Emil Sloth Grønbæk, Nicolai Grymer, Thomas Wells, 
Steven Houben, Marianne Graves PetersenAuthors Info & Claims
Article No.: 427, Pages 21 - 40
Abstract
Augmented Reality (AR) has shown great potential for supporting co-located collaboration. Yet, it is rarely articulated in the design rationales of AR systems that they promote a certain socio-spatial configuration of the users. Learning from proxemics, we argue that such configurations enable and constrain different co-located spatial behaviors with consequences for collaborative activities. We focus specifically on enabling different collaboration styles via the design of Handheld Collaborative Augmented Reality (HCAR) systems. Drawing upon notions of proxemics, we show how different HCAR designs enable different socio-spatial configurations. Through a design exploration, we demonstrate interaction techniques to expand on the notion of collaborative coupling styles by either deliberately designing for aligning with physical reality or going beyond. The main contributions are a proxemics-based conceptual lens and vocabulary for supporting interaction designers in being mindful of the proxemic consequences when developing handheld multi-user AR systems.
Supplementary Material
This is a presentation video of our work accepted at ISS 2023; "Reality and Beyond: Proxemics as a Lens for Designing Handheld Collaborative Augmented Reality". This main contribution of this work is; 1) a conceptual lens, which establishes a common vocabulary for comparison and articulation of collaborative AR systems in terms of their proxemic properties, 2) A broadening of the design space of interaction techniques for co-located collaborative AR, and 3) A discussion aimed to develop a nuanced socio-spatial literacy for understanding trade-offs when designing handheld AR interaction techniques for co-located settings.
- Download
- 257.73 MB
References
[1]
Apple. [n.d.]. Multipeer Connectivity | Apple Developer Documentation. https://developer.apple.com/documentation/ multipeerconnectivity. (Accessed on 10/02/ 2023 ).
[2]
Till Ballendat, Nicolai Marquardt, and Saul Greenberg. 2010. Proxemic Interaction: Designing for a Proximity and Orientation-Aware Environment. In ACM International Conference on Interactive Tabletops and Surfaces (Saarbrücken, Germany) (ITS '10). Association for Computing Machinery, New York, NY, USA, 121-130. https://doi.org/10.1145/ 1936652.1936676
[3]
Mark Billinghurst and Hirokazu Kato. 2002. Collaborative augmented reality. Commun. ACM 45, 7 ( 2002 ), 64-70.
[4]
Frederik Brudy, Christian Holz, Roman Rädle, Chi-Jui Wu, Steven Houben, Clemens Nylandsted Klokmose, and Nicolai Marquardt. 2019. Cross-Device Taxonomy: Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1-28. https://doi.org/10.1145/3290605. 3300792
[5]
Henry Chen, Austin S. Lee, Mark Swift, and John C. Tang. 2015. 3D Collaboration Method over HoloLens™ and Skype™ End Points. In Proceedings of the 3rd International Workshop on Immersive Media Experiences (Brisbane, Australia) ( ImmersiveME '15). Association for Computing Machinery, New York, NY, USA, 27-30. https://doi.org/10.1145/2814347. 2814350
[6]
Ella Dagan, Ana María Cárdenas Gasca, Ava Robinson, Anwar Noriega, Yu Jiang Tham, Rajan Vaish, and Andrés Monroy-Hernández. 2022. Project IRL : Playful Co-Located Interactions with Mobile Augmented Reality. Proc. ACM Hum.-Comput. Interact. 6, CSCW1, Article 62 (apr 2022 ), 27 pages. https://doi.org/10.1145/3512909
[7]
Florian Daiber, Felix Kosmalla, and Antonio Krüger. 2013. BouldAR: Using Augmented Reality to Support Collaborative Boulder Training. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (Paris, France) (CHI EA ' 13 ). Association for Computing Machinery, New York, NY, USA, 949-954. https://doi.org/10.1145/2468356.2468526
[8]
Ryan Anthony J de Belen, Huyen Nguyen, Daniel Filonik, Dennis Del Favero, and Tomasz Bednarz. 2019. A systematic review of the current state of collaborative mixed reality technologies: 2013-2018. AIMS Electronics and Electrical Engineering 3, 2 ( 2019 ), 181.
[9]
Barrett Ens, Joel Lanir, Anthony Tang, Scott Bateman, Gun Lee, Thammathip Piumsomboon, and Mark Billinghurst. 2019. Revisiting collaboration through mixed reality: The evolution of groupware. International Journal of HumanComputer Studies 131 ( 2019 ), 81-98.
[10]
Anna Fuste and Chris Schmandt. 2019. HyperCubes: A Playful Introduction to Computational Thinking in Augmented Reality. In Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (Barcelona, Spain) (CHI PLAY '19 Extended Abstracts). Association for Computing Machinery, New York, NY, USA, 379-387. https://doi.org/10.1145/3341215.3356264
[11]
J. Gehl and R. Rogers. 2013. Cities for People. Island Press. https://books.google.dk/books?id=kqtwtr3QhkIC
[12]
Zeinab Ghaemi, Ulrich Engelke, Barrett Ens, and Bernhard Jenny. 2022. Proxemic maps for immersive visualization. Cartography and Geographic Information Science 49, 3 ( 2022 ), 205-219. https://doi.org/10.1080/15230406. 2021. 2013946 arXiv:https://doi.org/10.1080/15230406. 2021.2013946
[13]
Saul Greenberg, Nicolai Marquardt, Till Ballendat, Rob Diaz-Marino, and Miaosen Wang. 2011. Proxemic Interactions: The New Ubicomp? Interactions 18, 1 (Jan. 2011 ), 42-50. https://doi.org/10.1145/1897239.1897250
[14]
Jens Emil Grønbaek, Mille Skovhus Knudsen, Kenton O'Hara, Peter Gall Krogh, Jo Vermeulen, and Marianne Graves Petersen. 2020. Proxemics Beyond Proximity: Designing for Flexible Social Interaction Through Cross-Device Interaction. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) ( CHI '20). Association for Computing Machinery, New York, NY, USA, 1-14. https://doi.org/10.1145/3313831.3376379
[15]
Jens Emil Grønbaek, Henrik Korsgaard, Marianne Graves Petersen, Morten Henriksen Birk, and Peter Gall Krogh. 2017. Proxemic Transitions: Designing Shape-Changing Furniture for Informal Meetings. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) ( CHI '17). Association for Computing Machinery, New York, NY, USA, 7029-7041. https://doi.org/10.1145/3025453.3025487
[16]
Jens Emil Grønbaek, Christine Linding, Anders Kromann, Thomas Fly Hylddal Jensen, and Marianne Graves Petersen. 2019. Proxemics Play: Exploring the Interplay between Mobile Devices and Interiors. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion (San Diego, CA, USA) ( DIS '19 Companion). Association for Computing Machinery, New York, NY, USA, 177-181. https://doi.org/10.1145/3301019.3323886
[17]
Anhong Guo, Ilter Canberk, Hannah Murphy, Andrés Monroy-Hernández, and Rajan Vaish. 2019. Blocks: Collaborative and Persistent Augmented Reality Experiences. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 3, Article 83 (sep 2019 ), 24 pages. https://doi.org/10.1145/3351241
[18]
Edward T. Hall. 1966. The Hidden Dimension.
[19]
Jim Hollan and Scott Stornetta. 1992. Beyond Being There. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Monterey, California, USA) ( CHI '92). Association for Computing Machinery, New York, NY, USA, 119-125. https://doi.org/10.1145/142750.142769
[20]
Ann Huang, Pascal Knierim, Francesco Chiossi, Lewis L Chuang, and Robin Welsch. 2022. Proxemics for Human-Agent Interaction in Augmented Reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) ( CHI '22). Association for Computing Machinery, New York, NY, USA, Article 421, 13 pages. https://doi.org/10.1145/3491102.3517593
[21]
Sebastian Hubenschmid, Johannes Zagermann, Simon Butscher, and Harald Reiterer. 2021. STREAM: Exploring the Combination of Spatially-Aware Tablets with Augmented Reality Head-Mounted Displays for Immersive Analytics. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 469, 14 pages. https://doi.org/10.1145/3411764.3445298
[22]
Duy-Nguyen Ta Huynh, Karthik Raveendran, Yan Xu, Kimberly Spreen, and Blair MacIntyre. 2009. Art of Defense: A Collaborative Handheld Augmented Reality Board Game. In Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games (New Orleans, Louisiana) (Sandbox '09). Association for Computing Machinery, New York, NY, USA, 135-142. https://doi.org/10.1145/1581073.1581095
[23]
Katherine Isbister. 2016. Connecting through Play. Interactions 23, 4 ( June 2016 ), 26-33. https://doi.org/10.1145/2946999
[24]
Katherine Isbister, Elena Márquez Segura, and Edward F. Melcer. 2018. Social Afordances at Play: Game Design Toward Socio-Technical Innovation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-10. https: //doi.org/10.1145/3173574.3173946
[25]
Petra Isenberg, Danyel Fisher, Sharoda A. Paul, Meredith Ringel Morris, Kori Inkpen, and Mary Czerwinski. 2012. Co-Located Collaborative Visual Analytics around a Tabletop Display. IEEE Transactions on Visualization and Computer Graphics 18, 5 ( 2012 ), 689-702. https://doi.org/10.1109/TVCG. 2011.287
[26]
Shunichi Kasahara, Valentin Heun, Austin S. Lee, and Hiroshi Ishii. 2012. Second Surface: Multi-User Spatial Collaboration System Based on Augmented Reality. In SIGGRAPH Asia 2012 Emerging Technologies (Singapore, Singapore) (SA '12). Association for Computing Machinery, New York, NY, USA, 1-4. https://doi.org/10.1145/2407707.2407727
[27]
Hannes Kaufmann and Dieter Schmalstieg. 2002. Mathematics and Geometry Education with Collaborative Augmented Reality. In ACM SIGGRAPH 2002 Conference Abstracts and Applications (San Antonio, Texas) ( SIGGRAPH '02). Association for Computing Machinery, New York, NY, USA, 37-41. https://doi.org/10.1145/1242073.1242086
[28]
Adam Kendon. 1990. Conducting interaction: Patterns of behavior in focused encounters. Vol. 7. CUP Archive.
[29]
Peter Gall Krogh, Marianne Graves Petersen, Kenton O'Hara, and Jens Emil Groenbaek. 2017. Sensitizing Concepts for Socio-Spatial Literacy in HCI. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) ( CHI '17). Association for Computing Machinery, New York, NY, USA, 6449-6460. https: //doi.org/10.1145/3025453.3025756
[30]
David Ledo, Steven Houben, Jo Vermeulen, Nicolai Marquardt, Lora Oehlberg, and Saul Greenberg. 2018. Evaluation Strategies for HCI Toolkit Research. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-17. https: //doi.org/10.1145/3173574.3173610
[31]
Laura Malinverni, Cristina Valero, Marie-Monique Schaper, and Narcis Pares. 2018. A Conceptual Framework to Compare Two Paradigms of Augmented and Mixed Reality Experiences. In Proceedings of the 17th ACM Conference on Interaction Design and Children (Trondheim, Norway) (IDC '18). Association for Computing Machinery, New York, NY, USA, 7-18. https://doi.org/10.1145/3202185.3202750
[32]
Nicolai Marquardt, Robert Diaz-Marino, Sebastian Boring, and Saul Greenberg. 2011. The Proximity Toolkit: Prototyping Proxemic Interactions in Ubiquitous Computing Ecologies. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) ( UIST '11). Association for Computing Machinery, New York, NY, USA, 315-326. https://doi.org/10.1145/2047196.2047238
[33]
Nicolai Marquardt, Ken Hinckley, and Saul Greenberg. 2012. Cross-Device Interaction via Micro-Mobility and fFormations. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (Cambridge, Massachusetts, USA) ( UIST '12). Association for Computing Machinery, New York, NY, USA, 13-22. https://doi.org/10. 1145/2380116.2380121
[34]
Paul Marshall, Yvonne Rogers, and Nadia Pantidi. 2011. Using F-Formations to Analyse Spatial Patterns of Interaction in Physical Environments. In Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work (Hangzhou, China) (CSCW '11). Association for Computing Machinery, New York, NY, USA, 445-454. https://doi.org/10.1145/ 1958824.1958893
[35]
Helena M. Mentis, Kenton O'Hara, Abigail Sellen, and Rikin Trivedi. 2012. Interaction Proxemics and Image Use in Neurosurgery. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) ( CHI '12). Association for Computing Machinery, New York, NY, USA, 927-936. https://doi.org/10.1145/2207676.2208536
[36]
Timothy Merritt, Christine Linding Nielsen, Frederik Lund Jakobsen, and Jens Emil Grønbaek. 2017. GlowPhones: Designing for Proxemics Play with Low-Resolution Displays in Location-Based Games. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Amsterdam, The Netherlands) (CHI PLAY '17). Association for Computing Machinery, New York, NY, USA, 69-81. https://doi.org/10.1145/3116595.3116598
[37]
Florian Mueller, Sophie Stellmach, Saul Greenberg, Andreas Dippon, Susanne Boll, Jayden Garner, Rohit Khot, Amani Naseem, and David Altimira. 2014. Proxemics Play: Understanding Proxemics for Designing Digital Play Experiences. In Proceedings of the 2014 Conference on Designing Interactive Systems (Vancouver, BC, Canada) ( DIS '14). Association for Computing Machinery, New York, NY, USA, 533-542. https://doi.org/10.1145/2598510.2598532
[38]
Nahal Norouzi, Kangsoo Kim, Myungho Lee, Ryan Schubert, Austin Erickson, Jeremy Bailenson, Gerd Bruder, and Greg Welch. 2019. Walking Your Virtual Dog: Analysis of Awareness and Proxemics with Simulated Support Animals in Augmented Reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 157-168. https://doi.org/10.1109/ISMAR. 2019. 000-8
[39]
Lev Poretski Poretski, Joel Lanir, Ram Margalit, and Ofer Arazy. 2021. Physicality As an Anchor for Coordination: Examining Collocated Collaboration in Physical and Mobile Augmented Reality Settings. Proc. ACM Hum.-Comput. Interact. 5, CSCW2, Article 470 (oct 2021 ), 29 pages. https://doi.org/10.1145/3479857
[40]
G. Reitmayr and D. Schmalstieg. 2001. Mobile collaborative augmented reality. In Proceedings IEEE and ACM International Symposium on Augmented Reality. 114-123. https://doi.org/10.1109/ISAR. 2001.970521
[41]
Pratiti Sarkar, Kapil Kadam, and Jayesh S Pillai. 2020. Learners' approaches, motivation and patterns of problem-solving on lines and angles in geometry using augmented reality. Smart Learning Environments 7, 1 ( 2020 ), 1-23.
[42]
Mickael Sereno, Xiyao Wang, Lonni Besançon, Michael J. McGufin, and Tobias Isenberg. 2022. Collaborative Work in Augmented Reality: A Survey. IEEE Transactions on Visualization and Computer Graphics 28, 6 ( 2022 ), 2530-2549. https://doi.org/10.1109/TVCG. 2020.3032761
[43]
Joon Gi Shin, Gary Ng, and Daniel Saakes. 2018. Couples Designing Their Living Room Together: A Study with Collaborative Handheld Augmented Reality. In Proceedings of the 9th Augmented Human International Conference (Seoul, Republic of Korea) (AH '18). Association for Computing Machinery, New York, NY, USA, Article 3, 9 pages. https://doi.org/10.1145/3174910.3174930
[44]
Anthony Tang, Melanie Tory, Barry Po, Petra Neumann, and Sheelagh Carpendale. 2006. Collaborative Coupling over Tabletop Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montréal, Québec, Canada) ( CHI '06). Association for Computing Machinery, New York, NY, USA, 1181-1190. https://doi.org/10.1145/ 1124772.1124950
[45]
Lili Tong, Audrey Serna, Simon Pageaud, Sébastien George, and Aurélien Tabard. 2016. It's Not How You Stand, It's How You Move: F-Formations and Collaboration Dynamics in a Mobile Learning Game. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (Florence, Italy) ( MobileHCI '16). Association for Computing Machinery, New York, NY, USA, 318-329. https://doi.org/10.1145/2935334.2935343
[46]
Unity. [n.d.]. About AR Foundation | AR Foundation | 4.0.12. https://docs.unity3d.com/Packages/com.unity.xr. [email protected]/manual/index.html. (Accessed on 10/02/ 2023 ).
[47]
Almar van der Stappen, Yunjie Liu, Jiangxue Xu, Xiaoyu Yu, Jingya Li, and Erik D. van der Spek. 2019. MathBuilder: A Collaborative AR Math Game for Elementary School Students. In Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (Barcelona, Spain) (CHI PLAY '19 Extended Abstracts). Association for Computing Machinery, New York, NY, USA, 731-738. https://doi.org/10.1145/3341215.3356295
[48]
Ana Villanueva, Zhengzhe Zhu, Ziyi Liu, Kylie Peppler, Thomas Redick, and Karthik Ramani. 2020. Meta-AR-App: An Authoring Platform for Collaborative Augmented Reality in STEM Classrooms. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) ( CHI '20). Association for Computing Machinery, New York, NY, USA, 1-14. https://doi.org/10.1145/3313831.3376146
[49]
Peng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Xiangyu Zhang, Shuxia Wang, Weiping He, Yuxiang Yan, and Hongyu Ji. 2021. AR/MR Remote Collaboration on Physical Tasks: A Review. Robotics and Computer-Integrated Manufacturing 72 ( 2021 ), 102071. https://doi.org/10.1016/j.rcim. 2020.102071
[50]
Thomas Wells and Steven Houben. 2020. CollabAR-Investigating the Mediating Role of Mobile AR Interfaces on Co-Located Group Collaboration. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) ( CHI '20). Association for Computing Machinery, New York, NY, USA, 1-13. https: //doi.org/10.1145/3313831.3376541
[51]
Hui Ye and Hongbo Fu. 2022. ProGesAR: Mobile AR Prototyping for Proxemic and Gestural Interactions with RealWorld IoT Enhanced Spaces. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) ( CHI '22). Association for Computing Machinery, New York, NY, USA, Article 130, 14 pages. https://doi.org/10.1145/3491102.3517689
[52]
Cik Suhaimi Yusof, Tian Sheng Low, Ajune Wanis Ismail, and Mohd Shahrizal Sunar. 2019. Collaborative Augmented Reality for Chess Game in Handheld Devices. In 2019 IEEE Conference on Graphics and Media (GAME). IEEE, 32-37. https://doi.org/10.1109/GAME47560. 2019.8980979
Index Terms
- Reality and Beyond: Proxemics as a Lens for Designing Handheld Collaborative Augmented Reality
Recommendations
Collaborative Augmented Reality: Multi-user Interaction in Urban Simulation
IVIC '09: Proceedings of the 1st International Visual Informatics Conference on Visual Informatics: Bridging Research and PracticeAugmented reality (AR) environment allows user or multi-user to interact with 2D and 3D data. AR simply can provide a collaborative interactive AR environment for urban simulation, where users can interact naturally and intuitively. AR collaboration ...
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented RealityAugmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Collaboration in Augmented Reality
Augmented Reality (AR) is a technology that allows users to view and interact in real time with virtual images seamlessly superimposed over the real world. AR systems can be used to create unique collaborative experiences. For example, co-located users ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
December 2023
482 pages
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 November 2023
Published in PACMHCI Volume 7, Issue ISS
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 388Total Downloads
- Downloads (Last 12 months)388
- Downloads (Last 6 weeks)18
Reflects downloads up to 22 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in