research-article

Open access

Understanding and Designing Avatar Biosignal Visualizations for Social Virtual Reality Entertainment

Authors:

Abdallah El Ali,

Maarten Wijntjes,

Pablo CesarAuthors Info & Claims

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Article No.: 425, Pages 1 - 15

https://doi.org/10.1145/3491102.3517451

Published: 29 April 2022 Publication History

All formats PDF

Abstract

Visualizing biosignals can be important for social Virtual Reality (VR), where avatar non-verbal cues are missing. While several biosignal representations exist, designing effective visualizations and understanding user perceptions within social VR entertainment remains unclear. We adopt a mixed-methods approach to design biosignals for social VR entertainment. Using survey (N=54), context-mapping (N=6), and co-design (N=6) methods, we derive four visualizations. We then ran a within-subjects study (N=32) in a virtual jazz-bar to investigate how heart rate (HR) and breathing rate (BR) visualizations, and signal rate, influence perceived avatar arousal, user distraction, and preferences. Findings show that skeuomorphic visualizations for both biosignals allow differentiable arousal inference; skeuomorphic and particles were least distracting for HR, whereas all were similarly distracting for BR; biosignal perceptions often depend on avatar relations, entertainment type, and emotion inference of avatars versus spaces. We contribute HR and BR visualizations, and considerations for designing social VR entertainment biosignal visualizations.

Supplementary Material

Supplemental Materials (3491102.3517451-supplemental-materials.zip)

Download
76.19 MB

MP4 File (3491102.3517451-video-preview.mp4)

Video Preview

Download
31.71 MB

MP4 File (3491102.3517451-talk-video.mp4)

Talk Video

Download
204.64 MB

References

[1]

N. Ambady and R. Rosenthal. 1992. Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.Psychological Bulletin 111 (1992), 256–274.

[2]

Dekker Andrew and Champion Erik. 2007. Please Biofeed the Zombies: Enhancing the Gameplay and Display of a Horror Game Using Biofeedback. In DiGRA གྷ - Proceedings of the 2007 DiGRA International Conference: Situated Play. The University of Tokyo, Tokyo, Japan, 550–558. http://www.digra.org/wp-content/uploads/digital-library/07312.18055.pdf

[3]

American Heart Association. 2021. All About Heart Rate (Pulse). https://www.heart.org/HEARTORG/Conditions/HighBloodPressure/GettheFactsAboutHighBloodPressure/All-About-Heart-Rate-Pulse_UCM_438850_Article.jsp

[4]

Laura Aymerich-Franch and Eduard Fosch-Villaronga. 2020. A Self-Guiding Tool to Conduct Research With Embodiment Technologies Responsibly. Frontiers in Robotics and AI 7 (2020), 22. https://doi.org/10.3389/frobt.2020.00022

[5]

Jeremy Bailenson, Nick Yee, Dan Merget, and Ralph Schroeder. 2006. The Effect of Behavioral Realism and Form Realism of Real-Time Avatar Faces on Verbal Disclosure, Nonverbal Disclosure, Emotion Recognition, and Copresence in Dyadic Interaction. Presence 15 (08 2006), 359–372. https://doi.org/10.1162/pres.15.4.359

Digital Library

[6]

Saskia Bakker, Doris Hausen, and Ted Selker. 2016. Peripheral Interaction: Challenges and Opportunities for HCI in the Periphery of Attention. Springer, Cham.

Digital Library

[7]

K.E. Barrett, S.M. Barman, S. Boitano, and H. Brooks. 2009. Ganong’s Review of Medical Physiology, 23rd Edition. Mcgraw-hill, New York, NY, USA. https://books.google.nl/books?id=fnEc16_ArFUC

[8]

Lyn Bartram, Abhisekh Patra, and Maureen Stone. 2017. Affective Color in Visualization. Association for Computing Machinery, New York, NY, USA, 1364?1374. https://doi.org/10.1145/3025453.3026041

Digital Library

[9]

Alejandro Beacco, Ramon Oliva, Carlos Cabreira, Jaime Gallego, and Mel Slater. 2021. Disturbance and Plausibility in a Virtual Rock Concert: A Pilot Study. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE Computer Society, Los Alamitos, CA, USA, 538–545. https://doi.org/10.1109/VR50410.2021.00078

[10]

Dhruv Beri and K. Jayasankara Reddy. 2019. Physiological Correlates of Arousal: A Metaanalytic Review. Journal of Neurology and Neuroscience 10 (01 2019). https://doi.org/10.36648/2171-6625.10.4.302

[11]

Guillermo Bernal and Pattie Maes. 2017. Emotional Beasts: Visually Expressing Emotions through Avatars in VR. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems(Denver, Colorado, USA) (CHI EA ’17). Association for Computing Machinery, New York, NY, USA, 2395?2402. https://doi.org/10.1145/3027063.3053207

Digital Library

[12]

Guillermo Bernal, Tao Yang, Abhinandan Jain, and Pattie Maes. 2018. PhysioHMD: A Conformable, Modular Toolkit for Collecting Physiological Data from Head-Mounted Displays. In Proceedings of the 2018 ACM International Symposium on Wearable Computers (Singapore, Singapore) (ISWC ’18). Association for Computing Machinery, New York, NY, USA, 160?167. https://doi.org/10.1145/3267242.3267268

Digital Library

[13]

Margaret M Bradley and Peter J Lang. 1994. Measuring emotion: the self-assessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry 25, 1(1994), 49–59.

[14]

Kathy Charmaz. 2006. Constructing grounded theory : a practical guide through qualitative analysis. Sage Publications, London; Thousand Oaks, Calif.http://www.amazon.com/Constructing-Grounded-Theory-Qualitative-Introducing/dp/0761973532

[15]

Hao Chen, Arindam Dey, Mark Billinghurst, and Robert W. Lindeman. 2017. Exploring the Design Space for Multi-Sensory Heart Rate Feedback in Immersive Virtual Reality. In Proceedings of the 29th Australian Conference on Computer-Human Interaction (Brisbane, Queensland, Australia) (OZCHI ’17). Association for Computing Machinery, New York, NY, USA, 108?116. https://doi.org/10.1145/3152771.3152783

Digital Library

[16]

Camille Cobb, Lucy Simko, Tadayoshi Kohno, and Alexis Hiniker. 2020. User Experiences with Online Status Indicators. Association for Computing Machinery, New York, NY, USA, 1?12. https://doi.org/10.1145/3313831.3376240

Digital Library

[17]

Max T. Curran, Jeremy Raboff Gordon, Lily Lin, Priyashri Kamlesh Sridhar, and John Chuang. 2019. Understanding Digitally-Mediated Empathy: An Exploration of Visual, Narrative, and Biosensory Informational Cues. Association for Computing Machinery, New York, NY, USA, 1?13. https://doi.org/10.1145/3290605.3300844

Digital Library

[18]

F. De Simone, J. Li, H. G. Debarba, A. E. Ali, S. N. B. Gunkel, and P. Cesar. 2019. Watching Videos Together in Social Virtual Reality: An Experimental Study on User’s QoE. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE Computer Society, Los Alamitos, CA, USA, 890–891. https://doi.org/10.1109/VR.2019.8798264

[19]

Arindam Dey, Hao Chen, Ashkan Hayati, Mark Billinghurst, and Robert W. Lindeman. 2019. Sharing Manipulated Heart Rate Feedback in Collaborative Virtual Environments. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE Computer Society, Los Alamitos, CA, USA, 248–257. https://doi.org/10.1109/ISMAR.2019.00022

[20]

Anind K. Dey and Ed de Guzman. 2006. From Awareness to Connectedness: The Design and Deployment of Presence Displays. Association for Computing Machinery, New York, NY, USA, 899?908. https://doi.org/10.1145/1124772.1124905

Digital Library

[21]

Aidan D’Souza, Bernd Ploderer, Madison Klarkowski, and Peta Wyeth. 2018. Augmenting Co-Located Social Play with Biofeedback: An Interactional Approach. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (Melbourne, VIC, Australia) (CHI PLAY ’18). Association for Computing Machinery, New York, NY, USA, 113?125. https://doi.org/10.1145/3242671.3242679

Digital Library

[22]

Abdallah El Ali, Xingyu Yang, Swamy Ananthanarayan, Thomas Röggla, Jack Jansen, Jess Hartcher-O’Brien, Kaspar Jansen, and Pablo Cesar. 2020. ThermalWear: Exploring Wearable On-Chest Thermal Displays to Augment Voice Messages with Affect. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1?14. https://doi.org/10.1145/3313831.3376682

Digital Library

[23]

Lisa A. Elkin, Matthew Kay, James J. Higgins, and Jacob O. Wobbrock. 2021. An Aligned Rank Transform Procedure for Multifactor Contrast Tests. arxiv:2102.11824 [stat.ME]

[24]

Milou A. Feijt, Joyce H.D.M. Westerink, Yvonne A.W. De Kort, and Wijnand A. IJsselsteijn. 2021. Sharing biosignals: An analysis of the experiential and communication properties of interpersonal psychophysiology. Human–Computer Interaction 0, 0 (2021), 1–30. https://doi.org/10.1080/07370024.2021.1913164 arXiv:https://doi.org/10.1080/07370024.2021.1913164

[25]

Catherine S. Fichten, Vicki Tagalakis, Darlene Judd, John Wright, and Rhonda Amsel. 1992. Verbal and Nonverbal Communication Cues in Daily Conversations and Dating. The Journal of Social Psychology 132, 6 (1992), 751–769. https://doi.org/10.1080/00224545.1992.9712105 arXiv:https://doi.org/10.1080/00224545.1992.9712105

[26]

Jérémy Frey. 2016. Remote Heart Rate Sensing and Projection to Renew Traditional Board Games and Foster Social Interactions. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (San Jose, California, USA) (CHI EA ’16). Association for Computing Machinery, New York, NY, USA, 1865?1871. https://doi.org/10.1145/2851581.2892391

Digital Library

[27]

Jérémy Frey, May Grabli, Ronit Slyper, and Jessica R. Cauchard. 2018. Breeze: Sharing Biofeedback through Wearable Technologies. Association for Computing Machinery, New York, NY, USA, 1?12. https://doi.org/10.1145/3173574.3174219

Digital Library

[28]

Maia Garau, Mel Slater, Vinoba Vinayagamoorthy, Andrea Brogni, Anthony Steed, and M. Angela Sasse. 2003. The Impact of Avatar Realism and Eye Gaze Control on Perceived Quality of Communication in a Shared Immersive Virtual Environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI ’03). Association for Computing Machinery, New York, NY, USA, 529?536. https://doi.org/10.1145/642611.642703

Digital Library

[29]

Sarah N. Garfinkel, Anil K. Seth, Adam B. Barrett, Keisuke Suzuki, and Hugo D. Critchley. 2015. Knowing your own heart: Distinguishing interoceptive accuracy from interoceptive awareness. Biological Psychology 104 (2015), 65–74. https://doi.org/10.1016/j.biopsycho.2014.11.004

[30]

Ceenu George and Mariam Hassib. 2019. Towards Augmenting IVR Communication with Physiological Sensing Data. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1?6. https://doi.org/10.1145/3290607.3313082

Digital Library

[31]

Renaud Gervais, Jérémy Frey, Alexis Gay, Fabien Lotte, and Martin Hachet. 2016. TOBE: Tangible Out-of-Body Experience. In Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (Eindhoven, Netherlands) (TEI ’16). Association for Computing Machinery, New York, NY, USA, 227?235. https://doi.org/10.1145/2839462.2839486

Digital Library

[32]

Erving Goffman. 1959. The Presentation of Self in Everyday Life. Anchor, Germany.

[33]

Stefan Gradl, Markus Wirth, Tobias Zillig, and Bjoern M Eskofier. 2018. Visualization of heart activity in virtual reality: A biofeedback application using wearable sensors. In 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN). IEEE Computer Society, Los Alamitos, CA, USA, 152–155. https://doi.org/10.1109/BSN.2018.8329681

[34]

Mariel Grassmann, Elke Vlemincx, Andreas Leupoldt, Justin Mittelstädt, and Omer Van den Bergh. 2016. Respiratory Changes in Response to Cognitive Load: A Systematic Review. Neural Plasticity 2016 (01 2016), 1–16. https://doi.org/10.1155/2016/8146809

[35]

Darylynn Griffin. 1998. Biofeedback: A Practitioner?s Guide, 2nd Edition. American Journal of Occupational Therapy 52, 4 (04 1998), 307–307. https://doi.org/10.5014/ajot.52.4.307b arXiv:https://ajot.aota.org/aota/content_public/journal/ajot/930024/307b.pdf

[36]

Andreas Halbig and Marc Erich Latoschik. 2021. A Systematic Review of Physiological Measurements, Factors, Methods, and Applications in Virtual Reality. Frontiers in Virtual Reality 2 (2021), 89. https://doi.org/10.3389/frvir.2021.694567

[37]

Jonathon D. Hart, Thammathip Piumsomboon, Louise Lawrence, Gun A. Lee, Ross T. Smith, and Mark Billinghurst. 2018. Emotion Sharing and Augmentation in Cooperative Virtual Reality Games. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (Melbourne, VIC, Australia) (CHI PLAY ’18 Extended Abstracts). Association for Computing Machinery, New York, NY, USA, 453?460. https://doi.org/10.1145/3270316.3271543

Digital Library

[38]

J. D. Hart, T. Piumsomboon, G. Lee, and M. Billinghurst. 2018. Sharing and Augmenting Emotion in Collaborative Mixed Reality. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE Computer Society, Los Alamitos, CA, USA, 212–213. https://doi.org/10.1109/ISMAR-Adjunct.2018.00069

[39]

Mariam Hassib, Daniel Buschek, Paweł W. Wozniak, and Florian Alt. 2017. HeartChat: Heart Rate Augmented Mobile Chat to Support Empathy and Awareness. Association for Computing Machinery, New York, NY, USA, 2239?2251. https://doi.org/10.1145/3025453.3025758

Digital Library

[40]

Mariam Hassib, Mohamed Khamis, Stefan Schneegass, Ali Sahami Shirazi, and Florian Alt. 2016. Investigating User Needs for Bio-sensing and Affective Wearables. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems(San Jose, California, USA) (CHI EA ’16). ACM, New York, NY, USA, 1415–1422. https://doi.org/10.1145/2851581.2892480

Digital Library

[41]

Paul Heidicker, Eike Langbehn, and Frank Steinicke. 2017. Influence of avatar appearance on presence in social VR. In 3D User Interfaces (3DUI), 2017 IEEE Symposium on. IEEE Computer Society, Los Alamitos, CA, USA, 233–234.

[42]

Javier Hernandez, Ivan Riobo, Agata Rozga, Gregory D. Abowd, and Rosalind W. Picard. 2014. Using Electrodermal Activity to Recognize Ease of Engagement in Children during Social Interactions. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Seattle, Washington) (UbiComp ’14). Association for Computing Machinery, New York, NY, USA, 307?317. https://doi.org/10.1145/2632048.2636065

Digital Library

[43]

Noura Howell, Laura Devendorf, Rundong (Kevin) Tian, Tomás Vega Galvez, Nan-Wei Gong, Ivan Poupyrev, Eric Paulos, and Kimiko Ryokai. 2016. Biosignals as Social Cues: Ambiguity and Emotional Interpretation in Social Displays of Skin Conductance. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (Brisbane, QLD, Australia) (DIS ’16). Association for Computing Machinery, New York, NY, USA, 865?870. https://doi.org/10.1145/2901790.2901850

Digital Library

[44]

Shinobu Ishihara 1918. Tests for color blindness. American Journal of Ophthalmology 1, 5 (1918), 376.

[45]

Joris Janssen, Jeremy Bailenson, Wijnand Ijsselsteijn, and Joyce Westerink. 2010. Intimate Heartbeats: Opportunities for Affective Communication Technology. Affective Computing, IEEE Transactions on 1 (07 2010), 72–80. https://doi.org/10.1109/T-AFFC.2010.13

Digital Library

[46]

Sungchul Jung, Christian Sandor, Pamela J. Wisniewski, and Charles E. Hughes. 2017. RealME: The Influence of Body and Hand Representations on Body Ownership and Presence. In Proceedings of the 5th Symposium on Spatial User Interaction (Brighton, United Kingdom) (SUI ’17). Association for Computing Machinery, New York, NY, USA, 3?11. https://doi.org/10.1145/3131277.3132186

Digital Library

[47]

S. Kiesler, J. Siegel, and T. McGuire. 1984. Social psychological aspects of computer-mediated communication. Computer Supported Cooperative Work 39, 10 (1984), 657–682.

[48]

M.L. Knapp, J.A. Hall, and T.G. Horgan. 2013. Nonverbal Communication in Human Interaction. Cengage Learning, Boston, USA. https://books.google.nl/books?id=-g7hkSR_mLoC

[49]

Sylvia D. Kreibig. 2010. Autonomic nervous system activity in emotion: A review. Biological Psychology 84, 3 (2010), 394–421. https://doi.org/10.1016/j.biopsycho.2010.03.010 The biopsychology of emotion: Current theoretical and empirical perspectives.

[50]

Benjamin J. Li, Jeremy N. Bailenson, Elise Ogle, and Jamil Zaki. 2020. Exploring the heart rate as a chronemic cue in virtual settings: how perceptions of consistent and varied heart rates of a storyteller influence self-reported other-arousal, empathy and social presence. Media Psychology 0, 0 (2020), 1–25. https://doi.org/10.1080/15213269.2020.1788394 arXiv:https://doi.org/10.1080/15213269.2020.1788394

[51]

Jie Li, Yiping Kong, Thomas Röggla, Francesca De Simone, Swamy Ananthanarayan, Huib de Ridder, Abdallah El Ali, and Pablo Cesar. 2019. Measuring and Understanding Photo Sharing Experiences in Social Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1?14. https://doi.org/10.1145/3290605.3300897

Digital Library

[52]

Ruby Lipson-Smith, Julie Bernhardt, Edoardo Zamuner, Leonid Churilov, Nick Busietta, and Damian Moratti. 2021. Exploring colour in context using Virtual Reality: Does a room change how you feel?Virtual Reality 25 (09 2021), 1–15. https://doi.org/10.1007/s10055-020-00479-x

Digital Library

[53]

Fannie Liu, Laura Dabbish, and Geoff Kaufman. 2017. Can Biosignals Be Expressive? How Visualizations Affect Impression Formation from Shared Brain Activity. Proc. ACM Hum.-Comput. Interact. 1, CSCW, Article 71 (Dec. 2017), 21 pages. https://doi.org/10.1145/3134706

Digital Library

[54]

Fannie Liu, Laura Dabbish, and Geoff Kaufman. 2017. Supporting Social Interactions with an Expressive Heart Rate Sharing Application. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 77 (Sept. 2017), 26 pages. https://doi.org/10.1145/3130943

Digital Library

[55]

Fannie Liu, Mario Esparza, Maria Pavlovskaia, Geoff Kaufman, Laura Dabbish, and Andrés Monroy-Hernández. 2019. Animo: Sharing Biosignals on a Smartwatch for Lightweight Social Connection. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 1, Article 18 (March 2019), 19 pages. https://doi.org/10.1145/3314405

Digital Library

[56]

Fannie Liu, Chunjong Park, Yu Jiang Tham, Tsung-Yu Tsai, Laura Dabbish, Geoff Kaufman, and Andrés Monroy-Hernández. 2021. Significant Otter: Understanding the Role of Biosignals in Communication. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 334, 15 pages. https://doi.org/10.1145/3411764.3445200

Digital Library

[57]

Sarah Lopez, Yi Yang, Kevin Beltran, Soo Jung Kim, Jennifer Cruz Hernandez, Chelsy Simran, Bingkun Yang, and Beste F. Yuksel. 2019. Investigating Implicit Gender Bias and Embodiment of White Males in Virtual Reality with Full Body Visuomotor Synchrony. Association for Computing Machinery, New York, NY, USA, 1?12. https://doi.org/10.1145/3290605.3300787

Digital Library

[58]

Ewa Lux, M. Adam, Verena Dorner, Sina Helming, M. T. Knierim, and C. Weinhardt. 2018. Live Biofeedback as a User Interface Design Element: A Review of the Literature. Commun. Assoc. Inf. Syst. 43 (2018), 18.

[59]

Dominique Makowski, Tam Pham, Zen J. Lau, Jan C. Brammer, François Lespinasse, Hung Pham, Christopher Schölzel, and S. H. Annabel Chen. 2021. NeuroKit2: A Python toolbox for neurophysiological signal processing. Behavior Research Methods 53 (02 Feb 2021), 1689?1696. https://doi.org/10.3758/s13428-020-01516-y

[60]

Tara Matthews, Anind K. Dey, Jennifer Mankoff, Scott Carter, and Tye Rattenbury. 2004. A Toolkit for Managing User Attention in Peripheral Displays. In Proc. UIST ’04 (Santa Fe, NM, USA). ACM, New York, NY, USA, 247–256.

Digital Library

[61]

Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability and Inter-Rater Reliability in Qualitative Research: Norms and Guidelines for CSCW and HCI Practice. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 72 (Nov. 2019), 23 pages. https://doi.org/10.1145/3359174

Digital Library

[62]

Daniel McDuff, Christophe Hurter, and Mar Gonzalez-Franco. 2017. Pulse and Vital Sign Measurement in Mixed Reality Using a HoloLens. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (Gothenburg, Sweden) (VRST ’17). Association for Computing Machinery, New York, NY, USA, Article 34, 9 pages. https://doi.org/10.1145/3139131.3139134

Digital Library

[63]

Daniel McDuff and Ewa M. Nowara. 2021. ?Warm Bodies?: A Post-Processing Technique for Animating Dynamic Blood Flow on Photos and Avatars. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 579, 9 pages. https://doi.org/10.1145/3411764.3445719

Digital Library

[64]

P.E. McSharry, G.D. Clifford, L. Tarassenko, and L.A. Smith. 2003. A dynamical model for generating synthetic electrocardiogram signals. IEEE Transactions on Biomedical Engineering 50, 3 (2003), 289–294. https://doi.org/10.1109/TBME.2003.808805

[65]

Nick Merrill and Coye Cheshire. 2016. Habits of the Heart(Rate): Social Interpretation of Biosignals in Two Interaction Contexts. In Proceedings of the 19th International Conference on Supporting Group Work (Sanibel Island, Florida, USA) (GROUP ’16). Association for Computing Machinery, New York, NY, USA, 31?38. https://doi.org/10.1145/2957276.2957313

Digital Library

[66]

Nick Merrill and Coye Cheshire. 2017. Trust Your Heart: Assessing Cooperation and Trust with Biosignals in Computer-Mediated Interactions. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW ’17). Association for Computing Machinery, New York, NY, USA, 2?12. https://doi.org/10.1145/2998181.2998286

Digital Library

[67]

Hyeryung Christine Min and Tek-Jin Nam. 2014. Biosignal Sharing for Affective Connectedness. In CHI ’14 Extended Abstracts on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI EA ’14). Association for Computing Machinery, New York, NY, USA, 2191?2196. https://doi.org/10.1145/2559206.2581345

Digital Library

[68]

Lennart Erik Nacke, Michael Kalyn, Calvin Lough, and Regan Lee Mandryk. 2011. Biofeedback Game Design: Using Direct and Indirect Physiological Control to Enhance Game Interaction. Association for Computing Machinery, New York, NY, USA, 103?112. https://doi.org/10.1145/1978942.1978958

Digital Library

[69]

Ville Nenonen, Aleksi Lindblad, Ville Häkkinen, Toni Laitinen, Mikko Jouhtio, and Perttu Hämäläinen. 2007. Using Heart Rate to Control an Interactive Game. Association for Computing Machinery, New York, NY, USA, 853?856. https://doi.org/10.1145/1240624.1240752

Digital Library

[70]

Helen Nissenbaum. 2011. A Contextual Approach to Privacy Online. Daedalus 140 (10 2011), 32–48. https://doi.org/10.1162/DAED_a_00113

[71]

Donald A. Norman and Stephen W. Draper. 1986. User Centered System Design; New Perspectives on Human-Computer Interaction. L. Erlbaum Associates Inc., USA.

Digital Library

[72]

Andres Pinilla, Jaime Garcia, William Raffe, Jan-Niklas Voigt-Antons, Robert P. Spang, and Sebastian Möller. 2021. Affective Visualization in Virtual Reality: An Integrative Review. Frontiers in Virtual Reality 2 (2021), 105. https://doi.org/10.3389/frvir.2021.630731

[73]

Thammathip Piumsomboon, Arindam Dey, Barrett Ens, Gun Lee, and Mark Billinghurst. 2019. The Effects of Sharing Awareness Cues in Collaborative Mixed Reality. Frontiers in Robotics and AI 6 (2019), 5. https://doi.org/10.3389/frobt.2019.00005

[74]

Zachary Pousman and John Stasko. 2006. A Taxonomy of Ambient Information Systems: Four Patterns of Design. In Proceedings of the Working Conference on Advanced Visual Interfaces (Venezia, Italy) (AVI ’06). Association for Computing Machinery, New York, NY, USA, 67?74. https://doi.org/10.1145/1133265.1133277

Digital Library

[75]

Mirjana Prpa, Ekaterina R. Stepanova, Thecla Schiphorst, Bernhard E. Riecke, and Philippe Pasquier. 2020. Inhaling and Exhaling: How Technologies Can Perceptually Extend Our Breath Awareness. Association for Computing Machinery, New York, NY, USA, 1?15. https://doi.org/10.1145/3313831.3376183

Digital Library

[76]

Susanne Putze, Dmitry Alexandrovsky, Felix Putze, Sebastian Höffner, Jan David Smeddinck, and Rainer Malaka. 2020. Breaking The Experience: Effects of Questionnaires in VR User Studies. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–15.

Digital Library

[77]

Madalena Ribeiro and Abel J. P. Gomes. 2019. Recoloring Algorithms for Colorblind People: A Survey. ACM Comput. Surv. 52, 4, Article 72 (Aug. 2019), 37 pages. https://doi.org/10.1145/3329118

Digital Library

[78]

Ronald Riggio and Howard Friedman. 1986. Impression Formation. The Role of Expressive Behavior. Journal of personality and social psychology 50 (03 1986), 421–7. https://doi.org/10.1037//0022-3514.50.2.421

[79]

Radiah Rivu, Yumeng Zhou, Robin Welsch, Ville Mäkelä, and Florian Alt. 2021. When Friends Become Strangers: Understanding the Influence of Avatar Gender on Interpersonal Distance in Virtual Reality. In Human-Computer-Interaction – INTERACT 2021, Carmelo Ardito, Rosa Lanzilotti, Alessio Malizia, Helen Petrie, Antonio Piccinno, Giuseppe Desolda, and Kori Inkpen (Eds.). Springer International Publishing, Cham, 234–250.

Digital Library

[80]

Thomas Röggla, Najereh Shirzadian, Zhiyuan Zheng, Alice Panza, and Pablo Cesar. 2017. Enhancing Music Events Using Physiological Sensor Data. In Proceedings of the 25th ACM International Conference on Multimedia (Mountain View, California, USA) (MM ’17). Association for Computing Machinery, New York, NY, USA, 1239?1240. https://doi.org/10.1145/3123266.3127919

Digital Library

[81]

Marc A. Russo, Danielle M. Santarelli, and Dean O’Rourke. 2017. The physiological effects of slow breathing in the healthy human. Breathe 13, 4 (2017), 298–309. https://doi.org/10.1183/20734735.009817 arXiv:https://breathe.ersjournals.com/content/13/4/298.full.pdf

[82]

M. Salminen, S. Jarvela, A. Ruonala, V. Harjunen, G. Jacucci, J. Hamari, and N. Ravaja. 5555. Evoking Physiological Synchrony and Empathy Using Social VR with Biofeedback. IEEE Transactions on Affective Computing 0, 01 (dec 5555), 1–1. https://doi.org/10.1109/TAFFC.2019.2958657

[83]

Mikko Salminen, Simo Järvelä, Antti Ruonala, Janne Timonen, Kristiina Mannermaa, Niklas Ravaja, and Giulio Jacucci. 2018. Bio-Adaptive Social VR to Evoke Affective Interdependence: DYNECOM. In 23rd International Conference on Intelligent User Interfaces (Tokyo, Japan) (IUI ’18). Association for Computing Machinery, New York, NY, USA, 73?77. https://doi.org/10.1145/3172944.3172991

Digital Library

[84]

Elizabeth B.-N. Sanders and Pieter Jan Stappers. 2008. Co-creation and the new landscapes of design. CoDesign 4, 1 (2008), 5–18. https://doi.org/10.1080/15710880701875068 arXiv:https://doi.org/10.1080/15710880701875068

[85]

Felix Schoeller, Philippe Bertrand, Lynda Joy Gerry, Abhinandan Jain, Adam Haar Horowitz, and Franck Zenasni. 2019. Combining Virtual Reality and Biofeedback to Foster Empathic Abilities in Humans. Frontiers in Psychology 9 (2019), 2741. https://doi.org/10.3389/fpsyg.2018.02741

[86]

Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The experience of presence: Factor analytic insights. Presence: Teleoperators & Virtual Environments 10, 3(2001), 266–281.

Digital Library

[87]

Nathan Semertzidis, Michaela Scary, Josh Andres, Brahmi Dwivedi, Yutika Chandrashekhar Kulwe, Fabio Zambetta, and Florian Floyd Mueller. 2020. Neo-Noumena: Augmenting Emotion Communication. Association for Computing Machinery, New York, NY, USA, 1?13. https://doi.org/10.1145/3313831.3376599

Digital Library

[88]

Sinem Semsioglu, Pelin Karaturhan, Saliha Akbas, and Asim Evren Yantac. 2021. Isles of Emotion: Emotionally Expressive Social Virtual Spaces for Reflection and Communication. In Creativity and Cognition (Virtual Event, Italy) (C&C ’21). Association for Computing Machinery, New York, NY, USA, Article 24, 10 pages. https://doi.org/10.1145/3450741.3466805

Digital Library

[89]

Najereh Shirzadian, Judith A. Redi, Thomas Röggla, Alice Panza, Frank Nack, and Pablo Cesar. 2018. Immersion and Togetherness: How Live Visualization of Audience Engagement Can Enhance Music Events. In Advances in Computer Entertainment Technology, Adrian David Cheok, Masahiko Inami, and Teresa Romão (Eds.). Springer International Publishing, Cham, 488–507.

[90]

Petr Slovák, Joris Janssen, and Geraldine Fitzpatrick. 2012. Understanding Heart Rate Sharing: Towards Unpacking Physiosocial Space. Association for Computing Machinery, New York, NY, USA, 859?868. https://doi.org/10.1145/2207676.2208526

Digital Library

[91]

Jaime Snyder, Mark Matthews, Jacqueline Chien, Pamara F. Chang, Emily Sun, Saeed Abdullah, and Geri Gay. 2015. MoodLight: Exploring Personal and Social Implications of Ambient Display of Biosensor Data. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (Vancouver, BC, Canada) (CSCW ’15). Association for Computing Machinery, New York, NY, USA, 143?153. https://doi.org/10.1145/2675133.2675191

Digital Library

[92]

Marc Steen. 2013. Co-Design as a Process of Joint Inquiry and Imagination. Design Issues 29 (04 2013), 16–28. https://doi.org/10.1162/DESI_a_00207

[93]

Wiens Stefan, Mezzacappa Elizabeth S., and Katkin Edward S.2000. Heartbeat detection and the experience of emotions. Cognition and Emotion 14, 3 (2000), 417–427. https://doi.org/10.1080/026999300378905 arXiv:https://doi.org/10.1080/026999300378905

[94]

Ekaterina R. Stepanova, John Desnoyers-Stewart, Philippe Pasquier, and Bernhard E. Riecke. 2020. JeL: Breathing Together to Connect with Others and Nature. Association for Computing Machinery, New York, NY, USA, 641?654. https://doi.org/10.1145/3357236.3395532

Digital Library

[95]

Anselm Strauss and Juliet M. Corbin. 1998. Basics of Qualitative Research : Techniques and Procedures for Developing Grounded Theory. SAGE Publications, Los Angeles, CA. http://www.amazon.co.uk/exec/obidos/ASIN/0803959400/citeulike-21

[96]

Patricia Valdez and Albert Mehrabian. 1994. Effects of Color on Emotions. Journal of Experimental Psychology: General 123, 4 (1994), 394–409. https://doi.org/10.1037/0096-3445.123.4.394

[97]

Eva Maria Veitmaa. 2020. Gallery of Heartbeats : soma design for increasing bodily awareness and social sharing of the heart rate through sensory stimuli. http://essay.utwente.nl/83625/

[98]

F. Visser, P. Stappers, Remko van der Lugt, and E. Sanders. 2005. Contextmapping: experiences from practice. CoDesign 1(2005), 119 – 149.

[99]

Froukje Sleeswijk Visser, Pieter Jan Stappers, Remko Van der Lugt, and Elizabeth BN Sanders. 2005. Contextmapping: experiences from practice. CoDesign 1, 2 (2005), 119–149.

[100]

Wouter Walmink, Danielle Wilde, and Florian ’Floyd’ Mueller. 2014. Displaying Heart Rate Data on a Bicycle Helmet to Support Social Exertion Experiences. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (Munich, Germany) (TEI ’14). Association for Computing Machinery, New York, NY, USA, 97?104. https://doi.org/10.1145/2540930.2540970

Digital Library

[101]

T. Waltemate, D. Gall, D. Roth, M. Botsch, and M. E. Latoschik. 2018. The Impact of Avatar Personalization and Immersion on Virtual Body Ownership, Presence, and Emotional Response. IEEE Transactions on Visualization and Computer Graphics 24, 4 (April 2018), 1643–1652. https://doi.org/10.1109/TVCG.2018.2794629

Digital Library

[102]

Joseph Walther. 2011. Theories of computer-mediated communication and interpersonal relations. The Handbook of Interpersonal Communication 1, 1 (01 2011), 443–479.

[103]

C. Wascher. 2021. Heart rate as a measure of emotional arousal in evolutionary biology. Philosophical Transactions of the Royal Society B 376 (2021), 1–X.

[104]

Julia Werner, Reto Wettach, and Eva Hornecker. 2008. United-Pulse: Feeling Your Partner’s Pulse. In Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services (Amsterdam, The Netherlands) (MobileHCI ’08). Association for Computing Machinery, New York, NY, USA, 535?538. https://doi.org/10.1145/1409240.1409338

Digital Library

[105]

R. Michael Winters, Bruce N. Walker, and Grace Leslie. 2021. Can You Hear My Heartbeat?: Hearing an Expressive Biosignal Elicits Empathy. Association for Computing Machinery, New York, NY, USA, 1?11. https://doi.org/10.1145/3411764.3445545

Digital Library

[106]

Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. Association for Computing Machinery, New York, NY, USA, 143?146. https://doi.org/10.1145/1978942.1978963

Digital Library

[107]

Kevin Yackle, Lindsay Schwarz, Kaiwen Kam, Jordan Sorokin, John Huguenard, Jack Feldman, Liqun Luo, and Mark Krasnow. 2017. Breathing control center neurons that promote arousal in mice. Science 355 (03 2017), 1411–1415. https://doi.org/10.1126/science.aai7984

[108]

Bin Yu, Jun Hu, Mathias Funk, and Loe Feijs. 2018. DeLight: biofeedback through ambient light for stress intervention and relaxation assistance. Personal and Ubiquitous Computing 22, 4 (01 Aug 2018), 787–805. https://doi.org/10.1007/s00779-018-1141-6

[109]

Bin Yu, Jun Hu, Mathias Funk, Rong-Hao Liang, Mengru Xue, and Loe Feijs. 2018. RESonance: Lightweight, Room-Scale Audio-Visual Biofeedback for Immersive Relaxation Training. IEEE Access 6(2018), 38336–38347. https://doi.org/10.1109/ACCESS.2018.2853406

[110]

R. Zajonc. 1968. Attitudinal effects of mere exposure.Journal of Personality and Social Psychology 9 (1968), 1–27.

Cited By

Wang YGong DXiao RWu XZhang H(2024)A Systematic Review on Extended Reality-Mediated Multi-User Social EngagementSystems10.3390/systems1210039612:10(396)Online publication date: 26-Sep-2024
https://doi.org/10.3390/systems12100396
Zechner OSchrom-Feiertag HWespi RPretolesi DNguyen QTscheligi M(2024)Enhancing Mixed Reality Simulation Training Technology with Real-Time Performance Visualization: Mixed Methods Study with Medical First Responders (Preprint)JMIR XR and Spatial Computing10.2196/57655Online publication date: 27-Feb-2024
https://doi.org/10.2196/57655
Kimmel SLandwehr EHeuten W(2024)Kinetic Connections: Exploring the Impact of Realistic Body Movements on Social Presence in Collaborative Virtual RealityProceedings of the ACM on Human-Computer Interaction10.1145/36869108:CSCW2(1-30)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686910
Show More Cited By

Index Terms

Understanding and Designing Avatar Biosignal Visualizations for Social Virtual Reality Entertainment
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Virtual reality

Recommendations

The effect of avatar realism in immersive social virtual realities
VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

This paper investigates the effect of avatar realism on embodiment and social interactions in Virtual Reality (VR). We compared abstract avatar representations based on a wooden mannequin with high fidelity avatars generated from photogrammetry 3D scan ...
Virtual reality in construction: a review
Engineering computational technology

Virtual Reality (VR) and the development of Virtual Environments (VEs) can make a significant impact on how construction project stakeholders can perceive and successfully complete their projects. VR techniques have the potential to enhance the ...
A Virtual Space for All: Exploring Children's Experience in Social Virtual Reality
CHI PLAY '20: Proceedings of the Annual Symposium on Computer-Human Interaction in Play

Social virtual reality (VR) is increasingly becoming an emerging online social ecosystem where multiple users can interact with one another through VR head-mounted displays in 3Dvirtual spaces. The co-existence of minors and adults in social VR presents ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

10459 pages

ISBN:9781450391573

DOI:10.1145/3491102

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA
,
Steven Drucker
Microsoft Research, USA
,
Julie Williamson
University of Glasgow, UK
,
Koji Yatani
University of Tokyo, Japan

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2022

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
3,061
Total Downloads

Downloads (Last 12 months)1,007
Downloads (Last 6 weeks)112

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang YGong DXiao RWu XZhang H(2024)A Systematic Review on Extended Reality-Mediated Multi-User Social EngagementSystems10.3390/systems1210039612:10(396)Online publication date: 26-Sep-2024
https://doi.org/10.3390/systems12100396
Zechner OSchrom-Feiertag HWespi RPretolesi DNguyen QTscheligi M(2024)Enhancing Mixed Reality Simulation Training Technology with Real-Time Performance Visualization: Mixed Methods Study with Medical First Responders (Preprint)JMIR XR and Spatial Computing10.2196/57655Online publication date: 27-Feb-2024
https://doi.org/10.2196/57655
Kimmel SLandwehr EHeuten W(2024)Kinetic Connections: Exploring the Impact of Realistic Body Movements on Social Presence in Collaborative Virtual RealityProceedings of the ACM on Human-Computer Interaction10.1145/36869108:CSCW2(1-30)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686910
You CChen HChen PLai WYuan CBi N(2024)Toward Understanding the Impact of Visualized Focus Levels in Virtual Reality on User Presence and ExperienceProceedings of the ACM on Human-Computer Interaction10.1145/36765278:MHCI(1-30)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676527
Chen HPeng YChen YCastellanos AChuang CChen YYou CKostakos VKay JHoang T(2024)Tell Me How You Play: Exploring Ways to Enhance the Gaming Experience in Asymmetric Multiplayer VR Games through Affective State VisualizationCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677577(19-24)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677577
Li SLi ALi X(2024)Distancebit: Designing a Technology Probe to Envision AR Glasses Enhancing Embodied Cross-Cultural Social InteractionAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680457(1-6)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680457
Armstrong MYang CSkiers KLim MGunasekaran TWang ZNarumi TMinamizawa KPai Y(2024)SealMates: Improving Communication in Video Conferencing using a Collective Behavior-Driven AvatarProceedings of the ACM on Human-Computer Interaction10.1145/36373958:CSCW1(1-23)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637395
Martinez RSquire K(2024)Engaging recently incarcerated and gang affiliated Black and Latino/a young adults in designing social collocated applications for mixed reality smart glasses through community-based participatory design workshops.Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642895(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642895
Huang ZCao XZhang YMa X(2024)Sharing Frissons among Online Video Viewers: Exploring the Design of Affective Communication for Aesthetic ChillsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642818(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642818
Bimberg PMinuth MZielasko DWeyers B(2024)Analyzing Exergame Recordings with Embedded Bio-Data in Immersive Virtual Reality2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00029(138-144)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00029
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents