research-article

Examining Design Choices of Questionnaires in VR User Studies

Authors:

Dmitry Alexandrovsky,

Michael Bonfert,

Sebastian Höffner,

Pitt Michelmann,

Jan David SmeddinckAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 21

https://doi.org/10.1145/3313831.3376260

Published: 23 April 2020 Publication History

Abstract

Questionnaires are among the most common research tools in virtual reality (VR) user studies. Transitioning from virtuality to reality for giving self-reports on VR experiences can lead to systematic biases. VR allows to embed questionnaires into the virtual environment which may ease participation and avoid biases. To provide a cohesive picture of methods and design choices for questionnaires in VR (inVRQ), we discuss 15 inVRQ studies from the literature and present a survey with 67 VR experts from academia and industry. Based on the outcomes, we conducted two user studies in which we tested different presentation and interaction methods of inVRQs and evaluated the usability and practicality of our design. We observed comparable completion times between inVRQs and questionnaires outside VR (nonVRQs) with higher enjoyment but lower usability for \inVRQs. These findings advocate the application of inVRQs and provide an overview of methods and considerations that lay the groundwork for inVRQ design.

Supplementary Material

ZIP File (paper133aux.zip)

The auxiliary materials contain the expert survey with all questions and response options.

Download
88.36 KB

MP4 File (paper133vf.mp4)

Supplemental video

Download
291.08 MB

MP4 File (paper133pv.mp4)

Preview video

Download
12.67 MB

MP4 File (a133-alexandrovsky-presentation.mp4)

Download
55.92 MB

References

[1]

Ethel Mary Abernethy. 1940. The Effect of Changed Environmental Conditions Upon the Results of College Examinations. The Journal of Psychology 10, 2 (Oct. 1940), 293--301.

[2]

Parastoo Abtahi, Mar Gonzalez-Franco, Eyal Ofek, and Anthony Steed. 2019a. I'm a Giant: Walking in Large Virtual Environments at High Speed Gains. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 522:1--522:13.

Digital Library

[3]

Parastoo Abtahi, Benoit Landry, Jackie (Junrui) Yang, Marco Pavone, Sean Follmer, and James A. Landay. 2019b. Beyond The Force: Using Quadcopters to Appropriate Objects and the Environment for Haptics in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 359:1--359:13.

[4]

Lu Ann Aday, Llewellyn J. Cornelius, Foreword Steven, B. Cohen, Lu Ann Aday, Llewellyn J. Cornelius, Foreword Steven, B. Cohen, and A. Wiley Imprint. 2006. Designing and Conducting Health Surveys: A Comprehensive Guide (thrid edition ed.). Jossey-Bass Publishers.

[5]

G. Ahamba, D. J. Roberts, and P. Eachus. 2017. Triggering and Measuring Social Inhibitory Response in Humans Immersed in Interactions with Virtual Humans. Annual Review of Cybertherapy and Telemedicine 15 (April 2017), 185--188.

[6]

Majed Al Zayer, Isayas B. Adhanom, Paul MacNeilage, and Eelke Folmer. 2019. The Effect of Field-of-View Restriction on Sex Bias in VR Sickness and Spatial Navigation Performance. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 354:1--354:12.

Digital Library

[7]

Judith Amores, Xavier Benavides, and Pattie Maes. 2015. ShowMe: A Remote Collaboration System That Supports Immersive Gestural Communication. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, NY, NY, USA, 1343--1348.

Digital Library

[8]

I. M. Arafat, S. M. Shahnewaz Ferdous, and J. Quarles. 2018. Cybersickness-Provoking Virtual Reality Alters Brain Signals of Persons with Multiple Sclerosis. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 1--120.

[9]

Rahul Arora, Rubaiat Habib Kazi, Fraser Anderson, Tovi Grossman, Karan Singh, and George Fitzmaurice. 2017. Experimental Evaluation of Sketching on Surfaces in VR. ACM Press, 5643--5654.

Digital Library

[10]

Association for Computing Machinery. 2019. ACM Digital Library. https://dl.acm.org/. (2019).

[11]

Mahdi Azmandian, Timofey Grechkin, and Evan Suma Rosenberg. 2017. An Evaluation of Strategies for Two-User Redirected Walking in Shared Physical Spaces. IEEE, 91--98.

[12]

Earl Babbie. 1990. Survey Research Methods (2nd ed. ed.). Wadsworth, Balmont, Calif.

[13]

Chris Baker, Nancy Pistrang, and Robert Elliott. 2002. Research Methods in Clinical Psychology: An Introduction for Students and Practitioners (second edition ed.). John Wiley & Sons Ltd.

[14]

Aaron Bangor, Kortum, Philip, and Miller, James. 2009. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. Journal of usability studies 4, 3 (2009), 114--123.

Digital Library

[15]

Soumya C. Barathi, Daniel J. Finnegan, Matthew Farrow, Alexander Whaley, Pippa Heath, Jude Buckley, Peter W. Dowrick, Burkhard C. Wuensche, James L. J. Bilzon, Eamonn O'Neill, and Christof Lutteroth. 2018. Interactive Feedforward for Improving Performance and Maintaining Intrinsic Motivation in VR Exergaming. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 408:1--408:14.

Digital Library

[16]

Jiwan Bhandari, Sam Tregillus, and Eelke Folmer. 2017. Legomotion: Scalable Walking-Based Virtual Locomotion. ACM Press, 1--8.

Digital Library

[17]

Anol Bhattacherjee. 2012. Social Science Research: Principles, Methods, and Practices (second edition ed.). Number 3 in Textbooks Collection.

[18]

Sam B Bhayani and Gerald L Andriole. 2005. Three-Dimensional (3D) Vision: Does It Improve Laparoscopic Skills? An Assessment of a 3D Head-Mounted Visualization System. Reviews in Urology 7, 4 (2005), 211--214.

[19]

Bobby Bodenheimer, Sarah Creem-Regehr, Jeanine Stefanucci, Elena Shemetova, and William B. Thompson. 2017. Prism Aftereffects for Throwing with a Self-Avatar in an Immersive Virtual Environment. IEEE, 141--147.

[20]

M. Boldt, M. Bonfert, I. Lehne, M. Cahnbley, K. Korschinq, L. Bikas, S. Finke, M. Hanci, V. Kraft, B. Liu, T. Nguyen, A. Panova, R. Singh, A. Steenbergen, R. Malaka, and J. Jan Smeddinck. 2018. You Shall Not Pass: Non-Intrusive Feedback for Virtual Walls in VR Environments with Room-Scale Mapping. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 143--150.

[21]

Stéphane Bouchard, Julie St-Jacques, Geneviève Robillard, and Patrice Renaud. 2008. Anxiety Increases the Feeling of Presence in Virtual Reality. Presence: Teleoperators and Virtual Environments 17, 4 (July 2008), 376--391.

Digital Library

[22]

LouAnne E. Boyd, Saumya Gupta, Sagar B. Vikmani, Carlos M. Gutierrez, Junxiang Yang, Erik Linstead, and Gillian R. Hayes. 2018. vrSocial: Toward Immersive Therapeutic VR Systems for Children with Autism. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 204:1--204:12.

Digital Library

[23]

Norman M. Bradburn and Seymour Sudman. 1979. Improving Interview Methods and Questionnaire Design - Response Effects to Threatening Qurstions in Survey Research. Jossey-Bass Publishers.

[24]

Andrea Brogni, Vinoba Vinayagamoorthy, Anthony Steed, and Mel Slater. 2006. Variations in Physiological Responses of Participants During Different Stages of an Immersive Virtual Environment Experiment. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '06). ACM, Limassol, Cyprus, 376--382.

Digital Library

[25]

John Brooke. 1996. SUS - A Quick and Dirty Usability Scale. (1996), 7.

[26]

Emily Brown and Paul Cairns. 2004. A Grounded Investigation of Game Immersion. In Extended Abstracts of the 2004 Conference on Human Factors and Computing Systems - CHI '04. ACM Press, Vienna, Austria, 1297.

Digital Library

[27]

Peter W. S. Butcher, Nigel W. John, and Panagiotis D. Ritsos. 2019. VRIA - A Framework for Immersive Analytics on the Web. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW2615:1--LBW2615:6.

[28]

Zekun Cao, Jason Jerald, and Regis Kopper. 2018. Visually-Induced Motion Sickness Reduction via Static and Dynamic Rest Frames. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 105--112.

[29]

Sibo Chen, Junce Wang, Santiago Guerra, Neha Mittal, and Soravis Prakkamakul. 2019. Exploring Word-Gesture Text Entry Techniques in Virtual Reality. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW0233:1--LBW0233:6.

[30]

Alan Cheng, Lei Yang, and Erik Andersen. 2017b. Teaching Language and Culture with a Virtual Reality Game. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 541--549.

Digital Library

[31]

Lung-Pan Cheng, Eyal Ofek, Christian Holz, Hrvoje Benko, and Andrew D. Wilson. 2017a. Sparse Haptic Proxy: Touch Feedback in Virtual Environments Using a General Passive Prop. ACM Press, 3718--3728.

Digital Library

[32]

Bernard C.K. Choi and Anita W.P. Pak. 2004. A Catalog of Biases in Questionnaires. Preventing Chronic Disease 2, 1 (Dec. 2004).

[33]

Tanvir Irfan Chowdhury, Sharif Mohammad Shahnewaz Ferdous, and John Quarles. 2017. Information Recall in a Virtual Reality Disability Simulation. ACM Press, 1--10.

Digital Library

[34]

Sebastian Cmentowski, Andrey Krekhov, and Jens Krueger. 2019. Outstanding: A Perspective-Switching Technique for Covering Large Distances in VR Games. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW1612:1--LBW1612:6.

[35]

Mihaly Csikszentmihalyi. 1990. Flow: The Psychology of Optimal Experience. Harper & Row, New York.

[36]

Lynnda M. Dahlquist, Kristine D. McKenna, Katia K. Jones, Lindsay Dillinger, Karen E. Weiss, and Claire Sonntag Ackerman. 2007. Active and Passive Distraction Using a Head-Mounted Display Helmet: Effects on Cold Pressor Pain in Children. Health Psychology: Official Journal of the Division of Health Psychology, American Psychological Association 26, 6 (Nov. 2007), 794--801.

[37]

Donald Degraen, André Zenner, and Antonio Krüger. 2019. Enhancing Texture Perception in Virtual Reality Using 3D-Printed Hair Structures. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 249:1--249:12.

Digital Library

[38]

Arindam Dey, Hao Chen, Mark Billinghurst, and Robert W. Lindeman. 2018. Effects of Manipulating Physiological Feedback in Immersive Virtual Environments. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, Melbourne, VIC, Australia, 101--111.

[39]

Cath Dillon, Edmund Keogh, and Jonathan Freeman. 2002. It's Been Emotional: Affect, Physiology, and Presence. In Proceedings of the Fifth Annual International Workshop on Presence, Porto, Portugal.

[40]

Tilman Dingler, Kai Kunze, and Benjamin Outram. 2018. VR Reading UIs: Assessing Text Parameters for Reading in VR. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, NY, NY, USA, LBW094:1--LBW094:6.

Digital Library

[41]

Alan Dix. 2009. Human-Computer Interaction. In Encyclopedia of Database Systems. Springer, 1327--1331.

[42]

E. Ebrahimi, L. S. Hartman, A. Robb, C. C. Pagano, and S. V. Babu. 2018. Investigating the Effects of Anthropomorphic Fidelity of Self-Avatars on Near Field Depth Perception in Immersive Virtual Environments. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 1--8.

[43]

Claudia Faita, Federico Vanni, Camilla Tanca, Emanuele Ruffaldi, Marcello Carrozzino, and Massimo Bergamasco. 2016. Investigating the Process of Emotion Recognition in Immersive and Non-Immersive Virtual Technological Setups. ACM Press, 61--64.

Digital Library

[44]

A. S. Fernandes and S. K. Feiner. 2016. Combating VR Sickness through Subtle Dynamic Field-of-View Modification. In 2016 IEEE Symposium on 3D User Interfaces (3DUI). 201--210.

[45]

Andy Field and Graham Hole. 2003. How to Design and Report Experiments. Sage Publications Ltd, London ; Thousand Oaks, Calif.

[46]

Kraig Finstad. 2010. The Usability Metric for User Experience. Interacting with Computers 22, 5 (Sept. 2010), 323--327.

Digital Library

[47]

Daniel Freeman, Polly Haselton, Jason Freeman, Bernhard Spanlang, Sameer Kishore, Emily Albery, Megan Denne, Poppy Brown, Mel Slater, and Alecia Nickless. 2018. Automated Psychological Therapy Using Immersive Virtual Reality for Treatment of Fear of Heights: A Single-Blind, Parallel-Group, Randomised Controlled Trial. The Lancet Psychiatry 5, 8 (Aug. 2018), 625--632.

[48]

Jonathan Freeman, S. E. Avons, Ray Meddis, Don E. Pearson, and Wijnand IJsselsteijn. 2000. Using Behavioral Realism to Estimate Presence: A Study of the Utility of Postural Responses to Motion Stimuli. Presence: Teleoperators and Virtual Environments 9, 2 (April 2000), 149--164.

Digital Library

[49]

Jonathan Freeman, S. E. Avons, Don E. Pearson, and Wijnand A. IJsselsteijn. 1999. Effects of Sensory Information and Prior Experience on Direct Subjective Ratings of Presence. Presence: Teleoperators and Virtual Environments 8, 1 (Feb. 1999), 1--13.

Digital Library

[50]

R. Fribourg, F. Argelaguet, L. Hoyet, and A. Lécuyer. 2018. Studying the Sense of Embodiment in VR Shared Experiences. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 273--280.

[51]

Thomas Fröhlich, Dmitry Alexandrovsky, Timo Stabbert, Tanja Döring, and Rainer Malaka. 2018. VRBox: A Virtual Reality Augmented Sandbox for Immersive Playfulness, Creativity and Exploration. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, Melbourne, VIC, Australia, 153--162.

Digital Library

[52]

Julian Frommel, Kim Fahlbusch, Julia Brich, and Michael Weber. 2017. The Effects of Context-Sensitive Tutorials in Virtual Reality Games. ACM Press, 367--375.

Digital Library

[53]

Julian Frommel, Michael Weber, Katja Rogers, Julia Brich, Daniel Besserer, Leonard Bradatsch, Isabel Ortinau, Ramona Schabenberger, Valentin Riemer, and Claudia Schrader. 2015. Integrated Questionnaires: Maintaining Presence in Game Environments for Self-Reported Data Acquisition. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY '15. ACM Press, London, United Kingdom, 359--368.

Digital Library

[54]

Wei Gai, Cheng Lin, Chenglei Yang, Yulong Bian, Chia Shen, Xiangxu Meng, Lu Wang, Juan Liu, Mingda Dong, and Chengjie Niu. 2017. Supporting Easy Physical-to-Virtual Creation of Mobile VR Maze Games: A New Genre. ACM Press, 5016--5028.

Digital Library

[55]

D. Gall and M. E. Latoschik. 2018. The Effect of Haptic Prediction Accuracy on Presence. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 73--80.

[56]

Maryrose Gerardi, Barbara Olasov Rothbaum, Kerry Ressler, Mary Heekin, and Albert Rizzo. 2008. Virtual reality exposure therapy using a virtual Iraq: Case report. Journal of Traumatic Stress 21, 2 (April 2008), 209--213.

[57]

Limesurvey GmbH. 2003. LimeSurvey: An Open Source Survey Tool. (2003).

[58]

D. R. Godden and A. D. Baddeley. 1975. Context-Dependent MemoryIn Two Natural Environments: On Land and Underwater. British Journal of Psychology 66, 3 (Aug. 1975), 325--331.

[59]

David Goedicke, Jamy Li, Vanessa Evers, and Wendy Ju. 2018. VR-OOM: Virtual Reality On-rOad Driving siMulation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 165:1--165:11.

Digital Library

[60]

Jeffrey I. Gold, Seok Hyeon Kim, Alexis J. Kant, Michael H. Joseph, and Albert "Skip" Rizzo. 2006. Effectiveness of Virtual Reality for Pediatric Pain Distraction during IV Placement. CyberPsychology & Behavior 9, 2 (April 2006), 207--212.

[61]

Google. 2018. Rendering Text in WebVR | Web. https://developers.google.com/web/showcase/2017/within. (Sept. 2018).

[62]

Google Developers. 2017. Designing Screen Interfaces for VR (Google I/O '17). (2017).

[63]

Antti Granqvist, Tapio Takala, Jari Takatalo, and Perttu Hämäläinen. 2018. Exaggeration of Avatar Flexibility in Virtual Reality. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, Melbourne, VIC, Australia, 201--209.

Digital Library

[64]

Anthony G. Greenwald and Brian A. Nosek. 2001. Health of the Implicit Association Test at Age 3. Experimental Psychology 48, 2 (April 2001), 85--93.

[65]

Rebecca A. Grier. 2015. How High Is High? A Meta-Analysis of NASA-TLX Global Workload Scores. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 59, 1 (Sept. 2015), 1727--1731.

[66]

Nathan Navarro Griffin, James Liu, and Eelke Folmer. 2018. Evaluation of Handsbusy vs Handsfree Virtual Locomotion. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, Melbourne, VIC, Australia, 211--219.

Digital Library

[67]

J. Grubert, L. Witzani, E. Ofek, M. Pahud, M. Kranz, and P. O. Kristensson. 2018a. Effects of Hand Representations for Typing in Virtual Reality. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 151--158.

[68]

J. Grubert, L. Witzani, E. Ofek, M. Pahud, M. Kranz, and P. O. Kristensson. 2018b. Text Entry in Immersive Head-Mounted Display-Based Virtual Reality Using Standard Keyboards. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 159--166.

[69]

Jan Gugenheimer, Evgeny Stemasov, Julian Frommel, and Enrico Rukzio. 2017. ShareVR: Enabling Co-Located Experiences for Virtual Reality between HMD and Non-HMD Users. ACM Press, 4021--4033.

Digital Library

[70]

Jan Gugenheimer, Evgeny Stemasov, Harpreet Sareen, and Enrico Rukzio. 2018. FaceDisplay: Towards Asymmetric Multi-User Interaction for Nomadic Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 54:1--54:13.

Digital Library

[71]

John Paulin Hansen, Astrid Kofod Trudslev, Sara Amdi Harild, Alexandre Alapetite, and Katsumi Minakata. 2019. Providing Access to VR Through a Wheelchair. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, CS07:1--CS07:8.

Digital Library

[72]

Sandra G. Hart. 1986. NASA Task Load Index (TLX). Volume 1.0; Paper and Pencil Package. Technical Report.

[73]

Jeremy Hartmann, Christian Holz, Eyal Ofek, and Andrew D. Wilson. 2019. RealityCheck: Blending Virtual Environments with Situated Physical Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 347:1--347:12.

[74]

Tianyu He, Xiaoming Chen, Zhibo Chen, Ye Li, Sen Liu, Junhui Hou, and Ying He. 2017. Immersive and Collaborative Taichi Motion Learning in Various VR Environments. IEEE, 307--308.

[75]

Teresa Hirzle, Jan Rixen, Jan Gugenheimer, and Enrico Rukzio. 2018. WatchVR: Exploring the Usage of a Smartwatch for Interaction in Mobile Virtual Reality. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, NY, NY, USA, LBW634:1--LBW634:6.

Digital Library

[76]

Philipp Hock, Sebastian Benedikter, Jan Gugenheimer, and Enrico Rukzio. 2017. CarVR: Enabling In-Car Virtual Reality Entertainment. ACM Press, 4034--4044.

Digital Library

[77]

Matthias Hoppe, Jakob Karolus, Felix Dietz, Pawel W. Wo´zniak, Albrecht Schmidt, and Tonja-Katrin Machulla. 2019. VRsneaky: Increasing Presence in VR Through Gait-Aware Auditory Feedback. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 546:1--546:9.

Digital Library

[78]

HTC. 2016. VIVE Now Shipping Immediately From HTC, Retail Partners Expand Demo Locations. https://www.htc.com/us/newsroom/2016-06-07/. (2016).

[79]

Brent E. Insko. 2003. Measuring Presence: Subjective, Behavioral and Physiological Methods. In Being There: Concepts, Effects and Measurements of User Presence in Synthetic Environments. IOS Press, Amsterdam, Netherlands, 109--119.

[80]

Institute of Electrical and Electronics Engineers. 2019. IEEE Xplore Digital Library. https://ieeexplore.ieee.org/. (2019).

[81]

International Organization for Standardization (ISO). 2006. Ergonomics of Human-System Interaction - Part 110: Dialogue Principles (ISO 9241--110). (2006).

[82]

Charlene Jennett, Anna L. Cox, Paul Cairns, Samira Dhoparee, Andrew Epps, Tim Tijs, and Alison Walton. 2008. Measuring and Defining the Experience of Immersion in Games. International Journal of Human-Computer Studies 66, 9 (Sept. 2008), 641--661.

Digital Library

[83]

Jason Jerald. 2016. The VR Book: Human-Centered Design for Virtual Reality (first edition ed.). Number 8 in ACM Books. acm, Association for Computing Machinery, New York. OCLC: 945367096.

[84]

S. Jung, P. J. Wisniewski, and C. E. Hughes. 2018. In Limbo: The Effect of Gradual Visual Transition Between Real and Virtual on Virtual Body Ownership Illusion and Presence. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 267--272.

[85]

HyeongYeop Kang, Geonsun Lee, Seongsu Kwon, Ohung Kwon, Seongpil Kim, and JungHyun Han. 2018. Flotation Simulation in a Cable-Driven Virtual Environment -- A Study with Parasailing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 632:1--632:11.

Digital Library

[86]

Oliver Beren Kaul and Michael Rohs. 2017. HapticHead: A Spherical Vibrotactile Grid around the Head for 3D Guidance in Virtual and Augmented Reality. ACM Press, 3729--3740.

Digital Library

[87]

Robert S. Kennedy, Norman E. Lane, Kevin S. Berbaum, and Michael G. Lilienthal. 1993. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology 3, 3 (July 1993), 203--220.

[88]

Behrang Keshavarz and Heiko Hecht. 2011. Validating an Efficient Method to Quantify Motion Sickness. Human Factors 53, 4 (Aug. 2011), 415--426.

[89]

Youngwon Ryan Kim and Gerard J. Kim. 2017. Presence and Immersion of "Easy" Mobile VR with Open Flip-on Lenses. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (VRST '17). ACM, NY, NY, USA, 38:1--38:7.

Digital Library

[90]

Tina Kjær, Christoffer B. Lillelund, Mie Moth-Poulsen, Niels C. Nilsson, Rolf Nordahl, and Stefania Serafin. 2017. Can You Cut It?: An Exploration of the Effects of Editing in Cinematic Virtual Reality. ACM Press, 1--4.

Digital Library

[91]

Jarrod Knibbe, Jonas Schjerlund, Mathias Petraeus, and Kasper Hornb\{ae}k. 2018. The Dream Is Collapsing: The Experience of Exiting VR. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 483:1--483:13.

Digital Library

[92]

Pascal Knierim, Valentin Schwind, Anna Maria Feit, Florian Nieuwenhuizen, and Niels Henze. 2018. Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 345:1--345:9.

Digital Library

[93]

Julian Kreimeier and Timo Götzelmann. 2019. First Steps Towards Walk-In-Place Locomotion and Haptic Feedback in Virtual Reality for Visually Impaired. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW2214:1--LBW2214:6.

[94]

Andrey Krekhov, Sebastian Cmentowski, Katharina Emmerich, Maic Masuch, and Jens Krüger. 2018. GulliVR: A Walking-Oriented Technique for Navigation in Virtual Reality Games Based on Virtual Body Resizing. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, Melbourne, VIC, Australia, 243--256.

Digital Library

[95]

Andrey Krekhov, Katharina Emmerich, Philipp Bergmann, Sebastian Cmentowski, and Jens Krüger. 2017. Self-Transforming Controllers for Virtual Reality First Person Shooters. ACM Press, 517--529.

Digital Library

[96]

Heinz W. Krohne, Boris Egloff, Carl-Walter Kohlmann, and Anja Tausch. 1996. Untersuchungen Mit Einer Deutschen Version Der "Positive and Negative Affect Schedule"(PANAS). Diagnostica-Gottingen- 42 (1996), 139--156.

[97]

W. Kruger, C. Bohn, B. Frohlich, H. Schuth, W. Strauss, and G. Wesche. 1995. The Responsive Workbench: A Virtual Work Environment. Computer 28, 7 (July 1995), 42--48.

[98]

D. M. Krum, S. Kang, and T. Phan. 2018. Influences on the Elicitation of Interpersonal Space with Virtual Humans. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 223--9.

[99]

David M. Krum, Sin-Hwa Kang, Thai Phan, Lauren Cairco Dukes, and Mark Bolas. 2017. Social Impact of Enhanced Gaze Presentation Using Head Mounted Projection. In Proceedings of the Human-Computer Interaction International Conference. Springer International Publishing.

Digital Library

[100]

L. Kruse, E. Langbehn, and F. Stelnlcke. 2018. I Can See on My Feet While Walking: Sensitivity to Translation Gains with Visible Feet. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 305--312.

[101]

Jérémy Lacoche, Nico Pallamin, Thomas Boggini, and Jérôme Royan. 2017. Collaborators Awareness for User Cohabitation in Co-Located Collaborative Virtual Environments. ACM Press, 1--9.

Digital Library

[102]

Joseph J. LaViola, Ernst Kruijff, Doug A. Bowman, Ryan P. McMahan, and Ivan Poupyrev. 2017. 3D User Interfaces: Theory and Practice. Addison-Wesley.

Digital Library

[103]

Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research Methods in Human-Computer Interaction (2 ed.). Morgan Kaufmann, Cambridge, MA.

[104]

Kangdon Lee. 2012. Augmented Reality in Education and Training. TechTrends 56, 2 (March 2012), 13--21.

[105]

Myungho Lee, Gerd Bruder, and Gregory F. Welch. 2017. Exploring the Effect of Vibrotactile Feedback through the Floor on Social Presence in an Immersive Virtual Environment. IEEE, 105--111.

[106]

Meng-Yun Liao, Ching-Ying Sung, Hao-Chuan Wang, Wen-Chieh Lin, and Fu-Yin Cherng. 2019. Embodying Historical Learners' Messages As Learning Companions in a VR Classroom. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW1115:1--LBW1115:6.

Digital Library

[107]

Pedro Lopes, Sijing You, Lung-Pan Cheng, Sebastian Marwecki, and Patrick Baudisch. 2017. Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation. ACM Press, 1471--1482.

Digital Library

[108]

Sarah Lopez, Yi Yang, Kevin Beltran, Soo Jung Kim, Jennifer Cruz Hernandez, Chelsy Simran, Bingkun Yang, and Beste F. Yuksel. 2019. Investigating Implicit Gender Bias and Embodiment of White Males in Virtual Reality with Full Body Visuomotor Synchrony. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 557.

[109]

C. Di Loreto, J. Chardonnet, J. Ryard, and A. Rousseau. 2018. WoaH: A Virtual Reality Work-at-Height Simulator. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 281--288.

[110]

J. Lugrin, M. Ertl, P. Krop, R. Klüpfel, S. Stierstorfer, B. Weisz, M. Rück, J. Schmitt, N. Schmidt, and M. E. Latoschik. 2018. Any "Body" There? Avatar Visibility Effects in a Virtual Reality Game. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 17--24.

[111]

Raquel Macedo, Nuno Correia, and Teresa Romão. 2019. Paralympic VR Game: Immersive Game Using Virtual Reality and Video. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW0235:1--LBW0235:6.

[112]

I. Scott MacKenzie, Tatu Kauppinen, and Miika Silfverberg. 2001. Accuracy Measures for Evaluating Computer Pointing Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, Seattle, Washington, USA, 9--16. MacKenzie2001.

Digital Library

[113]

I. Scott MacKenzie, Abigail Sellen, and William A. S. Buxton. 1991. A Comparison of Input Devices in Element Pointing and Dragging Tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '91). ACM, NY, NY, USA, 161--166.

Digital Library

[114]

Elhassan Makled, Yomna Abdelrahman, Noha Mokhtar, Valentin Schwind, Slim Abdennadher, and Albrecht Schmidt. 2018. I Like to Move It: Investigating the Effect of Head and Body Movement of Avatars in VR on User's Perception. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, NY, NY, USA, LBW099:1--LBW099:6.

Digital Library

[115]

Willie McClinton, Derek Caprio, Denis Laesker, Blanche Pinto, Sarah Garcia, and Marvin Andujar. 2019. P300-Based 3D Brain Painting in Virtual Reality. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW1119:1--LBW1119:6.

Digital Library

[116]

Michael Meehan. 2001. Physiological Reaction as an Objective Measure of Presence in Virtual Environments. Ph.D. Dissertation.

Digital Library

[117]

Daniel Mendes, Maurício Sousa, Rodrigo Lorena, Alfredo Ferreira, and Joaquim Jorge. 2017. Using Custom Transformation Axes for Mid-Air Manipulation of 3D Virtual Objects. ACM Press, 1--8.

Digital Library

[118]

Mark R Mine. 1997. ISAAC: A Meta-Cad System for Virtual Environments. Computer-Aided Design 29, 8 (Aug. 1997), 547--553.

[119]

Thomas B. Moeslund, Moritz Störring, and Erik Granum. 2002. A Natural Interface to a Virtual Environment through Computer Vision-Estimated Pointing Gestures. In Gesture and Sign Language in Human-Computer Interaction (Lecture Notes in Computer Science), Ipke Wachsmuth and Timo Sowa (Eds.). Springer Berlin Heidelberg, 59--63.

[120]

Aske Mottelson and Kasper Hornbæk. 2017. Virtual Reality Studies Outside the Laboratory. ACM Press, 1--10.

Digital Library

[121]

C. Mousas. 2018. Performance-Driven Dance Motion Control of a Virtual Partner Character. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 57--64.

[122]

Thomas Muender, Anke V. Reinschluessel, Sean Drewes, Dirk Wenig, Tanja Döring, and Rainer Malaka. 2019. Does It Feel Real?: Using Tangibles with Different Fidelities to Build and Explore Scenes in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 673:1--673:12.

Digital Library

[123]

Marcus R. Munafò, Brian A. Nosek, Dorothy V. M. Bishop, Katherine S. Button, Christopher D. Chambers, Nathalie Percie du Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J. Ware, and John P. A. Ioannidis. 2017. A Manifesto for Reproducible Science. Nature Human Behaviour 1, 1 (Jan. 2017), 0021.

[124]

S. Narang, A. Best, and D. Manocha. 2018. Simulating Movement Interactions Between Avatars Agents in Virtual Worlds Using Human Motion Constraints. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 9--16.

[125]

Cuong Nguyen, Stephen DiVerdi, Aaron Hertzmann, and Feng Liu. 2017. Vremiere: In-Headset Virtual Reality Video Editing. ACM Press, 5428--5438.

Digital Library

[126]

Cuong Nguyen, Stephen DiVerdi, Aaron Hertzmann, and Feng Liu. 2018. Depth Conflict Reduction for Stereo VR Video Interfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 64:1--64:9.

Digital Library

[127]

Sebastian Oberdörfer, David Heidrich, and Marc Erich Latoschik. 2019. Usability of Gamified Knowledge Learning in VR and Desktop-3D. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 175:1--175:13.

Digital Library

[128]

Oculus. 2019. Introduction to Best Practices. https://developer.oculus.com/design/latest/concepts/bookbp/. (Sept. 2019).

[129]

Eyal Ofek and Michel Pahud. 2018. Typing in the Virtual Office Environment: Examining the Effects of Keyboard and Hand Representation in VR. (Feb. 2018).

[130]

Roshan Lalintha Peiris, Wei Peng, Zikun Chen, Liwei Chan, and Kouta Minamizawa. 2017. ThermoVR: Exploring Integrated Thermal Haptic Feedback with Head Mounted Displays. ACM Press, 5452--5456.

Digital Library

[131]

Xiaolan Peng, Jin Huang, Linghan Li, Chen Gao, Hui Chen, Feng Tian, and Hongan Wang. 2019. Beyond Horror and Fear: Exploring Player Experience Invoked by Emotional Challenge in VR Games. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW1616:1--LBW1616:6.

Digital Library

[132]

Max Pfeiffer, Tobias Kröger, Jens Seifert, Sulaxan Somaskantharajan, Lukas Jahnich, Tobias Steinblum, Jan Speckamp, and Samuel Navas Medrano. 2019. WONDER -- Enhancing VR Training with Electrical Muscle Stimulation. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW0232:1--LBW0232:6.

Digital Library

[133]

Ken Pfeuffer, Matthias J. Geiger, Sarah Prange, Lukas Mecke, Daniel Buschek, and Florian Alt. 2019. Behavioural Biometrics in VR: Identifying People from Body Motion and Relations in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 110:1--110:12.

Digital Library

[134]

Dario Pittera, Elia Gatti, and Marianna Obrist. 2019. I'm Sensing in the Rain: Spatial Incongruity in Visual-Tactile Mid-Air Stimulation Can Elicit Ownership in VR Users. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 132:1--132:15.

Digital Library

[135]

Thammathip Piumsomboon, Gun A. Lee, Jonathon D. Hart, Barrett Ens, Robert W. Lindeman, Bruce H. Thomas, and Mark Billinghurst. 2018. Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 46:1--46:13.

Digital Library

[136]

Rüdiger Pohl and Rüdiger F. Pohl. 2004. Cognitive Illusions: A Handbook on Fallacies and Biases in Thinking, Judgement and Memory. Psychology Press.

[137]

Susanne Putze, Dmitry Alexandrovsky, Felix Putze, Sebastian Höffner, Jan D. Smeddinck, and Rainer Malaka. 2020. Breaking The Experience: Effects of Questionnaires in VR User Studies. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Honolulu, HI, USA.

Digital Library

[138]

J. T. Quaglia and A. Holecek. 2018. Lucid Virtual Dreaming: Antecedents and Consequents of Virtual Lucidity During Virtual Threat. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 1--72.

[139]

Nimesha Ranasinghe, Pravar Jain, Shienny Karwita, David Tolley, and Ellen Yi-Luen Do. 2017. Ambiotherm: Enhancing Sense of Presence in Virtual Reality by Simulating Real-World Environmental Conditions. ACM Press, 1731--1742.

Digital Library

[140]

Michael Rietzler, Florian Geiselhart, Julian Frommel, and Enrico Rukzio. 2018. Conveying the Perception of Kinesthetic Feedback in Virtual Reality Using State-of-the-Art Hardware. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 460:1--460:13.

Digital Library

[141]

Michael Rietzler, Florian Geiselhart, and Enrico Rukzio. 2017. The Matrix Has You: Realizing Slow Motion in Full-Body Virtual Reality. ACM Press, 1--10.

Digital Library

[142]

Katja Rogers, Jana Funke, Julian Frommel, Sven Stamm, and Michael Weber. 2019. Exploring Interaction Fidelity in Virtual Reality: Object Manipulation and Whole-Body Movements. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 414:1--414:14.

Digital Library

[143]

Katja Rogers, Giovanni Ribeiro, Rina R. Wehbe, Michael Weber, and Lennart E. Nacke. 2018. Vanishing Importance: Studying Immersive Effects of Game Audio Perception on Player Experiences in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 328:1--328:13.

Digital Library

[144]

Robert Rosenthal and Ralph L. Rosnow. 1984. Essentials of Behavioral Research: Methods and Data Analysis. McGraw-Hill, Inc.

[145]

D. Roth, C. Klelnbeck, T. Feigl, C. Mutschler, and M. E. Latoschik. 2018. Beyond Replication: Augmenting Social Behaviors in Multi-User Virtual Realities. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 215--222.

[146]

Barbara Olasov Rothbaum, Larry Hodges, Samantha Smith, Jeong Hwan Lee, and Larry Price. 2000. A Controlled Study of Virtual Reality Exposure Therapy for the Fear of Flying. Journal of Consulting and Clinical Psychology 68, 6 (2000), 1020--1026.

[147]

Roy A. Ruddle, Stephen J. Payne, and Dylan M. Jones. 1999. Navigating Large-Scale Virtual Environments: What Differences Occur Between Helmet-Mounted and Desk-Top Displays? Presence: Teleoperators and Virtual Environments 8, 2 (April 1999), 157--168.

Digital Library

[148]

Majed Samad, Elia Gatti, Anne Hermes, Hrvoje Benko, and Cesare Parise. 2019. Pseudo-Haptic Weight: Changing the Perceived Weight of Virtual Objects By Manipulating Control-Display Ratio. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 320:1--320:13.

Digital Library

[149]

James B. Sampson. 1993. Cognitive Performance of Individuals Using a Head-Mounted Display While Walking. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 37, 4 (Oct. 1993), 338--342.

[150]

Maria V. Sanchez-Vives and Mel Slater. 2005. From Presence to Consciousness through Virtual Reality. Nature Reviews Neuroscience 6, 4 (April 2005), 332.

[151]

Shyam Prathish Sargunam, Kasra Rahimi Moghadam, Mohamed Suhail, and Eric D. Ragan. 2017. Guided Head Rotation and Amplified Head Rotation: Evaluating Semi-Natural Travel and Viewing Techniques in Virtual Reality. IEEE, 19--28.

[152]

Willem E. Saris and Irmtraud N. Gallhofer. 2014. Design, Evaluation, and Analysis of Questionnaires for Survey Research. John Wiley & Sons.

[153]

Samuel B. Schorr and Allison M. Okamura. 2017. Fingertip Tactile Devices for Virtual Object Manipulation and Exploration. ACM Press, 3115--3119.

Digital Library

[154]

Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 1999. Embodied Presence in Virtual Environments. In Visual Representations and Interpretations, Ray Paton and Irene Neilson (Eds.). Springer London, 269--278.

[155]

Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The Experience of Presence: Factor Analytic Insights. Presence: Teleoper. Virtual Environ. 10, 3 (June 2001), 266--281.

Digital Library

[156]

Peter Schulz, Dmitry Alexandrovsky, Felix Putze, Rainer Malaka, and Johannes Schöning. 2019. The Role of Physical Props in VR Climbing Environments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 183:1--183:13.

Digital Library

[157]

Valentin Schwind. 2018. Implications of the Uncanny Valley of Avatars and Virtual Characters for Human-Computer Interaction. (2018).

[158]

Valentin Schwind, Pascal Knierim, Lewis Chuang, and Niels Henze. 2017. "Where's Pinky?": The Effects of a Reduced Number of Fingers in Virtual Reality. ACM Press, 507--515.

Digital Library

[159]

Valentin Schwind, Pascal Knierim, Nico Haas, and Niels Henze. 2019. Using Presence Questionnaires in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 360:1--360:12.

Digital Library

[160]

Valentin Schwind, Pascal Knierim, Cagri Tasci, Patrick Franczak, Nico Haas, and Niels Henze. 2017. "These Are Not My Hands!": Effect of Gender on the Perception of Avatar Hands in Virtual Reality. ACM Press, 1577--1582.

Digital Library

[161]

Valentin Schwind, Sven Mayer, Alexandre Comeau-Vermeersch, Robin Schweigert, and Niels Henze. 2018. Up to the Finger Tip: The Effect of Avatars on Mid-Air Pointing Accuracy in Virtual Reality. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, Melbourne, VIC, Australia, 477--488.

Digital Library

[162]

Sven Seele, Sebastian Misztal, Helmut Buhler, Rainer Herpers, and Jonas Schild. 2017. Here's Looking At You Anyway!: How Important Is Realistic Gaze Behavior in Co-Located Social Virtual Reality Games? ACM Press, 531--540.

Digital Library

[163]

John J. Shaughnessy and Eugene B. Zechmeister. 1985. Research Methods in Psychology. Alfred A. Knopf, New York, NY, US.

[164]

Emily Shaw, Tessa Roper, Tommy Nilsson, Glyn Lawson, Sue V.G. Cobb, and Daniel Miller. 2019. The Heat Is On: Exploring User Behaviour in a Multisensory Virtual Environment for Fire Evacuation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 626:1--626:13.

Digital Library

[165]

Lindsay Alexander Shaw, Burkhard Claus WÃ?nsche, Christof Lutteroth, S. Marks, and Rodolphe Callies. 2015. Challenges in Virtual Reality Exergame Design. 162 (Jan. 2015).

[166]

Thomas B. Sheridan. 1992. Musings on Telepresence and Virtual Presence. Presence: Teleoperators and Virtual Environments 1, 1 (Jan. 1992), 120--126.

Digital Library

[167]

Jotaro Shigeyama, Takeru Hashimoto, Shigeo Yoshida, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2019. Transcalibur: A Weight Shifting Virtual Reality Controller for 2D Shape Rendering Based on Computational Perception Model. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 11:1--11:11.

Digital Library

[168]

Ben Shneiderman, Catherine Plaisant, Maxine Cohen, Steven Jacobs, Niklas Elmqvist, and Nicholas Diakopoulos. 2016. Designing the User Interface: Strategies for Effective Human-Computer Interaction (6 edition ed.). Pearson, Boston.

[169]

Valerie J. Shute. 2011. Stealth Assessment in Computer-Based Games to Support Learning. Computer games and instruction 55, 2 (2011), 503--524.

[170]

Mel Slater. 2009. Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments. Philosophical Transactions of the Royal Society B: Biological Sciences 364, 1535 (Dec. 2009), 3549--3557.

[171]

Mel Slater and Anthony Steed. 2000. A Virtual Presence Counter. Presence: Teleoperators and Virtual Environments 9, 5 (2000), 413--434.

Digital Library

[172]

Mel Slater, Martin Usoh, and Yiorgos Chrysanthou. 1995. The Influence of Dynamic Shadows on Presence in Immersive Virtual Environments. In Virtual Environments '95 (Eurographics), Martin Göbel (Ed.). Springer Vienna, 8--21.

[173]

Jeongmin Son, Sunggeun Ahn, Sunbum Kim, and Geehyuk Lee. 2019. Improving Two-Thumb Touchpad Typing in Virtual Reality. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW2217:1--LBW2217:6.

[174]

Maurício Sousa, Daniel Mendes, Soraia Paulo, Nuno Matela, Joaquim Jorge, and Daniel Simões Lopes. 2017. VRRRRoom: Virtual Reality for Radiologists in the Reading Room. ACM Press, 4057--4062.

Digital Library

[175]

Beatriz Sousa Santos, Paulo Dias, Angela Pimentel, Jan-Willem Baggerman, Carlos Ferreira, Samuel Silva, and Joaquim Madeira. 2008. Head-Mounted Display versus Desktop for 3D Navigation in Virtual Reality: A User Study. Multimedia Tools and Applications 41, 1 (Aug. 2008), 161.

[176]

Marco Speicher, Jan Ehrlich, Vito Gentile, Donald Degraen, Salvatore Sorce, and Antonio Krüger. 2019. Pseudo-Haptic Controls for Mid-Air Finger-Based Menu Interaction. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW0285:1--LBW0285:6.

[177]

Marco Speicher, Anna Maria Feit, Pascal Ziegler, and Antonio Krüger. 2018. Selection-Based Text Entry in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 647:1--647:13.

Digital Library

[178]

Misha Sra, Xuhai Xu, and Pattie Maes. 2018. BreathVR: Leveraging Breathing As a Directly Controlled Interface for Virtual Reality Games. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 340:1--340:12.

Digital Library

[179]

J. Stauffert, F. Niebling, and M. E. Latoschik. 2018. Effects of Latency Jitter on Simulator Sickness in a Search Task. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 121--127.

[180]

Carolin Straßmann, Sabrina C. Eimler, Alexander Arntz, Dustin Keßler, Sarah Zielinski, Gabriel Brandenberg, Vanessa Dümpel, and Uwe Handmann. 2019. Relax Yourself - Using Virtual Reality to Enhance Employees' Mental Health and Work Performance. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, Glasgow, Scotland Uk, LBW0286:1--LBW0286:6.

Digital Library

[181]

Hemant Bhaskar Surale, Aakar Gupta, Mark Hancock, and Daniel Vogel. 2019a. TabletInVR: Exploring the Design Space for Using a Multi-Touch Tablet in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 13:1--13:13.

Digital Library

[182]

Hemant Bhaskar Surale, Fabrice Matulic, and Daniel Vogel. 2019b. Experimental Analysis of Barehand Mid-Air Mode-Switching Techniques in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 196:1--196:14.

Digital Library

[183]

Penelope Sweetser and Peta Wyeth. 2005. GameFlow: A Model for Evaluating Player Enjoyment in Games. Comput. Entertain. 3, 3 (July 2005), 3--3.

Digital Library

[184]

Theophilus Teo, Louise Lawrence, Gun A. Lee, Mark Billinghurst, and Matt Adcock. 2019. Mixed Reality Remote Collaboration Combining 360 Video and 3D Reconstruction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 201:1--201:14.

Digital Library

[185]

Balasaravanan Thoravi Kumaravel, Cuong Nguyen, Stephen DiVerdi, and Björn Hartmann. 2019. TutoriVR: A Video-Based Tutorial System for Design Applications in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 284:1--284:12.

Digital Library

[186]

Tanh Quang Tran, HyunJu Shin, Wolfgang Stuerzlinger, and JungHyun Han. 2017. Effects of Virtual Arm Representations on Interaction in Virtual Environments. ACM Press, 1--9.

Digital Library

[187]

Sam Tregillus, Majed Al Zayer, and Eelke Folmer. 2017. Handsfree Omnidirectional VR Navigation Using Head Tilt. ACM Press, 4063--4068.

Digital Library

[188]

Hsin-Ruey Tsai, Jun Rekimoto, and Bing-Yu Chen. 2019. ElasticVR: Providing Multilevel Continuously-Changing Resistive Force and Instant Impact Using Elasticity for VR. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 220:1--220:10.

Digital Library

[189]

Martin Usoh, Ernest Catena, Sima Arman, and Mel Slater. 2000. Using Presence Questionnaires in Reality. Presence: Teleoperators and Virtual Environments 9, 5 (2000), 497--503.

Digital Library

[190]

Adrien Verhulst, Jean-Marie Normand, Cindy Lombart, and Guillaume Moreau. 2017. A Study on the Use of an Immersive Virtual Reality Store to Investigate Consumer Perceptions and Purchase Behavior toward Non-Standard Fruits and Vegetables. IEEE, 55--63.

[191]

Alexandra Voit, Sven Mayer, Valentin Schwind, and Niels Henze. 2019. Online, VR, AR, Lab, and In-Situ: Comparison of Research Methods to Evaluate Smart Artifacts. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 507:1--507:12.

Digital Library

[192]

Emanuel Vonach, Clemens Gatterer, and Hannes Kaufmann. 2017. VRRobot: Robot Actuated Props in an Infinite Virtual Environment. IEEE, 74--83.

[193]

Daniel Wagner, Christian Hofmann, Heiko Hamann, and Sebastian von Mammen. 2017. Design and Exploration of Braiding Swarms in VR. ACM Press, 1--4.

Digital Library

[194]

Ryan Wedoff, Lindsay Ball, Amelia Wang, Yi Xuan Khoo, Lauren Lieberman, and Kyle Rector. 2019. Virtual Showdown: An Accessible Virtual Reality Game with Scaffolds for Youth with Visual Impairments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 141:1--141:15.

Digital Library

[195]

Florian Weidner, Anne Hoesch, Sandra Poeschl, and Wolfgang Broll. 2017. Comparing VR and Non-VR Driving Simulations: An Experimental User Study. IEEE, 281--282.

[196]

T. Weißker, A. Kunert, B. Fröhlich, and A. Kulik. 2018. Spatial Updating and Simulator Sickness During Steering and Jumping in Immersive Virtual Environments. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 97--104.

[197]

Martin Westhoven, Dennis Paul, and Thomas Alexander. 2016. Head Turn Scaling below the Threshold of Perception in Immersive Virtual Environments. ACM Press, 77--86.

Digital Library

[198]

C. Wienrich, K. Schindler, N. Döllinqer, S. Kock, and O. Traupe. 2018. Social Presence and Cooperation in Large-Scale Multi-User Virtual Reality - The Relevance of Social Interdependence for Location-Based Environments. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 207--214.

[199]

Graham Wilson and Mark McGill. 2018. Violent Video Games in Virtual Reality: Re-Evaluating the Impact and Rating of Interactive Experiences. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, Melbourne, VIC, Australia, 535--548.

Digital Library

[200]

Graham Wilson, Mark McGill, Matthew Jamieson, Julie R. Williamson, and Stephen A. Brewster. 2018. Object Manipulation in Virtual Reality Under Increasing Levels of Translational Gain. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 99:1--99:13.

Digital Library

[201]

Bob G. Witmer and Michael J. Singer. 1998. Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments 7, 3 (June 1998), 225--240.

Digital Library

[202]

Soojeong Yoo and Judy Kay. 2016. VRun: Running-in-Place Virtual Reality Exergame. In Proceedings of the 28th Australian Conference on Computer-Human Interaction (OzCHI '16). ACM, NY, NY, USA, 562--566.

Digital Library

[203]

Majed Al Zayer. 2019. Universal Usability of Virtual Reality. Dissertation. University of Nevada, Reno, Nevada, Reno.

[204]

André Zenner and Antonio Krüger. 2019. Drag:On: A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight Shift. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 211:1--211:12.

Digital Library

[205]

Yuhang Zhao, Edward Cutrell, Christian Holz, Meredith Ringel Morris, Eyal Ofek, and Andrew D. Wilson. 2019. SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 111:1--111:14.

Cited By

Safikhani SGattringer VSchmied MPirker JWriessnegger S(2024)The Influence of Realism on the Sense of Presence in Virtual Reality: Neurophysiological Insights Using EEGMultimodal Technologies and Interaction10.3390/mti81101048:11(104)Online publication date: 19-Nov-2024
https://doi.org/10.3390/mti8110104
Tran TLanglotz TYoung JSchubert TRegenbrecht H(2024)Classifying Presence Scores: Insights and Analysis from Two Decades of the Igroup Presence Questionnaire (IPQ)ACM Transactions on Computer-Human Interaction10.1145/368904631:5(1-26)Online publication date: 17-Aug-2024
https://dl.acm.org/doi/10.1145/3689046
Al Kassm NHenderson JTeather R(2024)Administering VR Questionnaires Generated in Google FormsProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3688888(1-2)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3688888
Show More Cited By

Index Terms

Examining Design Choices of Questionnaires in VR User Studies
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Breaking The Experience: Effects of Questionnaires in VR User Studies
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Questionnaires are among the most common research tools in virtual reality (VR) evaluations and user studies. However, transitioning from virtual worlds to the physical world to respond to VR experience questionnaires can potentially lead to systematic ...
A Pilot Study Examining the Unexpected Vection Hypothesis of Cybersickness.
VRST '21: Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology

The relationship between vection (illusory self-motion) and cybersickness is complex. This pilot study examined whether only unexpected vection provokes sickness during head-mounted display (HMD) based virtual reality (VR). 20 participants ran through ...
Building and Employing Cross-Reality

This issue's Works in Progress department lists five projects with a focus on cross reality. The first two involve using Project Wonderland for collaborative projects. The next three use virtual and augmented reality to enhance user capabilities and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

46
Total Citations
View Citations
2,491
Total Downloads

Downloads (Last 12 months)387
Downloads (Last 6 weeks)30

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Safikhani SGattringer VSchmied MPirker JWriessnegger S(2024)The Influence of Realism on the Sense of Presence in Virtual Reality: Neurophysiological Insights Using EEGMultimodal Technologies and Interaction10.3390/mti81101048:11(104)Online publication date: 19-Nov-2024
https://doi.org/10.3390/mti8110104
Tran TLanglotz TYoung JSchubert TRegenbrecht H(2024)Classifying Presence Scores: Insights and Analysis from Two Decades of the Igroup Presence Questionnaire (IPQ)ACM Transactions on Computer-Human Interaction10.1145/368904631:5(1-26)Online publication date: 17-Aug-2024
https://dl.acm.org/doi/10.1145/3689046
Al Kassm NHenderson JTeather R(2024)Administering VR Questionnaires Generated in Google FormsProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3688888(1-2)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3688888
Küntzer LSchwab SSpaderna HRock G(2024)ROVER: A Standalone Overlay Tool for Questionnaires in Virtual RealityCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661328(31-39)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3661328
Young JFerreira JPantidi N(2024)"I Shot the Interviewer!": The Effects of In-VR Interviews on Participant Feedback and RapportProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642707(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642707
Martinez EMalik AMcMahan R(2024)CLOVR: Collecting and Logging OpenVR Data from SteamVR Applications2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00095(485-492)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00095
Küntzer LSchwab SSpaderna HRock G(2024)Measuring User Experience of Older Adults during Virtual Reality Exergaming2024 16th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX61742.2024.10598263(153-159)Online publication date: 18-Jun-2024
https://doi.org/10.1109/QoMEX61742.2024.10598263
Ramousse FRaimbaud PBaert PHelfenstein-Didier CGay AMassoubre CGalusca BLavoué G(2023)Does this virtual food make me hungry? effects of visual quality and food type in virtual realityFrontiers in Virtual Reality10.3389/frvir.2023.12216514Online publication date: 14-Aug-2023
https://doi.org/10.3389/frvir.2023.1221651
Bonfert MHübinger MMalaka R(2023)Challenges of controlling the rotation of virtual objects with variable grip using force-feedback glovesFrontiers in Virtual Reality10.3389/frvir.2023.11904264Online publication date: 9-Oct-2023
https://doi.org/10.3389/frvir.2023.1190426
Guy MNormand JJeunet-Kelway CMoreau G(2023)The sense of embodiment in Virtual Reality and its assessment methodsFrontiers in Virtual Reality10.3389/frvir.2023.11416834Online publication date: 5-Dec-2023
https://doi.org/10.3389/frvir.2023.1141683
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents