Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3173574.3174227acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

The Effect of Offset Correction and Cursor on Mid-Air Pointing in Real and Virtual Environments

Published: 21 April 2018 Publication History

Abstract

Pointing at remote objects to direct others' attention is a fundamental human ability. Previous work explored methods for remote pointing to select targets. Absolute pointing techniques that cast a ray from the user to a target are affected by humans' limited pointing accuracy. Recent work suggests that accuracy can be improved by compensating systematic offsets between targets a user aims at and rays cast from the user to the target. In this paper, we investigate mid-air pointing in the real world and virtual reality. Through a pointing study, we model the offsets to improve pointing accuracy and show that being in a virtual environment affects how users point at targets. In the second study, we validate the developed model and analyze the effect of compensating systematic offsets. We show that the provided model can significantly improve pointing accuracy when no cursor is provided. We further show that a cursor improves pointing accuracy but also increases the selection time.

Supplementary Material

suppl.mov (pn4903-file5.mp4)
Supplemental video

References

[1]
Deepak Akkil and Poika Isokoski. 2016. Accuracy of Interpreting Pointing Gestures in Egocentric View. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16). ACM, New York, NY, USA, 262--273.
[2]
Ferran Argelaguet and Carlos Andujar. 2013. A survey of 3D object selection techniques for virtual environments. Computers & Graphics 37, 3 (2013), 121--136.
[3]
Ferran Argelaguet, Carlos Andujar, and Ramon Trueba. 2008. Overcoming Eye-hand Visibility Mismatch in 3D Pointing Selection. In Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology (VRST '08). ACM, New York, NY, USA, 43--46.
[4]
Ferran Argelaguet, Ludovic Hoyet, Michaël Trico, and Anatole Lécuyer. 2016. The role of interaction in virtual embodiment: Effects of the virtual hand representation. In Proceedings of the 2016 IEEE Virtual Reality Conference (IEEE/VR '16). Greenville, SC, USA, 3--10.
[5]
Tiziana Aureli, Maria Spinelli, Mirco Fasolo, Maria Concetta Garito, Paola Perucchini, and Laura D'Odorico. 2017. The Pointing-Vocal Coupling Progression in the First Half of the Second Year of Life. Infancy (2017), 18.
[6]
John V. Basmajian and Carlo J. De Luca. 1985. Muscles alive: their functions revealed by electromyography. Williams & Wilkins, Baltimore, London, Sydney. http://books.google.de/books?id=H9pqAAAAMAAJ
[7]
Scott Bateman, Regan L. Mandryk, Carl Gutwin, and Robert Xiao. 2013. Analysis and comparison of target assistance techniques for relative ray-cast pointing. International Journal of Human-Computer Studies 71, 5 (2013), 511--532.
[8]
Elizabeth Bates. 1979. The Emergence of Symbols: Cognition and Communication in Infancy. Academic Press. 404 pages. https://books.google.de/books?id=_45-AAAAMAAJ
[9]
Hrvoje Benko, Andrew D. Wilson, and Federico Zannier. 2014. Dyadic Projected Spatial Augmented Reality. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 645--655.
[10]
Richard A. Bolt. 1980. Put-that-there: Voice and gesture at the graphics interface. In Proceedings of the 7th annual conference on Computer graphics and interactive techniques (SIGGRAPH '80). ACM, New York, NY, USA, 262--270.
[11]
Jerome S. Bruner. 1975. The ontogenesis of speech acts. Journal of Child Language 2, 1 (1975), 1--19.
[12]
Luigia Camaioni, Paola Perucchini, Francesca Bellagamba, and Cristina Colonnesi. 2004. The Role of Declarative Pointing in Developing a Theory of Mind. Infancy 5, 3 (2004), 291--308.
[13]
C. N. Christakos and S. Lal. 1980. Lumped and population stochastic models of skeletal muscle: Implications and predictions. Biological Cybernetics 36, 2 (01 Feb 1980), 73--85.
[14]
Andy Cockburn, Philip Quinn, Carl Gutwin, Gonzalo Ramos, and Julian Looser. 2011. Air pointing: Design and evaluation of spatial target acquisition with and without visual feedback. International Journal of Human-Computer Studies 69, 6 (2011), 401--414.
[15]
Andrea Corradini and Philip R. Cohen. 2002. Multimodal speech-gesture interface for handfree painting on a virtual paper using partial recurrent neural networks as gesture recognizer. (2002).
[16]
Giambattista Della Porta. 1593. De refractione Optices Parte: Libri Novem... Ex officina Horatii Salviani, apud Jo. Jacobum Carlinum, & Antonium Pacem.
[17]
H. Henrik Ehrsson, Charles Spence, and Richard E. Passingham. 2004. That's My Hand! Activity in Premotor Cortex Reflects Feeling of Ownership of a Limb. Science 305, 5685 (2004), 875--877.
[18]
Rodger J. Elble and William C. Koller. 1990. Tremor. Johns Hopkins University Press, Baltimore. https://google.com/books?id=-LdrAAAAMAAJ
[19]
J. M. Foley and Richard Held. 1972. Visually directed pointing as a function of target distance, direction, and available cues. Perception & Psychophysics 12, 3 (01 May 1972), 263--268.
[20]
Edward G. Freedman and David L. Sparks. 2000. Coordination of the eyes and head: movement kinematics. Experimental Brain Research 131, 1 (01 Mar 2000), 22--32.
[21]
Michael Steven Anthony Graziano. 1999. Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position. Proceedings of the National Academy of Sciences 96, 18 (1999), 10418--10421.
[22]
Michael S. A. Graziano, Dylan F. Cooke, and Charlotte S. R. Taylor. 2000. Coding the Location of the Arm by Sight. Science 290 (2000), 1782--1786. http://www.jstor.org/stable/3081775
[23]
Jan Gugenheimer, Pascal Knierim, Christian Winkler, Julian Seifert, and Enrico Rukzio. 2015. UbiBeam: Exploring the Interaction Space for Home Deployed Projector-Camera Systems. In IFIP TC 13 International Conference on Human-Computer Interaction (INTERACT 2015). Springer International Publishing, Cham, 350--366.
[24]
Sandra G. Hart. 2006. NASA-Task Load Index (NASA-TLX); 20 years later. In Proceedings of the Human Factors and Ergonomic Society annual meeting, Vol. 50. SAGE Publications, SAGE Publications, Los Angeles, CA, USA, 904--908.
[25]
John B. Haviland. 2003. How to point in Zinacantán. Psychology Press, Mahwah, New Jersey, USA, 139--169.
[26]
Hao Jiang, Eyal Ofek, Neema Moraveji, and Yuanchun Shi. 2006. Direct Pointer: Direct Manipulation for Large-display Interaction Using Handheld Cameras. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, New York, NY, USA, 1107--1110.
[27]
Ricardo Jota, Miguel A. Nacenta, Joaquim A. Jorge, Sheelagh Carpendale, and Saul Greenberg. 2010. A Comparison of Ray Pointing Techniques for Very Large Displays. In Proceedings of Graphics Interface 2010 (GI '10). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 269--276. http://dl.acm.org/citation.cfm?id=1839214.1839261
[28]
Adam Kendon. 2004. Gesture: Visible Action as Utterance. Cambridge University Press, Cambridge. IX, 400 S. pages. https://books.google.de/books?id=WY2bBQAAQBAJ
[29]
Aarlenne Z. Khan and J. Douglas Crawford. 2003. Coordinating one hand with two eyes: optimizing for field of view in a pointing task. Vision Research 43, 4 (2003), 409--417.
[30]
Pascal Knierim, Valentin Schwind, Anna Feit, Florian Nieuwenhuizen, and Niels Henze. 2018. Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA.
[31]
Regis Kopper, Doug A. Bowman, Mara G. Silva, and Ryan P. McMahan. 2010. A Human Motor Behavior Model for Distal Pointing Tasks. International Journal of Human-Computer Studies 68, 10 (Oct. 2010), 603--615.
[32]
Alfred Kranstedt, Andy Lücking, Thies Pfeiffer, Hannes Rieser, and Marc Staudacher. 2006. Measuring and Reconstructing Pointing in Visual Contexts. In Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue (brandial'06). Universitätsverlag Potsdam, Potsdam, 82--89. http://opus.kobv.de/ubp/ volltexte/2006/1048/pdf/brandial06_proceedings.pdf
[33]
Lorraine Lin and Sophie Jörg. 2016. Need a hand? How Appearance Affects the Virtual Hand Illusion. In Proceedings of the ACM Symposium on Applied Perception (SAP '16). ACM Press, New York, New York, USA, 69--76.
[34]
Lars Lischke, Pascal Knierim, and Hermann Klinke. 2015. Mid-Air Gestures for Window Management on Large Displays. In Mensch und Computer 2015 Tagungsband. De Gruyter Oldenbourg, Berlin, 439--442.
[35]
Lars Lischke, Valentin Schwind, Kai Friedrich, Albrecht Schmidt, and Niels Henze. 2016. MAGIC-Pointing on Large High-Resolution Displays. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, New York, NY, USA, 1706--1712.
[36]
I. Scott MacKenzie and Shaidah Jusoh. 2001. An Evaluation of Two Input Devices for Remote Pointing. In Engineering for Human-Computer Interaction: 8th IFIP International Conference (EHCI'01). Springer Berlin Heidelberg, Berlin, Heidelberg, 235--250.
[37]
Ville Mäkelä, Tomi Heimonen, Matti Luhtala, and Markku Turunen. 2014. Information Wall: Evaluation of a Gesture-controlled Public Display. In Proceedings of the 13th International Conference on Mobile and Ubiquitous Multimedia (MUM '14). ACM, New York, NY, USA, 228--231.
[38]
Anders Markussen, Sebastian Boring, Mikkel R. Jakobsen, and Kasper Hornbæk. 2016. Off-Limits: Interacting Beyond the Boundaries of Large Displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5862--5873.
[39]
Anders Markussen, Mikkel Rønne Jakobsen, and Kasper Hornbæk. 2014. Vulture: A Mid-air Word-gesture Keyboard. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1073--1082.
[40]
Sven Mayer, Katrin Wolf, Stefan Schneegass, and Niels Henze. 2015. Modeling Distant Pointing for Compensating Systematic Displacements. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 4165--4168.
[41]
Victoria McArthur, Steven J. Castellucci, and I. Scott MacKenzie. 2009. An Empirical Comparison of "Wiimote" Gun Attachments for Pointing Tasks. In Proceedings of the 1st ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '09). ACM, New York, NY, USA, 203--208.
[42]
Samuel Navas Medrano, Max Pfeiffer, and Christian Kray. 2017. Enabling Remote Deictic Communication with Mobile Devices: An Elicitation Study. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, New York, NY, USA, Article 19, 13 pages.
[43]
Walter R. Miles. 1930. Ocular Dominance in Human Adults. The journal of general psychology 3, 3 (1930), 412--430.
[44]
Paul Milgram and Fumio Kishino. 1994. A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems 77, 12 (1994), 1321--1329. http://ci.nii.ac.jp/naid/110003209335/en/
[45]
Mark R. Mine. 1995. Virtual Environment Interaction Techniques. Technical Report. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA. http://www.cs.unc.edu/techreports/95-018.pdf
[46]
Mark R. Mine, Frederick P. Brooks Jr., and Carlo H. Sequin. 1997. Moving Objects in Space: Exploiting Proprioception in Virtual-environment Interaction. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '97). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 19--26.
[47]
Steven Morrison and J. Keogh. 2001. Changes in the dynamics of tremor during goal-directed pointing. Human Movement Science 20, 4 (2001), 675--693.
[48]
Mathieu Nancel, Emmanuel Pietriga, Olivier Chapuis, and Michel Beaudouin-Lafon. 2015. Mid-Air Pointing on Ultra-Walls. ACM Trans. Comput.-Hum. Interact. 22, 5, Article 21 (Aug. 2015), 62 pages.
[49]
Kai Nickel and Rainer Stiefelhagen. 2003. Pointing Gesture Recognition Based on 3D-tracking of Face, Hands and Head Orientation. In Proceedings of the 5th International Conference on Multimodal Interfaces (ICMI '03). ACM, New York, NY, USA, 140--146.
[50]
Dan R. Olsen, Jr. and Travis Nielsen. 2001. Laser Pointer Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, New York, NY, USA, 17--22.
[51]
Jeni Paay, Dimitrios Raptis, Jesper Kjeldskov, Mikael B. Skov, Eric V. Ruder, and Bjarke M. Lauridsen. 2017. Investigating Cross-Device Interaction Between a Handheld Device and a Large Display. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 6608--6619.
[52]
Jeffrey S. Pierce, Andrew S. Forsberg, Matthew J. Conway, Seung Hong, Robert C. Zeleznik, and Mark R. Mine. 1997. Image Plane Interaction Techniques in 3D Immersive Environments. In Proceedings of the 1997 Symposium on Interactive 3D Graphics (I3D '97). ACM, New York, NY, USA, 39--ff.
[53]
Katrin Plaumann, Matthias Weing, Christian Winkler, Michael Müller, and Enrico Rukzio. 2017. Towards accurate cursorless pointing: the effects of ocular dominance and handedness. Personal and Ubiquitous Computing (07 Dec 2017).
[54]
Cameron N. Riviere, R. Scott Rader, and Pradeep K. Khosla. 1997. Characteristics of hand motion of eye surgeons. In Proceedings of the 19th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE/EMBS '97), Vol. 4. IEEE, Chicago, IL, USA, USA, 1690--1693.
[55]
Valentin Schwind, Pascal Knierim, Cagri Tasci, Patrick Franczak, Nico Haas, and Niels Henze. 2017. "These Are Not My Hands!": Effect of Gender on the Perception of Avatar Hands in Virtual Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 1577--1582.
[56]
Mel Slater, Daniel Perez-marcos, and H. Henrik Ehrsson. 2009. Inducing illusory ownership of a virtual body. Frontiers in neuroscience 3, 2 (Sep 2009), 214--220.
[57]
R. William Soukoreff and I. Scott MacKenzie. 2004. Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts' law research in HCI. International Journal of Human-Computer Studies 61, 6 (2004), 751--789.
[58]
John S. Stahl. 1999. Amplitude of human head movements associated with horizontal saccades. Experimental Brain Research 126, 1 (01 Apr 1999), 41--54.
[59]
Daniel Vogel and Ravin Balakrishnan. 2004. Interactive Public Ambient Displays: Transitioning from Implicit to Explicit, Public to Personal, Interaction with Multiple Users. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST '04). ACM, New York, NY, USA, 137--146.
[60]
Daniel Vogel and Ravin Balakrishnan. 2005. Distant Freehand Pointing and Clicking on Very Large, High Resolution Displays. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology (UIST '05). ACM, New York, NY, USA, 33--42.
[61]
Christian Winkler, Ken Pfeuffer, and Enrico Rukzio. 2012. Investigating Mid-air Pointing Interaction for Projector Phones. In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces (ITS '12). ACM, New York, NY, USA, 85--94.
[62]
Nelson Wong and Carl Gutwin. 2014. Support for Deictic Pointing in CVEs: Still Fragmented After All These Years'. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14). ACM, New York, NY, USA, 1377--1387.
[63]
Shumin Zhai. 1998. User Performance in Relation to 3D Input Device Design. ACM SIGGRAPH Computer Graphics 32, 4 (Nov. 1998), 50--54.

Cited By

View all
  • (2024)TouchView: Mid-Air Touch on Zoomable 2D View for Distant Freehand Selection on a Virtual Reality User InterfaceSensors10.3390/s2422720224:22(7202)Online publication date: 11-Nov-2024
  • (2024)Experimental Analysis of Freehand Multi-object Selection Techniques in Virtual Reality Head-Mounted DisplaysProceedings of the ACM on Human-Computer Interaction10.1145/36981298:ISS(93-111)Online publication date: 24-Oct-2024
  • (2024)To Stick or Not to Stick? Studying the Impact of Offset Recovery Techniques During Mid-Air InteractionsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329520930:8(5493-5506)Online publication date: Aug-2024
  • Show More Cited By

Index Terms

  1. The Effect of Offset Correction and Cursor on Mid-Air Pointing in Real and Virtual Environments

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
    April 2018
    8489 pages
    ISBN:9781450356206
    DOI:10.1145/3173574
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 April 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. cursor
    2. mid-air pointing
    3. modeling
    4. offset correction
    5. ray casting
    6. virtual environment

    Qualifiers

    • Research-article

    Conference

    CHI '18
    Sponsor:

    Acceptance Rates

    CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI '25
    CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)172
    • Downloads (Last 6 weeks)12
    Reflects downloads up to 18 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)TouchView: Mid-Air Touch on Zoomable 2D View for Distant Freehand Selection on a Virtual Reality User InterfaceSensors10.3390/s2422720224:22(7202)Online publication date: 11-Nov-2024
    • (2024)Experimental Analysis of Freehand Multi-object Selection Techniques in Virtual Reality Head-Mounted DisplaysProceedings of the ACM on Human-Computer Interaction10.1145/36981298:ISS(93-111)Online publication date: 24-Oct-2024
    • (2024)To Stick or Not to Stick? Studying the Impact of Offset Recovery Techniques During Mid-Air InteractionsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329520930:8(5493-5506)Online publication date: Aug-2024
    • (2024)Continuous Prediction of Pointing Targets With Motion and Eye-Tracking in Virtual RealityIEEE Access10.1109/ACCESS.2024.335078812(5933-5946)Online publication date: 2024
    • (2024)LaserDex: Improvising Spatial Tasks Using Deictic Gestures and Laser Pointing for Human–Robot Collaboration in ConstructionJournal of Computing in Civil Engineering10.1061/JCCEE5.CPENG-571538:3Online publication date: May-2024
    • (2023)Malawi open crowd sourced virtual reality experiencei-manager's Journal on Augmented & Virtual Reality10.26634/javr.1.2.200841:2(18)Online publication date: 2023
    • (2023)WorldPoint: Finger Pointing as a Rapid and Natural Trigger for In-the-Wild Mobile InteractionsProceedings of the ACM on Human-Computer Interaction10.1145/36264787:ISS(357-375)Online publication date: 1-Nov-2023
    • (2023)Going, Going, Gone: Exploring Intention Communication for Multi-User Locomotion in Virtual RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581259(1-13)Online publication date: 19-Apr-2023
    • (2023)When XR and AI Meet - A Scoping Review on Extended Reality and Artificial IntelligenceProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581072(1-45)Online publication date: 19-Apr-2023
    • (2023)Selecting Real-World Objects via User-Perspective Phone OcclusionProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580696(1-13)Online publication date: 19-Apr-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media