abstract

Webcam-based Hand- and Object-Tracking for a Desktop Workspace in Virtual Reality

Authors:

Sebastian Pape,

Jonathan Heinrich Beierle,

Torsten Wolfgang Kuhlen,

Tim WeisskerAuthors Info & Claims

SUI '24: Proceedings of the 2024 ACM Symposium on Spatial User Interaction

Article No.: 64, Pages 1 - 2

https://doi.org/10.1145/3677386.3688879

Published: 07 October 2024 Publication History

Get Access

Abstract

As virtual reality overlays the user’s view, challenges arise when interaction with their physical surroundings is still needed. In a seated workspace environment interaction with the physical surroundings can be essential to enable productive working. Interaction with e.g. physical mouse and keyboard can be difficult when no visual reference is given to where they are placed. This demo shows a combination of computer vision-based marker detection with machine-learning-based hand detection to bring users’ hands and arbitrary objects into VR.

References

[1]

Camillo Lugaresi et al.2019. MediaPipe: A Framework for Perceiving and Processing Reality. In Third Workshop on Computer Vision for AR/VR at IEEE Computer Vision and Pattern Recognition (CVPR) 2019.

Google Scholar

[2]

S. Garrido-Jurado, R. Muñoz-Salinas, F.J. Madrid-Cuevas, and M.J. Marín-Jiménez. 2014. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition 47, 6 (2014), 2280–2292. https://doi.org/10.1016/j.patcog.2014.01.005

Digital Library

Google Scholar

[3]

Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.

Google Scholar

[4]

Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and Evaluation of a User Experience Questionnaire. USAB 2008 5298, 63–76. https://doi.org/10.1007/978-3-540-89350-9_6

Digital Library

Google Scholar

Index Terms

Webcam-based Hand- and Object-Tracking for a Desktop Workspace in Virtual Reality
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Matching
        Tracking
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Keyboards
    2. Interaction paradigms
      1. Virtual reality

Recommendations

Physical hand interaction for controlling multiple virtual objects in virtual reality
IWISC '18: Proceedings of the 3rd International Workshop on Interactive and Spatial Computing

Immersive virtual reality systems often strive to support realistic physical interaction through direct hand control, but it is not often possible to configure the physical environment to exactly match the virtual environment. We study methods that ...
Enabling Reusable Haptic Props for Virtual Reality by Hand Displacement
MuC '21: Proceedings of Mensch und Computer 2021

Virtual Reality (VR) enables compelling visual experiences. However, providing haptic feedback is still challenging. Previous work suggests utilizing haptic props to overcome such limitations and presents evidence that props could function as a single ...
User-elicited dual-hand interactions for manipulating 3D objects in virtual reality environments

Virtual reality technologies (VR) have advanced rapidly in the last few years. Prime examples include the Oculus RIFT and HTC Vive that are both head-worn/mounted displays (HMDs). VR HMDs enable a sense of immersion and allow enhanced natural interaction ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SUI '24: Proceedings of the 2024 ACM Symposium on Spatial User Interaction

October 2024

396 pages

ISBN:9798400710889

DOI:10.1145/3677386

Editors:
Florian Daiber
DFKI, Saarbrücken, Germany
,
André Zenner
Saarland University, Saarbrücken, Germany
,
Tony Huang
UTS, Sydney, Australia
,
Daniel Zielasko
University of Trier, Trier, Germany
,
Huyen Nguyen
Université Paris-Saclay, Orsay, France
,
Mayra Donaji Barrera Machuca
Dalhousie University, Halifax, Canada

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2024

Check for updates

Author Tags

Qualifiers

Abstract
Research
Refereed limited

Conference

SUI '24

Sponsor:

SUI '24: ACM Symposium on Spatial User Interaction

October 7 - 8, 2024

Trier, Germany

Acceptance Rates

Overall Acceptance Rate 86 of 279 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
33
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)26

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

References

Index Terms

Recommendations

Physical hand interaction for controlling multiple virtual objects in virtual reality

Enabling Reusable Haptic Props for Virtual Reality by Hand Displacement

User-elicited dual-hand interactions for manipulating 3D objects in virtual reality environments

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Login options

Full Access

View options

PDF

eReader

HTML Format

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations