research-article

Bimanual natural user interaction for 3D modelling application using stereo computer vision

Authors:

Roy Sirui Yang,

Alfonso Gastélum Strozzi,

Christof Lutteroth,

Patrice DelmasAuthors Info & Claims

CHINZ '12: Proceedings of the 13th International Conference of the NZ Chapter of the ACM's Special Interest Group on Human-Computer Interaction

Pages 44 - 51

https://doi.org/10.1145/2379256.2379264

Published: 02 July 2012 Publication History

Abstract

This paper presents a system that allows the user to perform 3D modelling and sculpting using postures and 3D movements of their hands. The system utilises the concept of a Natural User Interface using computer vision techniques. This enables the user to operate 3D modelling software. The system's bimanual control allows left hand postures to select control mode commands, while the right hand controls movements. To evaluate the real world performance of the concept of motion and hand-posture-based control in 3D modelling, a usability test with 10 people was conducted. Participants were asked to perform test tasks that involved moving an object in 3D space. These participants performed the tasks multiple times while being timed, both with the mouse and using the 3D hand tracking system. The results indicated that participants who used the hand tracking system completed the tasks more quickly than those who used the mouse. However, approximately half of the participants reported that they found it easier to use the mouse than the hand tracking system. Overall, the participants reported that they enjoyed using the system.

References

[1]

A. K. Agnihitri, B. Purwar, N. Jeebun, and S. Agnihotri. Determination of sex by hand dimensions. The Internet Journal of Forensic Science, 1(2), 2006.

[2]

Blender.org. Blender official site, retrieved: Mar 2012. http:www.Blender.org.

[3]

G. Bradski. Computer vision face tracking for use in a perceptual user interface. Intel Tech. Journal, 1998.

[4]

N. H. Dardas and E. M. Petriu. Hand gesture detection and recognition using principal component analysis. In Computational Intelligence for Measurement Systems and Applications (CIMSA), 2011 IEEE International Conference on, pages 1--6, 2011.

[5]

B. Freedman, A. Shpunt, M. Machline, and Y. Arieli. Depth mapping using projected patterns, 2007. Patent, US0240502A1.

[6]

L. Graves, N. Ridgers, and G. Stratton. The contribution of upper limb and total body movement to adolescentsâĂ&Zacute; energy expenditure whilst playing nintendo wii. European Journal of Applied Physiology, 104(4):617--623, 2008.

[7]

L. Graves, G. Stratton, N. D. Ridgers, and N. T. Cable. Energy expenditure in adolescents playing new generation computer games. British Journal of Sports Medicine, 42(7):592--594, 2008.

[8]

D. Jiangwen and H. T. Tsui. A PCA/MDA scheme for hand posture recognition. In Proc. Automatic Face and Gesture Recognition Conference, pages 294--299, 2002.

Digital Library

[9]

T. Kurata, T. Okuma, M. Kourogi, T. Kato, and K. Sakaue. Vizwear: Toward human-centered interaction through wearable vision and visualization advances in multimedia information processing --- pcm 2001. In H.-Y. Shum, M. Liao, and S.-F. Chang, editors, PCM, volume 2195 of Lecture Notes in Computer Science, pages 40--47. Springer Berlin/Heidelberg, 2001.

Digital Library

[10]

T. Kurata, T. Okuma, M. Kourogi, and K. Sakaue. The hand mouse: Gmm hand-color classification and mean shift tracking. In Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, 2001. Proceedings. IEEE ICCV Workshop on, pages 119--124, 2001.

Digital Library

[11]

B. Laundry, M. Masoodian, and B. Rogers. Interaction with 3D models on large displays using 3D input techniques. In Proc. of CHINZ'10 conference, pages 49--56, 2010.

Digital Library

[12]

J. C. Lee. Hacking the nintendo wii remote. Pervasive Computing, IEEE, 7(3):39--45, 2008.

Digital Library

[13]

J. C. Levesque, D. Laurendeau, and M. Mokhtari. Bimanual gestural interface for virtual environments. In Virtual Reality Conference, pages 223--224, 2011.

Digital Library

[14]

R. Likert. A technique for the measurement of attitudes. Archives of Psychology, 140:1--55, 1932.

[15]

Maya. Maya official site, retrieved: Mar 2012. http://usa.autodesk.com/maya/.

[16]

Microsoft. Holodesk microsoft research, retrieved: Feb 2012. http://research.microsoft.com/apps/video/default.aspx?id=154571.

[17]

Microsoft. Official site for microsoft xbox kinect, retrieved: Mar 2012. http://www.xbox.com/en-US/kinect.

[18]

Nintendo. Nintendo wii official site, retreived: Mar 2012. http://www.nintendo.com/wii.

[19]

Nintnedo. Wii - health and safety precautions, retrived feb 2012. http://www.nintendo.com/consumer/wiisafety.jsp.

[20]

A. Parkes, I. Poupyrev, and H. Ishii. Designing kinetic interactions for organic user interfaces. Special Issue of Communications of the ACM, 51(6):58--65, 2008.

Digital Library

[21]

M. Schlattman and R. Klein. Simultaneous 4 gestures 6 dof real-time two-hand tracking without any markers. In Proceedings of the 2007 ACM symposium on Virtual reality software and technology, VRST '07, pages 39--42, New York, NY, USA, 2007. ACM.

Digital Library

[22]

A. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake. Real time human pose recognition in parts from a single depth image. Computer Vision and Pattern Recognition 2011, pages 1297--1304, June 2011 2011.

Digital Library

[23]

Sony. Ps3 move faq, retrieved: Feb 2012. http://blog.us.playstation.com/2010/09/07/playstation-move-the-ultimate-faq/.

[24]

M. Turk and A. Pentland. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):71--86, 1991.

Digital Library

[25]

R. Y. Wang and J. Popović. Real-time hand-tracking with a color glove. ACM Trans. Graph., 28(3):63:1--63:8, July 2009.

Digital Library

[26]

R. S. Yang, A. Lau, Y. H. Chan, A. G. Strozzi, P. Delmas, and C. Lutteroth. Real time 3D hand tracking for 3D modelling applications. In Proc. of the 26^th IVCNZ Conference, 2011.

[27]

Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330--1334, 2000.

Digital Library

Cited By

Azmi NWyvill GKnott A(2014)Adapting a cheap game controller as a natural 3D input2014 7th International Conference on Human System Interactions (HSI)10.1109/HSI.2014.6860444(32-37)Online publication date: Jun-2014
https://doi.org/10.1109/HSI.2014.6860444
Karam HTanaka J(2014)Two-Handed Interactive Menu: An Application of Asymmetric Bimanual Gestures and Depth Based Selection TechniquesHuman Interface and the Management of Information. Information and Knowledge Design and Evaluation10.1007/978-3-319-07731-4_19(187-198)Online publication date: 2014
https://doi.org/10.1007/978-3-319-07731-4_19

Index Terms

Bimanual natural user interaction for 3D modelling application using stereo computer vision
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens

Recommendations

Bimanual natural user interaction for 3D modelling application using stereo computer vision
CHINZ '12: Proceedings of the 13th International Conference of the NZ Chapter of the ACM's Special Interest Group on Human-Computer Interaction

We demonstrate a system that allows the user to perform 3D modelling and sculpting using postures and 3D movements of their hands. The system utilises the concept of a Natural User Interface using computer vision techniques. This enables the user to ...
Finger identification and hand gesture recognition techniques for natural user interface
APCHI '13: Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction

The natural user interface using hand gesture have been popular field in Human-Computer-Interaction(HCI). Many research papers have been proposed in this field. They proposed vision-based, glove-based and depth-based approach for hand gesture ...
Robust computer vision-based detection of pinching for one and two-handed gesture input
UIST '06: Proceedings of the 19th annual ACM symposium on User interface software and technology

We present a computer vision technique to detect when the user brings their thumb and forefinger together (a pinch gesture) for close-range and relatively controlled viewing circumstances. The technique avoids complex and fragile hand tracking ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

CHINZ '12: Proceedings of the 13th International Conference of the NZ Chapter of the ACM's Special Interest Group on Human-Computer Interaction

July 2012

110 pages

ISBN:9781450314749

DOI:10.1145/2379256

Conference Chair:
Holger Regenbrecht
University of Otago
,
Program Chairs:
Christof Lutteroth,
Mark McGuire

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

New Zealand Chapter of ACM SIGCHI

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 July 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHINZ '12

Sponsor:

CHINZ '12: 13th Annual Conference of the NZ ACM Special Interest Group on Human-Computer Interaction

July 2 - 3, 2012

Dunedin, New Zealand

Acceptance Rates

Overall Acceptance Rate 8 of 23 submissions, 35%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
263
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Azmi NWyvill GKnott A(2014)Adapting a cheap game controller as a natural 3D input2014 7th International Conference on Human System Interactions (HSI)10.1109/HSI.2014.6860444(32-37)Online publication date: Jun-2014
https://doi.org/10.1109/HSI.2014.6860444
Karam HTanaka J(2014)Two-Handed Interactive Menu: An Application of Asymmetric Bimanual Gestures and Depth Based Selection TechniquesHuman Interface and the Management of Information. Information and Knowledge Design and Evaluation10.1007/978-3-319-07731-4_19(187-198)Online publication date: 2014
https://doi.org/10.1007/978-3-319-07731-4_19

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents