Abstract
3D gesture recognition and tracking are highly desired features of interaction design in future mobile and smart environments. Specifically, in virtual/augmented reality applications, intuitive interaction with the physical space seems unavoidable and 3D gestural interaction might be the most effective alternative for the current input facilities such as touchscreens. In this paper, we introduce a novel solution for real-time 3D gesture-based interaction by finding the best match from an extremely large gesture database. This database includes the images of various articulated hand gestures with the annotated 3D position/orientation parameters of the hand joints. Our unique matching algorithm is based on the hierarchical scoring of the low-level edge-orientation features between the query frames and database and retrieving the best match. Once the best match is found from the database in each moment, the pre-recorded 3D motion parameters can instantly be used for natural interaction. The proposed bare-hand interaction technology performs in real-time with high accuracy using an ordinary camera.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yousefi, S.: 3D Photo Browsing for Future Mobile Devices. In: Proceedings of the ACMMM12, Nara, Japan, 29 October–2 November, 2012
Yousefi, S.: Enabling media technologies for mobile photo browsing. Licentiate thesis. Umea University, Department of Applied Physics and Electronics. Umea, Sweden (2012). ISBN 978-91-7459-426-3
Dorfmueller-Ulhaas, K., Schmalstieg, D.: Finger tracking for interaction in augmented environments. In: 2nd ACM/IEEE Int’l Symposium on Augmented Reality (2001)
Maggioni, C.: A novel gestural input device for virtual reality. In: Virtual Reality Annual International Symposium, pp. 118–124. IEEE (1993)
Hardenberg, C.V., Berard, F.: Bare-hand human-computer interaction. In: Proceedings of the 2001 Workshop on Perceptive User Interfaces. ACM International Conference Proceeding Series, Orlando, Florida, vol. 15, pp. 1–8 (2001)
Iwai, D., Sato, K.: Heat sensation in image creation with thermal vision. In: ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (2005)
Kolsch, M., Turk, M.: Fast 2D hand tracking with flocks of features and multi-cue integration. In: Proceedings of the Computer Vision and Pattern Recognition Workshop (2004)
Ren, Z., Meng, J., Yuan, J., Zhang, Z.: Robust hand gesture recognition with kinect sensor. In: ACM Multimedia, pp. 759–760 (2011)
Erol, A., Bebis, G., Nicolescu, M., Boyle, R., Twombly, X.: Vision-based hand pose estimation: a review. Comput. Vis. Image Underst. 108, 52–73 (2007)
Stenger, B., Thayananthan, A., Torr, P., Cipolla, R.: Model-based hand tracking using a hierarchical Bayesian filter. IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1372–1384 (2006)
Yang, R., Sarkar, S.: Gesture recognition using hidden markov models from fragmented observations. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006)
Bencheikh, M., Bouzenada, M., Batouche, M.: A new method of finger tracking applied to the magic board. In: Conference on Industrial Technology (2004)
Zhou, H., Ruan, Q.: Finger countour tracking based on model. In: Conference on Computers, Comunications, Control and Power Engineering, p. 503 (2002)
Arce, F., Valdez, J.: Accelerometer-based hand gesture recognition using artificial neural networks. In: Castillo, O., Kacprzyk, J., Pedrycz, W. (eds.) Soft Computing for Intelligent Control and Mobile Robotics. SCI, pp. 67–77. Springer, Heidelberg (2011)
Choi, J., Song, K., Lee, S.: Enabling a gesture-based numeric input on mobile phones. In: IEEE International Conference on Consumer Electronics (ICCE), pp. 151–152 (2011)
Hrst, W., Wezel, C.: Gesture-based interaction via finger tracking for mobile augmented reality. Multimedia Tools Appl. 62, 1–26 (2012)
Baldauf, M., Zambanini, S., Fröhlich, P., Reichl, P.: Markerless visual fingertip detection for natural mobile device interaction. Mobile HCI, pp. 539–544 (2011)
Lee, D., Lee, S.: Vision-based finger action recognition by angle detection and contour analysis. ETRI J. 33(3), 415–422 (2011)
Hannuksela, J., Barnard, M., Sangi, P., Heikkil, J.: Camera-Based Motion Recognition for Mobile Interaction. ISRN Signal Processing (2011)
Hagbi, N., Bergig, O., El-Sana, J., Billinghurst, M.: Shape recognition and pose estimation for mobile augmented reality. In: 8th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2009), IEEE Computer Press (2009)
Cao, Y., Wang, C., Zhang, L., Zhang, L.: Edgel index for large-scale sketch-based image search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, pp. 761-768 (2011). ISBN 978-1-4577-0394-2
Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: Sketch-based image retrieval: benchmark and bag-of-features descriptors. IEEE Trans. Vis. Comput. Graphics 17(11), 1624–1636 (2011)
Ikizler, N., Duygulu, P.: Human action recognition using distribution of oriented rectangular patches. In: Workshop on Human Motion, pp. 271–284 (2007)
Kondori, F.A., Liu, L.: 3D Active human motion estimation for biomedical applications. In: World congress on Medical Physics and Biomedical Engineering (WC2012). Beijing, China (2012)
Kondori, F.A.: Human motion analysis for creating immersive experience. Licentiate thesis, Department of Applied Physics and Electronics, Umea University, Sweden (2012). ISBN 9789174594164
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Yousefi, S., Li, H. (2015). 3D Interaction Through a Real-Time Gesture Search Engine. In: Jawahar, C., Shan, S. (eds) Computer Vision - ACCV 2014 Workshops. ACCV 2014. Lecture Notes in Computer Science(), vol 9009. Springer, Cham. https://doi.org/10.1007/978-3-319-16631-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-16631-5_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16630-8
Online ISBN: 978-3-319-16631-5
eBook Packages: Computer ScienceComputer Science (R0)