Abstract
Human-robot interaction is an interdisciplinary research area that aims at the development of social robots. Since social robots are expected to interact with humans and understand their behavior through gestures and body movements, cognitive psychology and robot technology must be integrated. In this paper we present a biological and real-time framework for detecting and tracking hands and heads. This framework is based on keypoints extracted by means of cortical V1 end-stopped cells. Detected keypoints and the cells’ responses are used to classify the junction type. Through the combination of annotated keypoints in a hierarchical, multi-scale tree structure, moving and deformable hands can be segregated and tracked over time. By using hand templates with lines and edges at only a few scales, a hand’s gestures can be recognized. Head tracking and pose detection are also implemented, which can be integrated with detection of facial expressions in the future. Through the combinations of head poses and hand gestures a large number of commands can be given to a robot.
Chapter PDF
Similar content being viewed by others
References
Bandera, J.P., Marfil, R., Bandera, A., Rodríguez, J.A., Molina-Tanco, L., Sandoval, F.: Fast gesture recognition based on a two-level representation. Pattern Recogn. Lett. 30(13), 1181–1189 (2009)
du Buf, J.M.H.: Responses of simple cells: events, interferences, and ambiguities. Biol. Cybern. 68, 321–333 (1993)
Farrajota, M., Rodrigues, J.M.F., du Buf, J.M.H.: Optical flow by multi-scale annotated keypoints: a biological approach. In: Proc. Int. Conf. on Bio-inspired Systems and Signal Processing (BIOSIGNALS 2011), Rome, Italy, pp. 307–315 (2011)
Farrajota, M., Saleiro, M., Terzić, K., Rodrigues, J.M.F., du Buf, J.M.H.: Multi-scale cortical keypoints for realtime hand tracking and gesture recognition. In: Proc. 1st Int. Workshop on Cognitive Assistive Systems: Closing the Action-Perception Loop, pp. 9–15 (2012)
Hubel, D.H.: Eye, Brain and Vision. Scientific American Library (1995)
Kim, H., Kurillo, G., Bajcsy, R.: Hand tracking and motion detection from the sequence of stereo color image frames. In: Proc. IEEE Int. Conf. on Industrial Technology, pp. 1–6 (2008)
Rodrigues, J., du Buf, J.M.H.: Multi-scale keypoints in V1 and beyond: object segregation, scale selection, saliency maps and face detection. BioSystems 2, 75–90 (2006)
Rodrigues, J., du Buf, J.M.H.: A cortical framework for invariant object categorization and recognition. Cognitive Processing 10(3), 243–261 (2009)
Rodrigues, J., du Buf, J.M.H.: Multi-scale lines and edges in V1 and beyond: brightness, object categorization and recognition, and consciousness. BioSystems 95, 206–226 (2009)
Saleiro, M., Rodrigues, J., du Buf, J.M.H.: Automatic hand or head gesture interface for individuals with motor impairments, senior citizens and young children. In: Proc. Int. Conf. Softw. Dev. for Enhancing Accessibility and Fighting Info-Exclusion, pp. 165–171 (2009)
de Sousa, R.J.R., Rodrigues, J.M.F., du Buf, J.M.H.: Recognition of facial expressions by cortical multi-scale line and edge coding. In: Campilho, A., Kamel, M. (eds.) ICIAR 2010. LNCS, vol. 6111, pp. 415–424. Springer, Heidelberg (2010)
Suau, X., Ruiz-Hidalgo, J., Casas, J.R.: Real-time head and hand tracking based on 2.5d data. IEEE Trans. on Multimedia 14(3), 575–585 (2012)
Suk, H., Sin, B., Lee, S.: Hand gesture recognition based on dynamic Bayesian network framework. Pattern Recogn 43(9), 3059–3072 (2010)
Terzić, K., du Buf, J.M.H., Rodrigues, J.M.F.: Real-time object recognition based on cortical multi-scale keypoints. In: Accepted for 6th Iberian Conference on Pattern Recognition and Image Analysis, Madeira, Portugal, June 5-7 (2013)
Yi, L.: Hand gesture recognition using kinect. In: Proc. IEEE 3rd Int. Conf. on Softw. Engin. and Service Science, pp. 196–199 (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Saleiro, M., Farrajota, M., Terzić, K., Rodrigues, J.M.F., du Buf, J.M.H. (2013). A Biological and Real-Time Framework for Hand Gestures and Head Poses. In: Stephanidis, C., Antona, M. (eds) Universal Access in Human-Computer Interaction. Design Methods, Tools, and Interaction Techniques for eInclusion. UAHCI 2013. Lecture Notes in Computer Science, vol 8009. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39188-0_60
Download citation
DOI: https://doi.org/10.1007/978-3-642-39188-0_60
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39187-3
Online ISBN: 978-3-642-39188-0
eBook Packages: Computer ScienceComputer Science (R0)