Abstract
A color-based face tracking algorithm is proposed to be used as a human-computer interaction tool on mobile devices. The solution provides a natural means of interaction enabling a motion parallax effect in applications. The algorithm considers the characteristics of mobile use-constrained computational resources and varying environmental conditions. The solution is based on color comparisons and works on images gathered from the front camera of a device. In addition to color comparisons, the coherency of the facial pixels is considered in the algorithm. Several applications are also demonstrated in this work, which use the face position to determine the viewpoint in a virtual scene, or for browsing large images. The accuracy of the system is tested under different environmental conditions such as lighting and background, and the performance of the system is measured in different types of mobile devices. According to these measurements the system allows for accurate (7% RMS error) face tracking in real time (20–100 fps).
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Barnard, M., Hannuksela, J., Sangi, P., Heikkilä, J.: A vision based motion interface for mobile phones. In: Proc. of 5th International Conference on Computer Vision Systems (ICVS), Bielefeld, Germany (2007)
Bradski, G.R.: Computer vision face tracking for use in a perceptual user interface. Intel Technol. J. 95–103 (1998)
Brand, J., Mason, J.: A comparative assessment of three approaches to pixel-level human skin detection. In: Proc. of IEEE International Conf. Pattern Recognition, vol. 1, pp. 1056–1059 (2000)
Bulbul, A., Cipiloglu, Z., Capin, T.: A face tracking algorithm for user interaction in mobile devices. In: Proc. of Cyberworlds, International Conference, pp. 385–390 (2009)
Capin, T., Haro, A., Setlur, V., Wilkinson, S.: Camera-Based Virtual Environment Interaction on Mobile Devices, Lecture Notes in Computer Science, vol. 4263, pp. 765–773. Springer, Berlin (2006). ISBN: 9783540472421
Capin, T., Pulli, K., Akenine-Möller, T.: The state of the art in mobile graphics research. IEEE Comput. Graph. Appl. 28(4), 74–84 (2008)
Chai, D., Ngan, K.N.: Face segmentation using skin color map in videophone applications. IEEE Trans. Circuits Syst. Video Technol. 9(4), 551–564 (1999)
Hannuksela, J., Huttunen, S., Sangi, P., Heikkilä, J.: Motion-based finger tracking for user interaction with mobile phones. In: Proc. of 4th European Conference on Visual Media Production (CVMP). London, UK (2007)
Hannuksela, J., Sangi, P., Heikkilä, J.: A Vision-based approach for controlling user interfaces of mobile devices. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Vision for Human-Computer Interaction (V4HCI), vol. 6, p. 71, San Diego, CA (2005)
Haro, A., Mori, K., Capin, T., Wilkinson, S.: Mobile camera-based user interaction. In: Proc. of ICCV-HCI 2005, pp. 79–89 (2005)
Hjelmas, B.K.L.E.: Face detection: a survey. Comput. Vis. Image Underst. 3(3), 236–274 (2001)
Home of the Xith3D Project. Xith3D.org. http://xith.org/. Accessed 27 October 2009
Hunke, M., Waibel, A.: Face locating and tracking for human-computer interaction. In: Proc. of the 28th Asilomar Conf. on Signals, Systems and Computers, vol. 2, pp. 1277–1281 (1994)
Jaimes, A., Sebe, N.: Multimodal human computer interaction: a survey. In: Proc. of 11th IEEE International Workshop Human Computer Interaction (HCI) (2005)
Kakumanu, P., Makrogiannis, S., Bourbakis, N.: A survey of skin colormodeling and detectionmethods. Pattern Recognit. 40, 1106–1122 (2007)
Shirley, P.: Fundamentals of Computer Graphics. AK Peters, Natick (2002)
Siriluck, W., Kamolphiwong, S., Kamolphiwong, T., Sae-Whong, S.: Blink and click. In: Proc. of the 1st International Convention on Rehabilitation Engineering & Assistive Technology: in Conjunction with 1st Tan Tock Seng Hospital Neurorehabilitation Meeting, pp. 43–46 (2007)
Vezhnevets, V., Sazonov, V., Andreeva, A.: A survey on pixel-based skin color detection techniques. In: Proc. of GRAPHICON03, pp. 85–92 (2003)
Ware, C.: Space Perception and the Display of Data in Space. Information Visualization: Perception for Design. Morgan Kauffman, San Mateo (2004)
Yee, K.P.: Peephole displays: Pen interaction on spatially aware hand-held computers. In: Proc. of CHI 2003, pp. 1–8 (2003)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bulbul, A., Cipiloglu, Z. & Capin, T. A color-based face tracking algorithm for enhancing interaction with mobile devices. Vis Comput 26, 311–323 (2010). https://doi.org/10.1007/s00371-010-0419-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-010-0419-0