Abstract
We propose a robust semi-auto system of labeling faces for characters in video by combining face detection, tracking, and recognition. At the very first step, our system detects the faces automatically in each video frame. After that, the face detection responses will be linked together to form face sequences (raw tracklets) in each video shot by employing simple temporal-spatial constraints to associate. Then we apply a tracking algorithm to not only extend those raw tracklets bi-directionally to cover more appearance views of the object instead of focusing only the frontal view, but also to help fixing the “gaps” caused by missed detection. After being merged among the potential overlapped ones, in the next step, these extended non-overlapped face tracklets across the video are associated with each other by our proposed Heuristics clustering algorithm. In order to achieve high accuracy, we use both generative and discriminative appearance models of the faces and also the context information, which is the clothing color feature in our case. An extensive experiment is performed on approximately 3.5 hours of videos cut from two TV series “Friends”, “How I Met Your Mother” to show the robustness of our system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Satoh, S., Nakamura, Y., Kanade, T.: Name-it: naming and detecting faces in video by the integration of image and natural language processing. In: Proceedings of the Fifteenth International Joint Conference on Artifical Intelligence, IJCAI 1997, vol. 2, pp. 1488–1493. Morgan Kaufmann Publishers Inc., San Francisco (1997)
Yang, J., Hauptmann, A.G.: Naming every individual in news video monologues. In: Proceedings of the 12th Annual ACM International Conference on Multimedia, MULTIMEDIA 2004, pp. 580–587. ACM, New York (2004)
Houghton, R.: Named faces: Putting names to faces. IEEE Intelligent Systems 14(5), 45–50 (1999)
Sivic, J., Everingham, M., Zisserman, A.: Person spotting: Video shot retrieval for face sets. In: Leow, W.-K., Lew, M., Chua, T.-S., Ma, W.-Y., Chaisorn, L., Bakker, E.M. (eds.) CIVR 2005. LNCS, vol. 3568, pp. 226–236. Springer, Heidelberg (2005)
Everingham, M., Sivic, J., Zisserman, A.: “Hello! My name is... Buffy” – Automatic naming of characters in TV video. In: British Machine Vision Conference (2006)
Sivic, J., Everingham, M., Zisserman, A.: “Who are you?” – learning person specific classifiers from video. In: IEEE Conference on Computer Vision and Pattern Recognition (2009)
Aggarwal, G., Chowdhury, A.K.R., Chellappa, R.: A system identification approach for video-based face recognition. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR 2004), vol. 4, pp. 175–178. IEEE Computer Society, Washington, DC (2004)
Fraunhofer: SHORE - Sophisticated High-speed Object Recognition Engine (2009), http://www.iis.fraunhofer.de/en/bf/bsy/produkte/shore.html
Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-Learning-Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(7), 1409–1422 (2012)
Essa, I.A., Pentland, A.P.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 757–763 (1997)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 484–498 (1998)
Yacoob, Y., Davis, L.: Detection, analysis and matching of hair. In: Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV 2005), pp. 741–748. IEEE Computer Society (2005)
Naaman, M., Yeh, R.B., Garcia-Molina, H., Paepcke, A.: Leveraging context to resolve identity in photo albums. In: Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries, JCDL 2005, pp. 178–187. ACM, New York (2005)
Zhang, L., Chen, L., Li, M., Zhang, H.: Automated annotation of human faces in family albums. In: Proceedings of the Eleventh ACM International Conference on Multimedia, MULTIMEDIA 2003, pp. 355–358. ACM, New York (2003)
Sivic, J., Zitnick, C.L., Szeliski, R.: Finding people in repeated shots of the same scene. In: British Machine Vision Conference (2006)
Anguelov, D., Lee, K.C., Göktürk, S.B., Sumengen, B.: Contextual identity recognition in personal photo albums. In: 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2007), June 18-23. IEEE Computer Society, Minneapolis (2007)
Li, D., Wei, G., Sethi, I.K., Dimitrova, N.: Person identification in tv programs. J. Electronic Imaging 10(4), 930–938 (2001)
Tomasi, C., Kanade, T.: Detection and tracking of point features. Technical report. International Journal of Computer Vision (1991)
Köstinger, M., Wohlhart, P., Roth, P.M., Bischof, H.: Learning to recognize faces from videos and weakly related information cues. In: AVSS, pp. 23–28 (2011)
Kalal, Z., Mikolajczyk, K., Matas, J.: Face-TLD: Tracking-Learning-Detection applied to faces. In: ICIP, pp. 3789–3792 (2010)
Kalal, Z., Matas, J., Mikolajczyk, K.: P-N learning: Bootstrapping binary classifiers by structural constraints. In: The Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2010, June 13-18, pp. 49–56. IEEE, San Francisco (2010)
Interactive Video (2013), http://interactive-video.appspot.com
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nguyen, HT., Nguyen, NH., Dinh, T.B., Dinh, T.B. (2013). Interactive Face Labeling System in Real-World Videos. In: Murgante, B., et al. Computational Science and Its Applications – ICCSA 2013. ICCSA 2013. Lecture Notes in Computer Science, vol 7975. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39640-3_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-39640-3_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39639-7
Online ISBN: 978-3-642-39640-3
eBook Packages: Computer ScienceComputer Science (R0)