Abstract
With recent advances in artificial intelligence and pattern recognition, automatic facial expression recognition draws a great deal of interest. In this area, most of works involved 2D imagery. However, they present some challenges related to pose, illumination variation and self-occlusion. To deal with these problems, we propose to reconstruct the face in 3D space, from only one 2D image, using the 3D Morphable Model (3DMM). Thus, thanks to its robustness against pose and illumination variations, 3DMM offers high-resolution model and fast fitting functionality. Then, given the reconstructed 3D face, we extract a set of features, which are effective to describe shape changes and expression-related facial appearance, using Mesh-Local Binary Pattern (mesh-LBP). Obtained results proved the effectiveness of combining 3DMM and mesh-LBP for automatic facial expression recognition from 2D single image. In fact, to evaluate the proposed method against state-of-the-art methods, a comparative study shows that the method outperforms existing ones.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Han, H., Otto, C., Liu, X., Jain, A.K.: Demographic estimation from face images: human vs. machine performance. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1148–1161 (2015)
Phillips, P.J., Toole, A.J.O.: Comparison of human and computer performance across face recognition experiments. Image Vis. Comput. 32, 74–85 (2014)
Koelstra, S., Pantic, M., Patras, I.: A dynamic texture-based method to recognition of facial actions and their temporal models. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 1940–1954 (2010)
Ekman, P.: Universals and cultural differences in facial expressions of emotion. Nebr. Symp. Motiv. 207–283 (1972)
Bull, P.: Communication Under the Microscope: The Theory and Practice of Microanalysis. Routledge, Abingdon (2002)
Zhen, H.D., Wang, Y., Chen, L.: Muscular movement model-based automatic 3D/4D facial expression recognition: face segmentation, shape representation, and feature fusion. IEEE Trans. Multimedia 18(7), 1438–1450 (2016)
Xue, M., Mian, A., Liu, W., Li, L.: Fully automatic 3D facial expression recognition using local depth features. In: IEEE Winter Conference Applications of Computer Vision (WACV) (2014)
Li, X.L., Ruan, Q.Q., Jin, Y., An, G., Zhao, R.: Fully automatic 3D facial expression recognition using polytypic multi-block local binary patterns. Sig. Process. 108, 297–308 (2015)
Jan, A., Meng, H.: Automatic 3D facial expression recognition using geometric and textured feature fusion. In: 11th IEEE, Automatic Face and Gesture Recognition (FG) (2015)
Huber, P., Hu, G., Tena, R., Mortazavian, P., Koppen, W.P., Christmas, W., Rätsch, M., Kittler, J.: A multi resolution 3D morphable face model and fitting framework. In: International Conference on Computer Vision Theory and Applications (VISAPP) (2016)
King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. computer vision and pattern recognition (CVPR). In: IEEE Conference: pp. 1867–1874 (2014)
Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)
Werghi, N., Tortorici, C., Berretti, S., Del Bimbo, A.: Boosting 3D LBP-based face recognition by fusing shape and texture descriptors on the mesh. IEEE Trans. Inf. Forensics Secur. 11, 964–979 (2016)
Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29, 51–59 (1996)
Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L.: Bosphorus database for 3D face analysis. In: Schouten, B., Juul, N.C., Drygajlo, A., Tistarelli, M. (eds.) BioID 2008. LNCS, vol. 5372, pp. 47–56. Springer, Heidelberg (2008). doi:10.1007/978-3-540-89991-4_6
Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: Conference on Automatic Face and Gesture Recognition (FGR 2006), pp. 211–216 (2006)
Hewahi, N.M., AbdulRahman, M.B.: Emotion Recognition model based on facial expressions, ethnicity and gender using backpropagation neural network. Int. J. Technol. Diffus. 3, 33–43 (2012)
Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3D dynamic facial expression database. In: IEEE Automatic Face and Gesture Recognition (2008)
Xue, M., Mian, A., Liu, W., Li, L.: Automatic 4D facial expression recognition using DCT features. In: IEEE Winter Conference Applications of Computer Vision (WACV), pp. 199–206 (2015)
Reale, M., Zhang, X., Yin, L.: Nebula feature: a space time feature for posed and spontaneous 4D facial behavior analysis. In: 2013 10th IEEE International Conference and Workshops in Automatic Face and Gesture Recognition (FG), pp. 1–8 (2013)
Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high resolution 3D dynamic facial expression database. In: 8th IEEE International Conference in Automatic Face & Gesture Recognition, pp. 1–6 (2008)
Chang, C.C., Lin, C.J.: LIBSVM: A Library for support vector machines. ACM Trans.Intell. Syst. Technol. (2011)
Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)
Barr, J., Bowyer, K., Flynn, P., Biswas, S.: Face recognition from video: a review. Int. J. Pattern Recogn. Artif. Intell. 26(5), 1266002 (2012)
Zhang, X., Gao, Y.: Face recognition across pose: a review. Pattern Recogn. 42(11), 2876–2896 (2009)
Mishra, B., Fernandes, S., Abhishek, K., Alva, A., Shetty, C., Ajila, C., Shetty, D., Rao, H., Shetty, P.: Facial expression recognition using feature based techniques and model based techniques: a survey. In: Electronics and Communication Systems (ICECS), 2nd International Conference, pp. 589–594 (2015)
Sandbach, G., Zafeiriou, S., Pantic, M., Yin, L.: Static and dynamic 3D facial expression recognition: a comprehensive survey. Image Vis. Comput. 30, 683–697 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Bejaoui, H., Ghazouani, H., Barhoumi, W. (2017). Fully Automated Facial Expression Recognition Using 3D Morphable Model and Mesh-Local Binary Pattern. In: Blanc-Talon, J., Penne, R., Philips, W., Popescu, D., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2017. Lecture Notes in Computer Science(), vol 10617. Springer, Cham. https://doi.org/10.1007/978-3-319-70353-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-70353-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70352-7
Online ISBN: 978-3-319-70353-4
eBook Packages: Computer ScienceComputer Science (R0)