Face Reenactment Based Facial Expression Recognition
Pages 501 - 513
Abstract
Representations used for Facial Expression Recognition (FER) are usually contaminated with identity specific features. In this paper, we propose a novel Reenactment-based Expression-Representation Learning Generative Adversarial Network (REL-GAN) that employs the concept of face reenactment to disentangle facial expression features from identity information. In this method, the facial expression representation is learned by reconstructing an expression image employing an encoder-decoder based generator. More specifically, our method learns the disentangled expression representation by transferring the expression information from the source image to the identity of the target image. Experiments performed on widely used datasets (BU-3DFE, CK+, Oulu-CASIA, SEFW) show that the proposed technique produces comparable or better results than state-of-the-art methods.
References
[1]
Yang, H., Zhang, Z., Yin, L.: Identity-adaptive facial expression recognition through expression regeneration using conditional generative adversarial networks. In: International Conference on Automatic Face and Gesture Recognition, pp. 294–301 (2018)
[2]
Cai, J., Meng, Z., Khan, A.S., Li, Z., O’Reilly, J., Tong, Y.: Identity-free facial expression recognition using conditional generative adversarial network. CoRR abs/1903.08051 (2019). https://arxiv.org/abs/1903.08051
[3]
Bai, M., Xie, W., Shen, L.: Disentangled feature based adversarial learning for facial expression recognition. In: IEEE International Conference on Image Processing, pp. 31–35 (2019)
[4]
Yang, H., Ciftci, U., Yin, L.: Facial expression recognition by de-expression residue learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2168–2177, April 2018
[5]
Tran, L., Yin, X., Liu, X.: Disentangled representation learning GAN for pose-invariant face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1415–1424 (2017)
[6]
Song, B.C., Lee, M.K., Choi, D.Y.: Facial expression recognition via relation-based conditional generative adversarial network. In: International Conference on Multimodal Interaction, pp. 35–39 (2019)
[7]
Ali, K., Isler, I., Hughes, C.E.: Facial expression recognition using human to animated-character expression translation. CoRR abs/1910.05595, https://arxiv.org/abs/1910.05595 (2019)
[8]
Klaser, A., Marszałek, M., Schmid, C.: A spatio-temporal descriptor based on 3d-gradients. In: Proceedings British Machine Vision Conference (2008)
[9]
Chu WS, Torre F, and Cohn JF Selective transfer machine for personalized facial expression analysis IEEE Trans. Pattern Anal. Mach. Intell. 2016 39 529-545
[10]
Jiang, B., Valstar, M.F., Pantic, M.: Action unit detection using sparse appearance descriptors in space-time video volumes. In: International Conference on Automatic Face and Gesture Recognition, pp. 314–321 (2011)
[11]
Kim, B.K., Lee, H., Roh, J., Lee, S.Y.: Hierarchical committee of deep CNNs with exponentially-weighted decision fusion for static facial expression recognition. In: Proceedings of the International Conference on Multimodal Interaction, pp. 427–434 (2015)
[12]
Yu, Z., Zhang, C.: Image based static facial expression recognition with multiple deep network learning. In: Proceedings of the International Conference on Multimodal Interaction, pp. 435–442 (2015)
[13]
Ng, H.W., Nguyen, V.W., Vonikakis, V., Winkler, S.: Deep learning for emotion recognition on small datasets using transfer learning. In: Proceedings of the International Conference on Multimodal Interaction, pp. 443–449 (2015)
[14]
Liu, M., Li, S., Shan, S., Wang, R., Chen, X.: Deeply learning deformable facial action parts model for dynamic expression analysis. In: Asian Conference on Computer Vision, pp. 143–157 (2014)
[15]
Li, S., Deng, W., Du, J.P.: Reliable crowd sourcing and deep locality-preserving learning for expression recognition in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2852–2861 (2017)
[16]
Cai, J., Meng, Z., Khan, A.S., Li, Z., O’Reilly, J., Tong, Y.: Island loss for learning discriminative features in facial expression recognition. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 302–309 (2018)
[17]
Meng, Z., Liu, P., Cai, J., Han, S., Tong Y.: Identity-aware convolutional neural network for facial expression recognition. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 558–565 (2017)
[18]
Lucey, P, Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp. 94–101 (2010)
[19]
Zhao G, Huang X, Taini M, Li SZ, and Pietikainen M Facial expression recognition from near-infrared videos Image Vis. Comput. 2011 29 607-619
[20]
Ali, K., Hughes, C.E.: Facial expression recognition using disentangled adversarial learning. CoRR abs/1909.13135 (2019). https://arxiv.org/abs/1909.13135
[21]
Zadeh, A., Lim, Y.C., Baltrusaitis, T., Morency, L.P.: Convolutional experts constrained local model for 3d facial landmark detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2519–2528 (2017)
[22]
Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3d dynamic facial expression database. In: IEEE International Conference on Automatic Face and Gesture Recognition (2008)
[23]
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR abs/1511.06434 (2015). https://arxiv.org/abs/1511.06434
[24]
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in neural information processing systems, pp. 2672–2680 (2014)
[25]
Liu, M., Shan, S., Wang, R., Chen, X.: Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1749–1756 (2014)
[26]
Jung, H., Lee, S., Yim, J., Park, S., Kim, J.: Joint fine-tuning in deep neural net-works for facial expression recognition. In: Proceedings of the IEEE international conference on computer vision, pp. 2983–2991 (2015)
[27]
Zhao, X., et al.: Peak-piloted deep network for facial expression recognition. In: European Conference on Computer Vision, pp. 425–442 (2016)
[28]
Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3d facial expression database for facial behavior research. In: International Conference on Automatic Face and Gesture Recognition, pp. 211–216 (2006)
[29]
Wang, J., Yin, L., Wei, X., Sun, Y.: 3d facial expression recognition based on primitive surface feature distribution. In: IEEE Computer Vision and Pattern Recognition, pp. 1399–1406 (2006)
[30]
Berretti, S., Del Bimbo, A., Pala, P., Amor, B.B., Daoudi, M.: A set of selected sift features for 3d facial expression recognition. In: International Conference on Pattern Recognition, pp. 4125–4128 (2010)
[31]
Lopes AT, de Aguiar E, De Souza AF, and Oliveira-Santos T Facial expression recognition with convolutional neural networks: coping with few data and the training sample order Pattern Recogn. 2017 61 610-628
[32]
Song, L., Lu, Z., He, R., Sun, Z., Tan, T.: Geometry guided adversarial facial expression synthesis. In: Proceedings of the ACM International Conference on Multimedia, pp. 627–635 (2018)
[33]
Dhall, A., Murthy, O.R., Goecke, R., Joshi, J., Gedeon, T.: Video and image based emotion recognition challenges in the wild: Emotiw. In: Proceedings of the ACM on International Conference on Multimodal Interaction, pp. 423–426 (2015)
[34]
Zhang K, Zhang Z, Li Z, and Qiao Y Joint face detection and alignment using multitask cascaded convolutional networks IEEE Signal Process. Lett. 2016 23 10 1499-1503
[35]
Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4295–4304 (2015)
[36]
Liu, M., Li, S., Shan, S., Chen, X.: Au-aware deep networks for facial expression recognition. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1–6 (2013)
[37]
Levi, G., Hassner, T.: Emotion recognition in the wild via convolutional neural networks and mapped binary patterns. In: Proceedings of the ACM on International Conference on Multimodal Interaction, pp. 503–510 (2015)
[38]
Ding, H, Zhou, S.K., Chellappa, R.: Facenet2expnet: regularizing a deep face recognition net for expression recognition. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 118–126 (2017)
[39]
Halawa, M., Wollhaf, M., Vellasques, E., Sanz, U. S., Hellwich, O.: Learning Disentangled Expression Representations from Facial Images. CoRR abs/2008.07001 (2020). https://arxiv.org/abs/2008.07001
[40]
Alaghband, M., Yousefi, N., Garibay, I.: FePh: an annotated facial expression dataset for the RWTH-PHOENIX-Weather 2014 Dataset. CoRR abs/2003.08759 (2020). https://arxiv.org/abs/2003.08759
[41]
Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: Proceedings British Machine Vision Conference (2015)
Index Terms
- Face Reenactment Based Facial Expression Recognition
Index terms have been assigned to the content through auto-classification.
Recommendations
Expression-invariant face recognition by facial expression transformations
In this paper, we present a method of expression-invariant face recognition that transforms input face image with an arbitrary expression into its corresponding neutral facial expression image. When a new face image with an arbitrary expression is ...
Multi-pose Facial Expression Recognition Based on Unpaired Images
Image and GraphicsAbstractGiving machines the ability to perceive human emotions and enable them to recognize our emotional states is one of the important goals to realize human-computer interaction. In the past decades, facial expression recognition (FER) has always been ...
Pose-Robust Facial Expression Recognition Using View-Based 2D + 3D AAM
This paper proposes a pose-robust face tracking and facial expression recognition method using a view-based 2D 3D active appearance model (AAM) that extends the 2D 3D AAM to the view-based approach, where one independent face model is used for a ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Oct 2020
712 pages
ISBN:978-3-030-64555-7
DOI:10.1007/978-3-030-64556-4
© Springer Nature Switzerland AG 2020.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 05 October 2020
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 28 Sep 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in