Abstract
In the facial expression recognition field, RGB image-involved models have always achieved the best performance. Since RGB images are easily influenced by illumination, skin color, and cross-databases, the effect of these methods decreases accordingly. To avoid these issues, we propose a novel facial expression recognition framework in which the input only relies on a single depth image since depth image performs very stable in cross-situations. In our framework, we pretrain an RGB face image synthesis model by a generative adversarial network (GAN) using a public database. This pretrained model can synthesize an RGB face image under a unified imaging situation from a depth face image input. Then, introducing the attention mechanism based on facial landmarks into a convolutional neural network (CNN) for recognition, this attention mechanism can strengthen the weights of the key parts. Thus, our framework has a stable input (depth face image) while retaining the natural merits of RGB face images for recognition. Experiments conducted on public databases demonstrate that the recognition rate of our framework is better than that of the state-of-the-art methods, which are also based on depth images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yang, H., Yin, L.: CNN based 3D facial expression recognition using masking and landmark features. In: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 556–560. IEEE, October 2017
Li, H., Sun, J., Wang, D., Xu, Z., Chen, L.: Deep representation of facial geometric and photometric attributes for automatic 3D facial expression recognition. arXiv preprint arXiv:1511.03015 (2015)
Oyedotun, O.K., Demisse, G., El Rahman Shabayek, A., Aouada, D., Ottersten, B.: Facial expression recognition via joint deep learning of RGB-depth map latent representations. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3161–3168 (2017)
Jan, A., Ding, H., Meng, H., Chen, L., Li, H.: Accurate facial parts localization and deep learning for 3D facial expression recognition. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 466–472. IEEE, May 2018
Zhu, K., Du, Z., Li, W., Huang, D., Wang, Y., Chen, L.: Discriminative attention-based convolutional neural network for 3D facial expression recognition. In: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), pp. 1–8. IEEE, May 2019
Zhang, F., Zhang, T., Mao, Q., Xu, C.: Joint pose and expression modeling for facial expression recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3359–3368 (2018)
Yang, H., Ciftci, U., Yin, L.: Facial expression recognition by de-expression residue learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2168–2177 (2018)
Yang, H., Zhang, Z., Yin, L.: Identity-adaptive facial expression recognition through expression regeneration using conditional generative adversarial networks. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 294–301. IEEE, May 2018
Ali, K., Hughes, C.E.: Facial expression recognition using disentangled adversarial learning. arXiv preprint arXiv:1909.13135 (2019)
Cai, J., Meng, Z., Khan, A.S., Li, Z., O’Reilly, J., Tong, Y.: Identity-free facial expression recognition using conditional generative adversarial network. arXiv preprint arXiv:1903.08051 (2019)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Mian, A., Bennamoun, M., Owens, R.: Automatic 3D face detection, normalization and recognition. In: Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT 2006), pp. 735–742. IEEE, June 2006
Savran, A., et al.: Bosphorus database for 3D face analysis. In: Schouten, B., Juul, N.C., Drygajlo, A., Tistarelli, M. (eds.) BioID 2008. LNCS, vol. 5372, pp. 47–56. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89991-4_6
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society (2016)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift (2015)
Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: 7th International Conference on Automatic Face and Gesture Recognition, FGR 2006. IEEE (2006)
Li, H., Chen, L., Huang, D., Wang, Y., Morvan, J.M.: 3D facial expression recognition via multiple kernel learning of multi-scale local normal patterns. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 2577–2580. IEEE, November 2012
Yang, X., Huang, D., Wang, Y., Chen, L.: Automatic 3D facial expression recognition using geometric scattering representation. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). IEEE (2015)
Li, H., Sun, J., Xu, Z., Chen, L.: Multimodal 2D+ 3D facial expression recognition with deep fusion convolutional neural network. IEEE Trans. Multimed. 19(12), 2816–2831 (2017)
Fu, Y., Ruan, Q., Luo, Z., Jin, Y., An, G., Wan, J.: FERLrTc: 2D+3D facial expression recognition via low-rank tensor completion. Signal Process. 161, 74–88 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Cai, J., Xie, H., Li, J., Li, S. (2020). Facial Expression Recognition with an Attention Network Using a Single Depth Image. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1332. Springer, Cham. https://doi.org/10.1007/978-3-030-63820-7_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-63820-7_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63819-1
Online ISBN: 978-3-030-63820-7
eBook Packages: Computer ScienceComputer Science (R0)