short-paper

Facial Expression Recognition via Relation-based Conditional Generative Adversarial Network

Authors:

Byung Cheol Song,

Dong Yoon ChoiAuthors Info & Claims

ICMI '19: 2019 International Conference on Multimodal Interaction

Pages 35 - 39

https://doi.org/10.1145/3340555.3353753

Published: 14 October 2019 Publication History

Abstract

Recognizing emotions by adapting to various human identities is very difficult. In order to solve this problem, this paper proposes a relation-based conditional generative adversarial network (RcGAN), which recognizes facial expressions by using the difference (or relation) between neutral face and expressive face. The proposed method can recognize facial expression or emotion independently of human identity. Experimental results show that the proposed method provides higher accuracies of 97.93% and 82.86% for CK+ and MMI databases, respectively than conventional method.

References

[1]

Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2018. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8789–8797.

[2]

Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37(ICML’15). JMLR.org, 448–456. http://dl.acm.org/citation.cfm?id=3045118.3045167

[3]

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125–1134.

[4]

Heechul Jung, Sihaeng Lee, Junho Yim, Sunjeong Park, and Junmo Kim. 2015. Joint fine-tuning in deep neural networks for facial expression recognition. In Proceedings of the IEEE international conference on computer vision. 2983–2991.

Digital Library

[5]

Youngsung Kim, ByungIn Yoo, Youngjun Kwak, Changkyu Choi, and Junmo Kim. 2017. Deep generative-contrastive networks for facial expression recognition. arXiv preprint arXiv:1703.07140(2017).

[6]

Oliver Langner, Ron Dotsch, Gijsbert Bijlstra, Daniel HJ Wigboldus, Skyler T Hawk, and AD Van Knippenberg. 2010. Presentation and validation of the Radboud Faces Database. Cognition and emotion 24, 8 (2010), 1377–1388.

[7]

Seung Ho Lee, Konstantinos N Kostas Plataniotis, and Yong Man Ro. 2014. Intra-class variation reduction using training expression images for sparse representation based facial expression recognition. IEEE Transactions on Affective Computing 5, 3 (2014), 340–351.

[8]

Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. 2010. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. IEEE, 94–101.

[9]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, Nov (2008), 2579–2605.

[10]

Zibo Meng, Ping Liu, Jie Cai, Shizhong Han, and Yan Tong. 2017. Identity-aware convolutional neural network for facial expression recognition. In 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). IEEE, 558–565.

Digital Library

[11]

Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784(2014).

[12]

Augustus Odena, Christopher Olah, and Jonathon Shlens. 2017. Conditional image synthesis with auxiliary classifier gans. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2642–2651.

Digital Library

[13]

Maja Pantic, Michel Valstar, Ron Rademaker, and Ludo Maat. 2005. Web-based database for facial expression analysis. In 2005 IEEE international conference on multimedia and Expo. IEEE, 5–pp.

[14]

Albert Pumarola, Antonio Agudo, Aleix M Martinez, Alberto Sanfeliu, and Francesc Moreno-Noguer. 2018. Ganimation: Anatomically-aware facial animation from a single image. In Proceedings of the European Conference on Computer Vision (ECCV). 818–833.

Digital Library

[15]

Scott E Reed, Yi Zhang, Yuting Zhang, and Honglak Lee. 2015. Deep visual analogy-making. In Advances in neural information processing systems. 1252–1260.

[16]

Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022(2016).

[17]

Huiyuan Yang, Umur Ciftci, and Lijun Yin. 2018. Facial expression recognition by de-expression residue learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2168–2177.

[18]

Huiyuan Yang, Zheng Zhang, and Lijun Yin. 2018. Identity-adaptive facial expression recognition through expression regeneration using conditional generative adversarial networks. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, 294–301.

Digital Library

[19]

Stefanos Zafeiriou and Maria Petrou. 2010. Sparse representations for facial expressions recognition via l 1 optimization. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. IEEE, 32–39.

[20]

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision. 2223–2232.

Cited By

Kim DSong BKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)Optimal transport-based identity matching for identity-invariant facial expression recognitionProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3601632(18749-18762)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3601632
Zhou HDu JZhang YWang QLiu QLee C(2021)Information Fusion in Attention Networks Using Adaptive and Multi-Level Factorized Bilinear Pooling for Audio-Visual Emotion RecognitionIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2021.309603729(2617-2629)Online publication date: 14-Jul-2021
https://dl.acm.org/doi/10.1109/TASLP.2021.3096037
Ali KHughes C(2020)Face Reenactment Based Facial Expression RecognitionAdvances in Visual Computing10.1007/978-3-030-64556-4_39(501-513)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1007/978-3-030-64556-4_39

Recommendations

Expression-invariant face recognition by facial expression transformations

In this paper, we present a method of expression-invariant face recognition that transforms input face image with an arbitrary expression into its corresponding neutral facial expression image. When a new face image with an arbitrary expression is ...
Multi-pose Facial Expression Recognition Based on Unpaired Images
Image and Graphics
Abstract
Giving machines the ability to perceive human emotions and enable them to recognize our emotional states is one of the important goals to realize human-computer interaction. In the past decades, facial expression recognition (FER) has always been ...
Pose-Invariant Facial Expression Recognition Based on 3D Face Morphable Model and Domain Adversarial Learning
Image and Graphics
Abstract
Pose is one of the most important factors affecting performance of face related recognition algorithms including facial expression recognition (FER). Traditionally, non-frontal FER is conducted by either performing face formalization or designing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICMI '19: 2019 International Conference on Multimodal Interaction

October 2019

601 pages

ISBN:9781450368605

DOI:10.1145/3340555

Editors:
Wen Gao
Peking University, China
,
Helen Mei Ling Meng
Chinese University of Hong Kong, China
,
Matthew Turk
Toyota Technological Institute at Chicago, USA
,
Susan R. Fussell
Cornell University, USA
,
Björn Schuller
Imperial College London / University of Augsburg, UK
,
Yale Song
Microsoft Research, USA
,
Kai Yu
Shanghai Jiao Tong University, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Funding Sources

the Ministry of Trade, industry & Energy

Conference

ICMI '19

ICMI '19: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

October 14 - 18, 2019

Suzhou, China

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
285
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kim DSong BKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)Optimal transport-based identity matching for identity-invariant facial expression recognitionProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3601632(18749-18762)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3601632
Zhou HDu JZhang YWang QLiu QLee C(2021)Information Fusion in Attention Networks Using Adaptive and Multi-Level Factorized Bilinear Pooling for Audio-Visual Emotion RecognitionIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2021.309603729(2617-2629)Online publication date: 14-Jul-2021
https://dl.acm.org/doi/10.1109/TASLP.2021.3096037
Ali KHughes C(2020)Face Reenactment Based Facial Expression RecognitionAdvances in Visual Computing10.1007/978-3-030-64556-4_39(501-513)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1007/978-3-030-64556-4_39

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents