research-article

Dense Attention Network for Facial Expression Recognition in the Wild

Authors:

Yanfu YanAuthors Info & Claims

MMAsia '19: Proceedings of the 1st ACM International Conference on Multimedia in Asia

Article No.: 30, Pages 1 - 6

https://doi.org/10.1145/3338533.3366568

Published: 10 January 2020 Publication History

Abstract

Recognizing facial expression is significant for human-computer interaction system and other applications. A certain number of facial expression datasets have been published in recent decades and helped with the improvements for emotion classification algorithms. However, recognition of the realistic expressions in the wild is still challenging because of uncontrolled lighting, brightness, pose, occlusion, etc. In this paper, we propose an attention mechanism based module which can help the network focus on the emotion-related locations. Furthermore, we produce two network structures named DenseCANet and DenseSANet by using the attention modules based on the backbone of DenseNet. Then these two networks and original DenseNet are trained on wild dataset AffectNet and lab-controlled dataset CK+. Experimental results show that the DenseSANet has improved the performance on both datasets comparing with the state-of-the-art methods.

References

[1]

Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, and Tat-Seng Chua. 2017. Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5659--5667.

[2]

Luefeng Chen, Mengtian Zhou, Wanjuan Su, Min Wu, Jinhua She, and Kaoru Hirota. 2018. Softmax regression based deep sparse autoencoder network for facial emotion recognition in human-robot interaction. Information Sciences 428 (2018), 49--61.

Digital Library

[3]

Ciprian Adrian Corneanu, Marc Oliu Simón, Jeffrey F Cohn, and Sergio Escalera Guerrero. 2016. Survey on rgb, 3d, thermal, and multi-modal approaches for facial expression recognition: History, trends, and affect-related applications. IEEE transactions on pattern analysis and machine intelligence 38, 8 (2016), 1548--1568.

[4]

Hui Ding, Shaohua Kevin Zhou, and Rama Chellappa. 2017. Facenet2expnet: Regularizing a deep face recognition net for expression recognition. In 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). IEEE, 118--126.

Digital Library

[5]

Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132--7141.

[6]

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4700--4708.

[7]

Pooya Khorrami, Thomas Paine, and Thomas Huang. 2015. Do deep neural networks learn facial action units when doing expression recognition?. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 19--27.

Digital Library

[8]

Yong Li, Jiabei Zeng, Shiguang Shan, and Xilin Chen. 2018. Occlusion aware facial expression recognition using cnn with attention mechanism. IEEE Transactions on Image Processing 28, 5 (2018), 2439--2450.

[9]

Mengyi Liu, Shaoxin Li, Shiguang Shan, and Xilin Chen. 2013. Au-aware deep networks for facial expression recognition. In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). IEEE, 1--6.

[10]

Xiaofeng Liu, BVK Vijaya Kumar, Jane You, and Ping Jia. 2017. Adaptive deep metric learning for identity-aware facial expression recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 20--29.

[11]

Ilya Loshchilov and Frank Hutter. 2016. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016).

[12]

Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. 2010. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. IEEE, 94--101.

[13]

Ali Mollahosseini, Behzad Hasani, and Mohammad H Mahoor. 2017. Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing 10, 1 (2017), 18--31.

Digital Library

[14]

Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. 2015. Learning deconvolution network for semantic segmentation. In Proceedings of the IEEE international conference on computer vision. 1520--1528.

Digital Library

[15]

Caifeng Shan, Shaogang Gong, and Peter W McOwan. 2009. Facial expression recognition based on local binary patterns: A comprehensive study. Image and vision Computing 27, 6 (2009), 803--816.

[16]

Leslie N Smith. 2017. Cyclical learning rates for training neural networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 464--472.

[17]

Yichuan Tang. 2013. Deep learning using linear support vector machines. arXiv preprint arXiv:1306.0239 (2013).

[18]

Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xiaogang Wang, and Xiaoou Tang. 2017. Residual attention network for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3156--3164.

[19]

Siyue Xie, Haifeng Hu, and Yongbo Wu. 2019. Deep multi-path convolutional neural network joint with salient region attention for facial expression recognition. Pattern Recognition 92 (2019), 177--191.

Digital Library

[20]

Yanfu Yan, Ke Lu, Jian Xue, Pengcheng Gao, and Jiayi Lyu. 2019. FEAFA: A Well-Annotated Dataset for Facial Expression Analysis and 3D Facial Animation. In IEEE International Conference on Multimedia and Expo Workshops.

[21]

Jiabei Zeng, Shiguang Shan, and Xilin Chen. 2018. Facial expression recognition with inconsistently annotated datasets. In Proceedings of the European conference on computer vision (ECCV). 222--237.

Digital Library

Cited By

Sharma KNandal RKumar SJoshi K(2024)VEDANet: A dense blocked network for visual emotion analysis in multimedia retrievalMultimedia Tools and Applications10.1007/s11042-024-19646-2Online publication date: 16-Jul-2024
https://doi.org/10.1007/s11042-024-19646-2
Hu MLi JA XZhao YLu MLi W(2023)FSAU-Net: a network for extracting buildings from remote sensing imagery using feature self-attentionInternational Journal of Remote Sensing10.1080/01431161.2023.217712544:5(1643-1664)Online publication date: 22-Mar-2023
https://doi.org/10.1080/01431161.2023.2177125
Wang CXue JLu KYan Y(2022)Light Attention Embedding for Facial Expression RecognitionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2021.308332632:4(1834-1847)Online publication date: Apr-2022
https://doi.org/10.1109/TCSVT.2021.3083326
Show More Cited By

Index Terms

Dense Attention Network for Facial Expression Recognition in the Wild
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations
      2. Computer vision tasks
        Activity recognition and understanding

Recommendations

Expression-invariant face recognition by facial expression transformations

In this paper, we present a method of expression-invariant face recognition that transforms input face image with an arbitrary expression into its corresponding neutral facial expression image. When a new face image with an arbitrary expression is ...
Facial expression recognition with Convolutional Neural Networks

Facial expression recognition has been an active research area in the past 10 years, with growing application areas including avatar animation, neuromarketing and sociable robots. The recognition of facial expressions is not an easy problem for machine ...
Discriminative attention-augmented feature learning for facial expression recognition in the wild
Abstract
Facial expression recognition (FER) in-the-wild is challenging due to unconstraint settings such as varying head poses, illumination, and occlusions. In addition, the performance of a FER system significantly degrades due to large intra-class ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MMAsia '19: Proceedings of the 1st ACM International Conference on Multimedia in Asia

December 2019

403 pages

ISBN:9781450368414

DOI:10.1145/3338533

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 January 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Instrument Developing Project of the Chinese Academy of Sciences
National Key R&D Program of China
Natural Science Foundation of Beijing Municipality
Scientific Research Program of Beijing Municipal Education Commission
National Natural Science Foundation of China
University of Chinese Academy of Sciences

Conference

MMAsia '19

Sponsor:

SIGMM

MMAsia '19: ACM Multimedia Asia

December 15 - 18, 2019

Beijing, China

Acceptance Rates

MMAsia '19 Paper Acceptance Rate 59 of 204 submissions, 29%;

Overall Acceptance Rate 59 of 204 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
299
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)3

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sharma KNandal RKumar SJoshi K(2024)VEDANet: A dense blocked network for visual emotion analysis in multimedia retrievalMultimedia Tools and Applications10.1007/s11042-024-19646-2Online publication date: 16-Jul-2024
https://doi.org/10.1007/s11042-024-19646-2
Hu MLi JA XZhao YLu MLi W(2023)FSAU-Net: a network for extracting buildings from remote sensing imagery using feature self-attentionInternational Journal of Remote Sensing10.1080/01431161.2023.217712544:5(1643-1664)Online publication date: 22-Mar-2023
https://doi.org/10.1080/01431161.2023.2177125
Wang CXue JLu KYan Y(2022)Light Attention Embedding for Facial Expression RecognitionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2021.308332632:4(1834-1847)Online publication date: Apr-2022
https://doi.org/10.1109/TCSVT.2021.3083326
Liu XCheng XLee K(2021)GA-SVM-Based Facial Emotion Recognition Using Facial Geometric FeaturesIEEE Sensors Journal10.1109/JSEN.2020.302807521:10(11532-11542)Online publication date: 15-May-2021
https://doi.org/10.1109/JSEN.2020.3028075
Wang CLu KXue JYan YLiu WGan CSmith JSong JZhang DHuang W(2020)R-FENet: A Region-based Facial Expression Recognition Method Inspired by Semantic Information of Action UnitsProceedings of the 1st International Workshop on Human-centric Multimedia Analysis10.1145/3422852.3423482(43-51)Online publication date: 12-Oct-2020
https://dl.acm.org/doi/10.1145/3422852.3423482

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents