Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

FGENet: a lightweight facial expression recognition algorithm based on FasterNet

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

To solve the problem of poor recognition accuracy due to the complex network structure in the facial expression recognition algorithm, an improved lightweight network based on FasterNet is proposed: FGENet. Firstly, the PConv in the FasterNet block module is replaced with the RFE module to enhance the network’s ability to perceive key features of facial expressions. Secondly, the GSConv module is introduced into the residual branch, which helps to strengthen the feature modeling of different regions and improves the expression feature extraction ability of the network. Finally, the ECA attention mechanism is introduced into the Ghost module to design the deep attention mechanism DEA, which is applied to the standard convolution after global pooling to better capture the global context information. FGENet is designed to keep the network lightweight while improving performance. Through a series of experiments, it has been proven that FGENet has achieved performance improvements on the three data sets of Fer2013, CK+ and RAF, reaching recognition accuracy rates of 70.49%, 97.89% and 86.72% respectively, further verifying its effectiveness in facial expression recognition tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The data that support the finding of this article are not publicly available due to privacy. They can be requested from the author at email: yancm2022@163.com.

References

  1. Mehrabian, A.: Communication without words[M]. In: Communication Theory, pp. 193–200. Routledge (2017)

  2. Lalitharatne, T.D., Tan, Y., Leong, F., et al.: Facial expression rendering in medical training simulators: current status and future directions[J]. IEEE Access 8, 215874–215891 (2020)

    Article  Google Scholar 

  3. Kawamura, R., Murase, K.: Concentration estimation in E-learning based on learner’s facial reaction to teacher’s action[C]. In: Proceedings of the 25th International Conference on Intelligent User Interfaces Companion, pp. 103–104 (2020)

  4. Kim, C.M., Hong, E.J., Chung, K., et al.: Driver facial expression analysis using LFA-CRNN-based feature extraction for health-risk decisions[J]. Appl. Sci. 10(8), 2956 (2020)

    Article  Google Scholar 

  5. Lundqvist, D., Flykt, A., Hman, A.: The karolinska directed emotional faces—KDEF, CD ROM from Department of Clinical Neuroscience. Psychol. Sec. [J] (1998)

  6. Kim, S., An, G.H., Kang, S.J.: Facial expression recognition system using machine learning[C]. In: International SoC Design Conference (2017). https://doi.org/10.1109/ISOCC.2017.8368887.

  7. Yue, C., Liang, J., Qu, B., et al.: Sparse representation feature for facial expression recognition[C]. In: Proceedings of ELM-2017. pp. 12–21. Springer International Publishing (2019)

  8. Abdulrahman, M., Gwadabe, T.R., Abdu, F.J., et al.: Gabor wavelet transform based facial expression recognition using PCA and LBP[C]. In: 2014 22nd Signal Processing And Communications Applications Conference (SIU), pp. 2265–2268, IEEE (2014)

  9. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks[J]. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  10. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, (2016)

  11. Howard, A.G., Zhu, M., Chen, B., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861 (2017)

  12. Sandler, M., Howard, A., Zhu, M., et al.: Mobilenetv2: Inverted residuals and linear bottlenecks[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

  13. Howard, A., Sandler, M., Chu, G., et al.: Searching for mobilenetv3[C]. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1314–1324 (2019)

  14. Kong, Y., Ren, Z., Zhang, K., et al.: Lightweight facial expression recognition method based on attention mechanism and key region fusion[J]. J. Electron. Imaging 30(6), 063002–063002 (2021)

    Article  Google Scholar 

  15. Nan, Y., Ju, J., Hua, Q., et al.: A-MobileNet: an approach of facial expression recognition[J]. Alex. Eng. J. 61(6), 4435–4444 (2022)

    Article  Google Scholar 

  16. Chen, J., Kao, S., He, H., et al.: Run, don’t walk: chasing higher FLOPS for faster neural networks[C]. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12021–12031 (2023)

  17. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  18. Wang, Q., Wu, B., Zhu, P., et al.: ECA-Net: efficient channel attention for deep convolutional neural networks[C]. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11534–11542 (2020)

  19. Woo, S., Park, J., Lee, J.Y., et al.: Cbam: convolutional block attention module[C]. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  20. Hou, Q., Zhou, D., Feng, J.: Coordinate attention for efficient mobile network design[C]. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13713–13722 (2021)

  21. Szegedy, C., Vanhoucke, V., Ioffe, S., et al.: Rethinking the inception architecture for computer vision[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)

  22. Li, H., Li, J., Wei, H., et al.: Slim-neck by GSConv: a better design paradigm of detector architectures for autonomous vehicles[J]. arXiv preprint arXiv:2206.02424 (2022)

  23. Han, K., Wang, Y., Tian, Q., et al.: Ghostnet: More features from cheap operations[C]. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1580–1589 (2020)

  24. Goodfellow, I.J., Erhan, D., Carrier, P.L., et al.: Challenges in representation learning: a report on three machine learning contests[C]. In: Neural Information Processing: 20th International Conference, ICONIP 2013, Daegu, Korea, November 3–7, 2013. Proceedings, Part III 20. Springer berlin heidelberg, pp. 117–124 (2013)

  25. Lucey, P., Cohn, J.F., Kanade, T., et al.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression[C]. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp. 94–101, IEEE (2010)

  26. Li, S., Deng, W., Du, J.P.: Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2852–2861 (2017)

  27. Sidhom, O., Ghazouani, H., Barhoumi, W.: Three-phases hybrid feature selection for facial expression recognition[J]. J. Supercomput. 1–35 (2023)

  28. Goyani, M., Patel, N.: Multi-level haar wavelet based facial expression recognition using logistic regression[J]. Indian J. Sci. Technol. 10(9), 1–9 (2017)

    Article  Google Scholar 

  29. Sang, D.V., Ha, P.T.: Discriminative deep feature learning for facial emotion recognition[C]. In: 2018 1st International Conference on Multimedia Analysis and Pattern Recognition (MAPR), pp. 1–6, IEEE (2018)

  30. Mukhopadhyay, M., Dey, A., Kahali, S.: A deep-learning-based facial expression recognition method using textural features[J]. Neural Comput. Appl. 35(9), 6499–6514 (2023)

    Article  Google Scholar 

  31. He, Y.: Facial expression recognition using multi-branch attention convolutional neural network[J]. IEEE Access 11, 1244–1253 (2022)

    Article  Google Scholar 

  32. Sun, X., Xia, P., Zhang, L., et al.: A ROI-guided deep architecture for robust facial expressions recognition[J]. Inf. Sci. 522, 35–48 (2020)

    Article  Google Scholar 

  33. Minaee, S., Minaei, M., Abdolrashidi, A.: Deep-emotion: Facial expression recognition using attentional convolutional network[J]. Sensors 21(9), 3046 (2021)

    Article  Google Scholar 

  34. Sun, X., Zheng, S., Fu, H.: ROI-attention vectorized CNN model for static facial expression recognition[J]. IEEE Access 8, 7183–7194 (2020)

    Article  Google Scholar 

  35. Fan, X., Jiang, M., Shahid, A.R., et al.: Hierarchical scale convolutional neural network for facial expression recognition. Cogn. Neurodyn. 16, 847–858 (2022). https://doi.org/10.1007/s11571-021-09761-3

    Article  Google Scholar 

  36. Fan, X., Jiang, M., Yan, H.: A deep learning based light-weight face mask detector with residual context attention and Gaussian heatmap to fight against COVID-19[J]. Ieee Access 9, 96964–96974 (2021)

    Article  Google Scholar 

  37. Zeng, G., Zhou, J., Jia, X., et al.: Hand-crafted feature guided deep learning for facial expression recognition[C]. In: 2018 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2018), pp. 423–430, IEEE (2018)

  38. Ji, L., Wu, S., Gu, X.: A facial expression recognition algorithm incorporating SVM and explainable residual neural network[J]. SIViP 17(8), 4245–4254 (2023)

    Article  Google Scholar 

  39. Zhao, S., Cai, H., Liu, H., et al.: Feature selection mechanism in CNNs for facial expression recognition[C]. BMVC. 12, 317 (2018)

    Google Scholar 

  40. Mehta, S., Rastegari, M.: Separable self-attention for mobile vision transformers[J]. arXiv preprint arXiv:2206.02680 (2022)

  41. Maaz, M., Shaker, A., Cholakkal, H., et al.: Edgenext: efficiently amalgamated cnn-transformer architecture for mobile vision applications[C]. In: European conference on computer vision, pp. 3–20, Springer, Cham (2022)

  42. Chollet, F.: Xception: deep learning with depthwise separable convolutions[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

  43. Ma, N., Zhang, X., Zheng, H.T., et al.: Shufflenet v2: practical guidelines for efficient cnn architecture design[C]. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 116–131 (2018)

Download references

Acknowledgements

This work was funded by the National Natural Science Foundation of China (Grant no.61961037) and the Industrial Support Plan Project of Gansu Provincial Department of Education (Grant no 2021CYZC-30).

Author information

Authors and Affiliations

  1. School of Physics and Electronic, Northwest Normal University, Lanzhou, 730070, People’s Republic of China

    Miaomiao Sun

  2. School of Physics and Electronic, Northwest Normal University, Engineering Research Center of Gansu Province for Intelligent Information Technology and Application, Lanzhou, 730070, People’s Republic of China

    Chunman Yan

Authors
  1. Miaomiao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunman Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S: Conceptualization, Methodology, Software, Verification,Writing - original manuscript. Y: Formal analysis, validation of methodology, writing - review, obtaining funding.

Corresponding author

Correspondence to Chunman Yan.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interest related to this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, M., Yan, C. FGENet: a lightweight facial expression recognition algorithm based on FasterNet. SIViP 18, 5939–5956 (2024). https://doi.org/10.1007/s11760-024-03283-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-024-03283-1

Keywords

Search

Navigation