Abstract
Emotion recognition has been of great interest in psychology, machine intelligence, human–machine interaction and biomedical fields. This paper proposes a novel soft computing technique for facial emotion recognition by introducing edge- enhanced bidimensional empirical mode decomposition (EEBEMD) as a feature extraction tool for facial emotion recognition. Facial images are subjected to optimized cost function-based self-guided edge enhancement algorithm. BEMD has been applied on the edge- enhanced facial images, and the first four intrinsic mode functions (IMFs) and the residue have been calculated. On the basis of an empirical analysis, the first IMF is selected for further analysis. A proposed fusion model that consists of selected features from the gray-level co-occurrence matrix, the histogram of oriented gradients and the local ternary pattern of the IMF response is fed to a recursive feature elimination-based algorithm to select the appropriate feature subsets for classification. These feature vectors have been trained in three machine learning algorithms namely multi-class SVM, ELM with RBF kernel and k-NN classifier independently. The IMFs have been subjected to principal component analysis and linear discriminant analysis (LDA) algorithm successively for dimensionality reduction, and the facial images with different emotions have been clustered in different zones in the LDA subspace. The proposed method demonstrates promising accuracy when tested on the JAFFE database, Cohn–Kanade database and the eNTERFACE database.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ali H (2015) Facial emotion recognition using empirical mode decomposition. Expert Syst Appl 42(3):1261–1277
Bhattacharya A, Choudhury D, Dey D (2016) Emotion recognition from facial image analysis Using composite similarity measure aided bidimensional empirical mode decomposition. In: IEEE conference on control, measurement and instrumentation, Jadavpur, Kolkata
Chakraborti T, Chatterjee A, Konar A, Roy AH (2015) Automated emotion recognition employing a novel modified binary quantum-behaved gravitational search algorithm with differential mutation. Expert Syst Appl 32(4):552–530
Chatterjee S , Shi H (2010) A novel neuro fuzzy approach to human emotion determination. In: Digital image computing: techniques and applications, pp 282–287
Clausi David A (2002) An analysis of co-occurrence texture statistics as a function of grey level quantization. Can J Remote Sens 28(1):45–62
Dahmane M, Meunier J (2011) Emotion recognition using dynamic grid-based HOG features. In: IEEE gesture recognition, Santa Barbara, CA, USA
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Internal computer society conference on computer visions and pattern recognition
Deng HB, Jin LW, Zhen LX, Huang JC (2005) A new facial expression recognition method based on local Gabor filter bank and PCA plus LDA. Int J Inform Technol 11(11):86–96
Ekman P, Frisen W (1978) Facial action coding system (FACS). Consulting Psychologists Press, Palo Alto
Eleyan A, Demirel H (2011) Co-occurrence matrix and its statistical features as a new approach of face recognition. Turk J ElecEng Comp Sci 19(1):97–107
Gomathi V, Ramar K, Jeevakumar AS (2009) Human facial expression recognition using MANFIS model. Int J Comput Sci Eng 3(2):93–97
Gu WF, Venkatesh YV, Xiang C (2010) A novel application of self-organizing network for facial expression recognition from radial encoded contours. Soft Comput 14:113–122
Gupta SK, Agarwal S, Meena YK, Nain N (2011) A hybrid method of feature extraction for facial expression recognition. In: 7th international conference on signal image technology & internet-based systems, pp 422–425
Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern 3(6):610–621
He K, Sun J, Tang X (2013) Guided image filtering. IEEE Trans Pattern Anal Mach Intell 35(6):1397–1409
Huang et al (2002) Emotion recognition from cauchy naive based classifier. In: 16th international conference on pattern recognition, vol 1
Huang N et al (1998) The empirical mode decomposition and the Hilbert spectrum for non-linear and non-stationary time series analysis. Proc R Soc Lond A 454:903–995
Huang CL, Wang CJ (2006) A GA-based feature selection and parameters optimization for support vector machines. Expert Syst Appl 31(2):231–240
Hussain S, Triggs B (2010) Feature sets and dimensionality reduction for visual object detection. In: Proceedings of the British machine vision conference, pp 112.1–112.10
Ilbeygi M, Hosseini HS (2012) A novel fuzzy facial expression recognition system based on facial feature extraction from color face images. Eng Appl Artif Intell 25:130–146
Jin Y, Ruan QQ (2009) Face recognition using Gabor-based improved supervised locality preserving projections. Comput Inform 28:81–95
Kanade T, Cohn JF, Tian YL (2000) Comprehensive database for facial expression analysis. In: Proceedings of 4th IEEE international conference on automatic face and gesture recognition (FG’00), pp 46–53
Kazmi SB, Qurat-ul-Ain, Jaffar MA (2012) Wavelet-based facial expression recognition using a bank of support vector machines. Soft Comput 16(3):369–379
Kharat GU, Dudul SV (2009) Emotion recognition from facial expression using neural networks. In: Hippe ZS, Kulikowski JL (eds) Human–computer systems interaction, AISC 60, pp 207–219
Kim DJ, Song WK, Han JS, Zenn Bien Z (2003) Soft computing based intention reading techniques as a means of human-robot interaction for human centered system. Soft Comput Fusion Found Methodol Appl 7(3):160–166
Koley B, Dey D (2012) An ensemble system for automatic sleep stage classification using single channel EEG signal. Comput Biol Med 42(12):1186–1195
Lee CM, Narayanan S, Pieraccini R (2001) Recognition of negative emotions from the speech signal. In: IEEE workshop on automatic speech recognition and understanding, 2001
Lin HJ, Wang SY, Yen SH, Kao YT (2005) Face detection based on skin color segmentation and neural network. In: International conference on neural network and brain. Beijing, pp 1144 – 1149
Lin X, Yang F, Zhou L, Yin P, Kong H, Xing W, Lu X, Jia L, Wang Q, Xu G (2012) A support vector machine-recursive feature elimination feature selection method based on artificial contrast variables and mutual information. J Chromatogr 910:149–155
Liu Z, Wang S (2011) Emotion recognition using hidden markov models from facial temperature sequence. In: Affective computing and intelligent interaction. vol 6975. Springer, Heidelberg, pp 240–247
Martin O, Kotsia I, Macq B, Pitas I (2005) The eNTERFACE’05 audio-visual emotion database. In: Proceedings of the first IEEE workshop on multimedia database management, Atlanta
Matsuno K, Lee CW, Kimura S , Tsuji S (1995) Automatic recognition of human facial expressions. In: Proceeding of the fifth international conference on computer vision (ICCV’ 95)
Murthy GRS, Jadon RS (2009) Effectiveness of Eigenspaces for facial expressions recognition. Int J Comput Theory Eng 1(5):1793–8201
Nunes JC, Bouaoune Y, Delechelle E, Niang O, Bunel P (2003) Image analysis by bidimensional empirical mode decomposition. Image Vis Comput 21(2003):1019–1026
Oliveira LS, Koerich RL, Mansano M (2011) 2D Principal Component Analysis for Face and Facial-Expression Recognition. J Comput Sci Eng 13(3):9–13. doi: 10.1109/MCSE.2010.149
Rahulamathavan Y, Phan RCW, Chambers JA, Parish DJ (2013) Facial expression recognition in the encrypted domain based on local fisher discriminant analysis. IEEE Trans Affect Comput 4(1):83–92
Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27:803–816
Shih FY, Chuang CF, Wang PSP (2008) Performance comparisons of facial expression recognition in JAFFE database. Int J Pattern Recognit 22(3):445–459
Soh L, Tsatsoulis C (1999) Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices. IEEE Trans Geosci Remote Sens 37(2):780–795
Sugiyama M (2006) Local fisher discriminant analysis for supervised dimensionality reduction. In: Proceedings of the 23rd international conference on machine learning, Pittsburgh
Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19:1635–1650
Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86
Visutsak P (2013) Emotion classification through lower facial expressions using adaptive support vector machines. J Man Mach Technol 2(1):12–20
Wang Z, Ruan Q (2010) Facial expression based orthogonal local fisher discriminant analysis. In: Proceedings of ICSP 2010, pp 1358–1361
Xu X, Liang J, Lv S, Wu Q (2014) Human facial expression analysis based on granule LPP. Int J Mach Learn Cybern 5:907–921
Zhang S, Zhao X, Lei B (2012) Facial expression recognition based on local binary patterns and LFDA. WSEAS Trans Signal Process 8(1):21–31
Zhang L, Tjondronegoro D (2011) Facial expression recognition using facial movement features. IEEE Trans Affect Comput 2(4):219–229
Zhao X, Zhang S (2011) Facial expression recognition based on local binary patterns and kernel discriminant isomap. Sensors 11(10):9573–9588
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Bhattacharya, A., Choudhury, D. & Dey, D. Edge-enhanced bi-dimensional empirical mode decomposition-based emotion recognition using fusion of feature set. Soft Comput 22, 889–903 (2018). https://doi.org/10.1007/s00500-016-2395-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-016-2395-4