Abstract
Emotion recognition is an important part of human behavior analysis. It finds many applications including human-computer interaction, driver safety, health care, stress detection, psychological analysis, forensics, law enforcement and customer care. The focus of this paper is to use a pattern recognition framework based on facial expression features and two classifiers (linear discriminant analysis and k-nearest neighbor) for emotion recognition. The extended Cohn-Kanade database is used to classify 5 emotions, namely, ‘neutral, angry, disgust, happy, and surprise’. The Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), the Walsh-Hadamard Transform (FWHT) and a new 7-dimensional feature based on condensing the Facial Action Coding System (FACS) are compared. Ensemble systems using decision level, score fusion and Borda count are also studied. Fusion of the four features leads to slightly more than a 90 % accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Huang, D., Zhang, H., Ang, K., Guan, C., Pan, Y., Wang, C., Yu, J.: Fast emotion detection from EEG using asymmetric spatial filtering. In: IEEE International Conference on Acoustics, Speech and Signal Processing, Kyoto, Japan, pp. 589–592 (2012)
Tariq, U., Huang, T.S.: Features and fusion for expression recognition - A comparative analysis. In: IEEE Computer Vision and Pattern Recognition Workshop, pp. 146–152 (2012)
Tsai, H.-H., Lai, Y.-S., Zhang, Y.-C.: Using SVM to design facial expression recognition for shape and texture features. In: International Conference on Machine Learning and Cybernetics, Qingdao, China, pp. 2697–2704 (2010)
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In: IEEE Computer Vision and Pattern Recognition Workshop, pp. 94–101 (2010)
Lucey, P., Cohn, J. F., Lucey, S., Matthews, I., Sridharan, S., Prkachin, K.: Automatically detecting pain using facial actions. In: International Conference on Affective Computing and Intelligent Interaction, pp. 1–8 (2009)
Nehra, D.K., Sharma, V., Mushtaq, H., Sharma, N., Sharma, M., Nehra, S.: Emotional intelligence and self esteem in cannabis abusers. J. Indian Acad. Appl. Psychol. 38(2), 385–393 (2012)
Jain, A.K., Ross A., Nandakumar K.: Introduction to Biometrics. Springer, New York (2011)
Campbell, J.P., Shen, W., Campbell, W.M., Schwartz, R., Bonastre, J.-F., Matrouf, D.: Forensic speaker recognition. IEEE Signal Process. Mag. 26, 95–103 (2009)
Gupta, N., Gilbert, M., Di Fabrizio, G.: Emotion detection in email customer care. Comput. Intell. 59 (2010)
Gupta, S.K., Agrwal, S., Meena Y.K., Nain, N.: A hybrid method of feature extraction for facial expression recognition. In: International Conference on Signal Image Technology and Internet-Based Systems, pp. 422–425 (2011)
Koolagudi, S.G., Kumar, N., Rao, K.S.: Speech emotion recognition using segmental level prosodic analysis. In: International Conference on Devices and Communications, pp. 1–5 (2011)
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C.M., Kazemzadeh, A., Lee, S., Neumann, U., Narayanan, S.: Analysis of emotion recognition using facial expressions, speech and multimodal Information. In: ACM International Conference on Multimodal Interfaces, State College, Pennsylvania (2004)
Cai, J., Liu, G., Hao, M.: The research on emotion recognition from ECG Signal. In: International Conference on Information Technology and Computer Science, pp. 497–500 (2009)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, New York (2001)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)
Polikar, R.: Ensemble based systems in decision making. IEEE Circuits Syst. Mag. 6, 21–45 (2006)
Bose, T.: Digital Signal and Image Processing. Wiley, New York (2004)
Smirnov, D.V., Banger, S., Davis, S.H., Muraleedharan, R., Ramachandran, R.P.: Automated human behavioral analysis framework using facial feature extraction and machine learning. In: 47th Annual Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, California (2013)
Viola, P.A., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57, 137–154 (2004)
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. 16, 321–357 (2002)
Yates, R.D., Goodman, D.J.: Probability and Stochastic Processes. Wiley, New York (1999)
Acknowledgement
This work was supported by the National Science Foundation through Grant DUE-1122296.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Smirnov, D.V., Muraleedharan, R., Ramachandran, R.P. (2015). A Comparison of Facial Features and Fusion Methods for Emotion Recognition. In: Arik, S., Huang, T., Lai, W., Liu, Q. (eds) Neural Information Processing. ICONIP 2015. Lecture Notes in Computer Science(), vol 9492. Springer, Cham. https://doi.org/10.1007/978-3-319-26561-2_68
Download citation
DOI: https://doi.org/10.1007/978-3-319-26561-2_68
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26560-5
Online ISBN: 978-3-319-26561-2
eBook Packages: Computer ScienceComputer Science (R0)