Abstract
Emotion identification is beginning to be considered as an essential feature in human-computer interaction. However, most of the studies are mainly focused on facial expression classifications and speech recognition and not much attention has been paid until recently to physiological pattern recognition. In this paper, an integrative approach is proposed to emotional interaction by fusing multi-modal signals. Subjects are exposed to pictures selected from the International Affective Picture System (IAPS). A feature extraction procedure is used to discriminate between four affective states by means of a Mahalanobis distance classifier. The average classifications rate (74.11%) was encouraging. Thus, the induced affective state is mirrored through an avatar by changing its facial characteristics and generating a voice message sympathising with the user’s mood. It is argued that multi-physiological patterning in combination with anthropomorphic avatars may contribute to the enhancement of affective multi-modal interfaces and the advancement of machine emotional intelligence.
Chapter PDF
Similar content being viewed by others
References
Bradley, M.M., Codispoti, M., Cuthbert, B.N., Lang, P.J.: Emotion and motivation I: Defensive and appetitive reactions in picture processing. Emotion 1(3), 276–298 (2001)
Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International Affective Picture System (IAPS): Technical Manual and Affective Ratings. NIMH Center for the Study of Emotion and Attention (1997)
Picard, R.W., et al.: Affective learning – A Manifesto. BT Technology Journal 22(4), 253–269 (2004)
Reeves, B., Nass, C.: The Media Equation: How People Treat Computers. Cambridge Univ. Press, Cambridge (1996)
Cockton, G.: Editorial:From doing to being:bringing emotion into interaction. Interacting with Computers 14(2), 89–92 (2002)
Picard, R.W., Healey, J.: Affective Wearables. Personal and Ubiquitous Computing 1(4), 231–240 (1997)
Konstantinidis, E.I., Bamidis, P.D., Koufogiannis, D.: Development of A Generic And Flexible Human Body Wireless Sensor Network. In: 6th European Symposium on Biomedical Engineering (ESBME 2008), Chania, Crete Island, Greece (2008)
Healey, J., Picard, R.W.: Digital processing of affective signals. In: International Conference on Acoustics, Speech and Signal Processing, Seattle, USA, vol. 6, pp. 3749–3752 (1998)
Picard, R.W., Vyzas, E., Healey, J.: Toward Machine Emotional Intelligence: Analysis of Affective Physiological State. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(10), 1175–1191 (2001)
Haag, A., Goronzy, S., Schaich, P., Williams, J.: Emotion recognition Using Bio-Sensors: First Steps Towards an Automatic System. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS, vol. 3068, pp. 36–48. Springer, Heidelberg (2004)
Nasoz, F., Lisetti, C.L., Alvarez, K., Finkelstein, N.: Emotion Recognition from Physiological Signals for User Modeling of Affect. In: 3rd Workshop on Affective and Attitude User Modeling, USA (2003)
Bamidis, P.D., Luneski, A., Vivas, A., Papadelis, C., Maglaveras, N., Pappas, C.: Multi-channel physiological sensing of human emotion: insights into Emotion-Aware Computing using Affective Protocols, Avatars and Emotion. Stud. Health Technol. Inform. 129(Pt 2), 1068–1072 (2007)
Frantzidis, C., Lithari, C., Vivas, A., Papadelis, C., Pappas, C., Bamidis, P.D.: Towards Emotion Aware Computing: a Study of Arousal Modulation with Multichannel Event Related Potentials, Delta Oscillatory Activity and Skin Conductivity Responses. In: 8th IEEE International Conference on BioInformatics and BioEngineering (BIBE 2008), Athens, Greece, pp. 1–6 (2008)
Brody, L.R.: Gender differences in emotional development: A review of theories and research. Journal of Personality 53(2), 102–149 (2006)
Luneski, A., Bamidis, P.D.: Towards an Emotion Specification Method: Representing Emotional Physiological Signals. In: Proc. of the 20th IEEE International Symposium on Computer-Based Medical Systems (CBMS 2007), Maribor, Slovenia, pp. 313–320 (2007)
The free Haptek Guide, http://haptek.com/ (last access 24/2/2009)
Loquendo: global supplier of speech recognition and speech synthesis technology and solutions, http://www.loquendo.com/ (last access 24/2/2009)
Cincotti, F., et al.: The use of EEG modifications due to motor imagery for brain-computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering 11(2), 131–133 (2003)
Luneski, A., Konstantinidis, E.I., Hitoglou-Antoniadou, M., Bamidis, P.D.: Affective Computer-Aided Learning for Autistic Children. In: 1st Workshop on Child, Computer and Interaction, ICMI 2008, Chania, Crete, Greece, October 20-23 (2008)
Lisetti, C.L., Nasoz, F.: Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals. EURASIP Journal on Applied Signal Processing 2004(11), 1672–1687 (2004)
Boulos, K., Maged, N., Wheeler, S.: The emerging Web 2.0 social software: an enabling suite of sociable technologies in health and health care education. Health Information and Libraries Journal 24(1), 2–23 (2007)
Vilon, O., Lisetti, C.L.: Toward Recognizing Individual’s Subjective Emotion from Physiological Signals in Practical Application. In: Proc. of 20th IEEE International Symposium on Computer-Based Medical Systems (CBMS 2007), Maribor, Slovenia, pp. 357–362 (2007)
Luneski, A., Bamidis, P.D., Hitoglou-Antoniadou, M.: Affective computing and medical informatics: state of the art in emotion-aware medical applications. Stud. Health Technol. Inform. 136, 517–522 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bamidis, P.D. et al. (2009). An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence. In: Jacko, J.A. (eds) Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction. HCI 2009. Lecture Notes in Computer Science, vol 5612. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02580-8_62
Download citation
DOI: https://doi.org/10.1007/978-3-642-02580-8_62
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02579-2
Online ISBN: 978-3-642-02580-8
eBook Packages: Computer ScienceComputer Science (R0)