Abstract
A multi-modal emotion perceptual experiment is conducted cross-culturally to investigate the difference of expressing and perceiving emotions across Chinese and Japanese. Focus is on the cultural effect on the interaction between the combination of vocal and facial expression and perception. In this paper, part of the perceptual results is reported for the AV-conflicting stimuli produced by a Chinese female speaker and perceived by Chinese and Japanese listeners. The results support the assumptions that (i) When listeners decoding the conflicting AV stimuli, they might rely on some modality more than another across different emotions. (ii) Although common psychological factor contributes to the emotional communication, the decoding of conflicting AV information will be affected by culture background, and (iii) the emotional McGurk effect exists, and it may also be related to cultural norms of the encoder/listener.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Darwin, C.: The expression of the emotions in man and animals. John Murray, Oxford University Press, London, New York (1988, Original work published in 1872) (reprinted with introduction, afterword, and commentary by Ekman, P. (ed.))
Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. Journal of Personality and Social Psychology 17, 124–129 (1971)
Izard, C.E.: Innate and universal facial expressions: Evidence from developmental and cross-cultural research. Psychological Bulletin 115, 288–299 (1994)
Scherer, K.R.: A Cross-Cultural Investigation of Emotion Inferences from Voice and Speech: Implications for Speech Technology. In: ICSLP 2000 (2000)
Scherer, K.R., Banse, R., Wallbott, H.G.: Emotion Inferences from Vocal Expression Correlate Across Languages and Cultures. Journal of Cross-Cultural Psychology 32, 76 (2001)
Scherer, K.R.: Vocal communication of emotion: A review of research paradigms. Speech Communication 40, 227–256 (2003)
Abelin, A., Allwood, J.: Cross Linguistic Interpretation of Emotional Prosody. In: Proc. ISCA Workshop on Speech and Emotion, Belfast (2000)
Yanushevskaya, I., Chasaide, A.N., Gobl, C.: Cross-Language Study of Vocal Correlates of Affective States. In: Interspeech 2008 (2008)
Abelin, Å.: Cross-Cultural Multimodal Interpretation of Emotional Expressions – An Experimental Study of Spanish and Swedish. In: SP 2004 (2004)
Huang, C.F., Akagi, M.: A three-layered model for expressive speech perception. Speech Communication 50, 810–828 (2008)
Barkhuysen, P., Krahmer, E., Swerts, M.: Incremental perception of acted and real emotional speech. In: ICPHS (2007)
Fagel, S.: Emotional McGurk effect. In: Speech Prosody 2006 (2006)
Grandjean, D., Scherer, K.R.: Unpacking the Cognitive Architecture of Emotion Processes. Emotion 8(3), 341–351 (2008)
Erickson, D.: Expressive speech: Production, perception and application to speech synthesis. Japan Acoust. Sci. & Tech. 26, 4 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, A., Fang, Q., Jia, Y., Dang, J. (2013). Emotional McGurk Effect? A Cross-Cultural Investigation on Emotion Expression under Vocal and Facial Conflict. In: Sun, M., Zhang, M., Lin, D., Wang, H. (eds) Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data. NLP-NABD CCL 2013 2013. Lecture Notes in Computer Science(), vol 8202. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41491-6_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-41491-6_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41490-9
Online ISBN: 978-3-642-41491-6
eBook Packages: Computer ScienceComputer Science (R0)