article

Vocal communication of emotion: a review of research paradigms

Author:

Klaus R. SchererAuthors Info & Claims

Speech Communication, Volume 40, Issue 1-2

Pages 227 - 256

https://doi.org/10.1016/S0167-6393(02)00084-5

Published: 01 April 2003 Publication History

Abstract

The current state of research on emotion effects on voice and speech is reviewed and issues for future research efforts are discussed. In particular, it is suggested to use the Brunswikian lens model as a base for research on the vocal communication of emotion. This approach allows one to model the complete process, including both encoding (expression), transmission, and decoding (impression) of vocal emotion communication. Special emphasis is placed on the conceptualization and operationalization of the major elements of the model (i.e., the speaker's emotional state, the listener's attribution, and the mediating acoustic cues). In addition, the advantages and disadvantages of research paradigms for the induction or observation of emotional expression in voice and speech and the experimental manipulation of vocal cues are discussed, using pertinent examples drawn from past and present research.

References

[1]

Abadjieva, E., Murray, I.R., Arnott, J.L., 1995. Applying analysis of human emotional speech to enhance synthetic speech. In: Proc. Eurospeech 95, pp. 909-912.

[2]

Allport, G.W., Cantril, H., 1934. Judging personality from voice. J. Soc. Psychol. 5, 37-55.

[3]

Alpert, M., Kurtzberg, R.L., Friedhoff, A.J., 1963. Transient voice changes associated with emotional stimuli. Arch. Gen. Psychiatr. 8, 362-365.

[4]

Ambady, N., Koo, J., Lee, F., Rosenthal, R., 1996. More than words: Linguistic and nonlinguistic politeness in two cultures. J. Pers. Soc. Psychol. 70 (5), 996-1011.

[5]

Amir, N., Ron, S., 1998. Towards an automatic classification of emotions in speech. In: Proc. ICSLP 98, Vol. 3, pp. 555-558.

[6]

Anolli, L., Ciceri, R., 1997. The voice of deception: Vocal strategies of naive and able liars. J. Nonverb. Behav. 21 (4), 259-284.

[7]

Baber, C., Noyes, J., 1996. Automatic speech recognition in adverse environments. Hum. Fact. 38 (1), 142-155.

[8]

Bachorowski, J.A., 1999. Vocal expression and perception of emotion. Current Direct. Psychol. Sci. 8 (2), 53-57.

[9]

Bachorowski, J.A., Owren, M.J., 1995. Vocal expression of emotion: Acoustic properties of speech are associated with emotional intensity and context. Psychol. Sci. 6 (4), 219-224.

[10]

Banse, R., Scherer, K.R., 1996. Acoustic profiles in vocal emotion expression. J. Pers. Soc. Psychol. 70 (3), 614-636.

[11]

Bänziger, T., Scherer, K.R., 2001. Relations entre caractéristiques vocales perçues et émotions attribuées. In: Proc. Journées Prosodie, Grenoble, 10-11 October 2001.

[12]

Bonner, M.R., 1943. Changes in the speech pattern under emotional tension. Amer. J. Psychol. 56, 262-273.

[13]

Borod, J.C., Pick, L.H., Hall, S., Sliwinski, M., Madigan, N., Obler, L.K., Welkowitz, J., Canino, E., Erhan, H.M., Goral, M., Morrison, C., Tabert, M., 2000. Relationships among facial, prosodic, and lexical channels of emotional perceptual processing. Cogn. Emot. 14 (2), 193-211.

[14]

Bortz, J., 1966. Physikalisch-akustische Korrelate der vokalen Kommunikation. Arbeiten aus dem psychologischen Institut der Universität Hamburg, 9 {Reports of the Psychology Department, University of Hamburg, Vol. 9}.

[15]

Breitenstein, C., Van Lancker, D., Daum, I., 2001. The contribution of speech rate and pitch variation to the perception of vocal emotions in a German and an American sample. Cogn. Emot. 15 (1), 57-79.

[16]

Brunswik, E., 1956. Perception and the Representative Design of Psychological Experiments. University of California Press, Berkeley.

[17]

Bryman, A., Cramer, D., 1990. Quantitative Data Analysis for Social Scientists. Taylor and Francis, Hove, Great Britain.

[18]

Burkhardt, F., Sendlmeier, W.F., 2000. Verification of acoustical correlates of emotional speech using formant-synthesis. In: Proc. ISCA Workshop on Speech and Emotion, 5-7 September 2000, Newcastle, Northern Ireland, pp. 151-156.

[19]

Caffi, C., Janney, R.W., 1994. Toward a pragmatics of emotive communication. J. Pragmat. 22, 325-373.

[20]

Cahn, J., 1990. The generation of affect in synthesised speech. J. Amer. Voice I/O Soc. 8, 1-19.

[21]

Carlson, R., 1992. Synthesis: Modeling variability and constraints. Speech Communication 11 (2-3), 159-166.

Digital Library

[22]

Carlson, R., Granström, B., Nord, L., 1992. Experiments with emotive speech--acted utterances and synthesized replicas. In: Proc. ICSLP 92, Vol. 1, pp. 671-674.

[23]

Coleman, R.F., Williams, R., 1979. Identification of emotional states using perceptual and acoustic analyses. In: Lawrence, V., Weinberg, B. (Eds.), Transcript of the 8th Symposium: Care of the Professional Voice, Part I. The Voice Foundation, New York.

[24]

Cosmides, L., 1983. Invariances in the acoustic expression of emotion during speech. J. Exp. Psychol.: Hum. Percept. Perform. 9, 864-881.

[25]

Costanzo, F.S., Markel, N.N., Costanzo, P.R., 1969. Voice quality profile and perceived emotion. J. Couns. Psychol. 16, 267-270.

[26]

Cowie, R., Douglas-Cowie, E., 1996. Automatic statistical analysis of the signal and prosodic signs of emotion in speech. In: Proc. ICSLP'96, Philadelphia, pp. 1989-1992.

[27]

Darwin, C., 1872. The Expression of Emotions in Man and Animals. John Murray, London (third ed., P. Ekman (Ed.), 1998, Harper Collins, London).

[28]

Davitz, J.R., 1964. The Communication of Emotional Meaning. McGraw-Hill, New York.

[29]

de Gelder, B., 2000. Recognizing emotions by ear and by eye. In: Lane, R.D., Nadel, L. (Eds.), Cognitive Neuroscience of Emotion. Oxford University Press, New York, pp. 84-105.

[30]

de Gelder, B., Vroomen, J., 2000. Bimodal emotion perception: Integration across separate modalities, cross-modal perceptual grouping or perception of multimodal events? Cogn. Emot. 14, 321-324.

[31]

Duncan, G., Laver, J., Jack, M.A., 1983. A psychoacoustic interpretation of variations in divers' voice fundamental frequency in a pressured helium-oxygen environment. Work in Progress, 16, 9-16. Department of Linguistics, University of Edinburgh, UK.

[32]

Ekman, P., 1972. Universals and cultural differences in facial expression of emotion. In: Cole, J.R. (Ed.), Nebraska Symposium on Motivation. University of Nebraska Press, Lincoln, pp. 207-283.

[33]

Ekman, P., 1992. An argument for basic emotions. Cogn. Emot. 6 (3/4), 169-200.

[34]

Ekman, P., 1994. Strong evidence for universals in facial expressions: A reply to Russell's mistaken critique. Psychol. Bull. 115, 268-287.

[35]

Ekman, P., O'Sullivan, M., Friesen, W.V., Scherer, K.R., 1991. Face, voice, and body in detecting deception. J. Nonverb. Behav. 15, 125-135.

[36]

Eldred, S.H., Price, D.B., 1958. A linguistic evaluation of feeling states in psychotherapy. Psychiatry 21, 115-121.

[37]

Erickson, D., Fujimura, O., Pardo, B., 1998. Articulatory correlates of prosodic control: Emphasis and emotion. Lang. Speech 41, 399-417.

[38]

Fairbanks, G., Hoaglin, L.W., 1941. An experimental study of the durational characteristics of the voice during the expression of emotion. Speech Monogr. 8, 85-90.

[39]

Fairbanks, G., Pronovost, W., 1939. An experimental study of the pitch characteristics of the voice during the expression of emotion. Speech Monogr. 6, 87-104.

[40]

Feldman, R.S., Rimé, B. (Eds.), 1991. Fundamentals of Nonverbal Behavior. Cambridge University Press, New York.

[41]

Fichtel, C., Hammerschmidt, K., Jürgens, U., 2001. On the vocal expression of emotion. A multi-parametric analysis of different states of aversion in the squirrel monkey. Behaviour 138 (1), 97-116.

[42]

Fonagy, I., 1978. A new method of investigating the perception of prosodic features. Lang. Speech 21, 34-49.

[43]

Fonagy, I., 1983. La vive voix. Payot, Paris.

[44]

Fonagy, I., Magdics, K., 1963. Emotional patterns in intonation and music. Z. Phonetik 16, 293-326.

[45]

Frank, M.G., Stennett, J., 2001. The forced-choice paradigm and the perception of facial expressions of emotion. J. Pers. Soc. Psychol. 80 (1), 75-85.

[46]

Frick, R.W., 1985. Communicating emotion: The role of prosodic features. Psychol. Bull. 97, 412-429.

[47]

Friend, M., Farrar, M.J., 1994. A comparison of contentmasking procedures for obtaining judgments of discrete affective states. J. Acoust. Soc. Amer. 96 (3), 1283-1290.

[48]

Frolov, M.V., Milovanova, G.B., Lazarev, N.V., Mekhedova, A.Y., 1999. Speech as an indicator of the mental status of operators and depressed patients. Hum. Physiol. 25 (1), 42-47.

[49]

Gifford, R., 1994. A lens-mapping framework for understanding the encoding and decoding of interpersonal dispositions in nonverbal behavior. J. Pers. Soc. Psychol. 66, 398-412.

[50]

Gramming, P., Sundberg, J., 1988. Spectrum factors relevant to phonetogram measurement. J. Acoust. Soc. Amer. 83, 2352-2360.

[51]

Granström, B., 1992. The use of speech synthesis in exploring different speaking styles. Speech Communication 11, 347-355.

Digital Library

[52]

Green, R.S., Cliff, N., 1975. Multidimensional comparison of structures of vocally and facially expressed emotion. Percept. Psychophys. 17, 429-438.

[53]

Gross, J.J., Levenson, R.W., 1997. Hiding feelings: The acute effects of inhibiting negative and positive emotion. J. Abnorm. Psychol. 106 (1), 95-103.

[54]

Hammond, K.R., Stewart, T.R. (Eds.), 2001. The Essential Brunswik: Beginnings, Explications, Applications. Oxford University Press, New York.

[55]

Hargreaves, W.A., Starkweather, J.A., Blacker, K.H., 1965. Voice quality in depression. J. Abnorm. Psychol. 70, 218-220.

[56]

Harper, R.G., Wiens, A.N., Matarazzo, J.D., 1978. Nonverbal Communication: The State of the Art. Wiley, New York.

[57]

Hauser, M.D., 2000. The sound and the fury: Primate vocalizations as reflections of emotion and thought. In: Wallin, N,L., Merker, B. (Eds.), The Origins of Music. The MIT Press, Cambridge, MA, USA, pp. 77-102.

[58]

Havrdova, Z., Moravek, M., 1979: Changes of the voice expression during suggestively influenced states of experiencing. Activitas Nervosa Superior 21, 33-35.

[59]

Helfrich, H., Standke, R., Scherer, K.R., 1984. Vocal indicators of psychoactive drug effects. Speech Communication 3, 245-252.

[60]

Herzog, A., 1933. Stimme und Persönlichkeit {Voice and personality}. Z. Psychol. 130, 300-379.

[61]

Heuft, B., Portele, T., Rauth, M., 1996. Emotions in time domain synthesis. In: Proc. ICSLP 96, Vol. 3, Philadelphia, USA, pp. 1974-1977.

[62]

Hicks, J.W., 1979. An acoustical/temporal analysis of emotional stress in speech. Diss. Abstr. Intern. 41 (4-A).

[63]

Hochschild, A.R., 1983. The Managed Heart: The Commercialization of Human Feeling. University of California Press, Berkeley.

[64]

Höffe, W.L., 1960. Über Beziehungen von Sprachmelodie und Lautstärke {On the relationships between speech melody and intensity}. Phonetica 5, 129-159.

[65]

Huttar, G.L., 1968. Relations between prosodic variables and emotions in normal American English utterances. J. Speech Hear. Res. 11, 481-487.

[66]

Iida, A., Campbell, N., Iga, S., Higuchi, F., Yasumura, M., 1998. Acoustic nature and perceptual testing of corpora of emotional speech. In: Proc. ICSLP 98, Sydney, Vol. 4, pp. 1559-1562.

[67]

Isserlin, M., 1925. Psychologisch-phonetische Untersuchungen. II. Mitteilung {Psychological-phonetic studies. Second communication}. Z. Gesamte Neurol. Psychiatr. 94, 437-448.

[68]

Izard, C.E., 1971. The Face of Emotion. Appleton-Century-Crofts, New York.

[69]

Izard, C.E., 1977. Human Emotions. Plenum Press, New York.

[70]

Johannes, B., Petrovitsh Salnitski, V., Gunga, H.C., Kirsch, K., 2000. Voice stress monitoring in space - possibilities and limits. Aviat. Space Environ. Md. 71 (9, Section 2, Suppl.), A58-A65.

[71]

Johnstone, T., 2001. The communication of affect through modulation of non-verbal vocal parameters. Ph.D. Thesis, University of Western Australia.

[72]

Johnstone, T., Scherer, K.R., 2000. Vocal communication of emotion. In: Lewis, M., Haviland, J. (Eds.), Handbook of emotion, second ed. Guilford, New York, pp. 220-235.

[73]

Johnstone, T., van Reekum, C.M., Scherer, K.R., 2001. Vocal correlates of appraisal processes. In: Scherer, K.R., Schorr, A., Johnstone, T. (Eds.), Appraisal Processes in Emotion: Theory, Methods, Research. Oxford University Press, New York and Oxford, pp. 271-284.

[74]

Juslin, P.N., 2000. Cue utilization in communication of emotion in music performance: Relating performance to perception. J. Exp. Psychol.: Hum. Percep. Perform. 26, 1797-1813.

[75]

Juslin, P.N., Laukka, P., 2001. Impact of intended emotion intensity on cue utilization and decoding accuracy in vocal expression of emotion. Emotion 1 (4), 381-412.

[76]

Kaiser, L., 1962. Communication of affects by single vowels. Synthese 14, 300-319.

[77]

Kappas, A., 1997. His master's voice: Acoustic analysis of spontaneous vocalizations in an ongoing active coping task. In: Thirty-Seventh Annual Meeting of the Society for Psychophysiological Research, Cape Cod.

[78]

Kappas, A., Hess, U., Scherer, K.R., 1991. Voice and emotion. In: Rimé, B., Feldman, R.S. (Eds.), Fundamentals of Nonverbal Behavior. Cambridge University Press, Cambridge and New York, pp. 200-238.

[79]

Karlsson, I., Bänziger, T., Dankovicova, J., Johnstone, T., Lindberg, J, Melin, H., Nolan, F., Scherer, K.R., 1998. Speaker verification with elicited speaking styles in the VeriVox project. In: Proc. RLA2C, Avignon, pp. 207-210.

Digital Library

[80]

Karlsson, I., Banziger, T., Dankovicova, J., Johnstone, T., Lindberg, J., Melin, H., Nolan, F., Scherer, K.R., 2000. Speaker verification with elicited speaking styles in the VeriVox project. Speech Communication 31 (2-3), 121-129.

Digital Library

[81]

Kennedy, G., 1972. The Art of Rhetoric in the Roman World. 300 BC-AD 300. Princeton University Press, Princeton, NJ.

[82]

Kienast, M., Paeschke, A., Sendlmeier, W.F., 1999. Articulatory reduction in emotional speech. In: Proc. Eurospeech 99, Budapest, Vol. 1, pp. 117-120.

[83]

Klasmeyer, G., 1999. Akustische Korrelate des stimmlich emotionalen Ausdrucks in der Lautsprache. In: Wodarz, H.-W., Heike, G., Janota, P., Mangold, M. (Eds.), Forum Phoneticum, Vol. 67. Hector, Frankfurt am Main.

[84]

Klasmeyer, G., 2000. An automatic description tool for time-contours and long-term average voice features in large emotional speech databases. In: Proc. ISCA Workshop Speech and Emotion, Newcastle, Northern Ireland, pp. 66-71.

[85]

Klasmeyer, G., Meier, T., 1999. Rhythm in Emotional Speech, DAGA-Tag der ASA-Tagung 99 in Berlin, CD-Rom zur ASA-EAA-DEGA Gemeinschaftstagung Berlin 99.

[86]

Klasmeyer, G., Sendlmeier, W.F., 1997. The classification of different phonation types in emotional and neutral speech. Forensic Linguist. 4, 104-124.

[87]

Klasmeyer, G., Sendlmeier, W.F., 1999. Voice and emotional states. In: Kent, R., Ball, M. (Eds.), Voice Quality Measurement. Singular Publishing Group, San Diego, CA, pp. 339-359.

[88]

Knapp, M.L., 1972. Nonverbal Communication in Human Interaction. Holt Rinehart and Winston, New York.

[89]

Kotlyar, G.M., Morozov, V.P., 1976. Acoustical correlates of the emotional content of vocalized speech. J. Acoust. Acad. Sci. USSR 22, 208-211.

[90]

Kuroda, I., Fujiwara, O., Okamura, N., Utusuki, N., 1979. Method for determinating pilot stress through analysis of voice communication. Aviat. Space Environ. Md. 47, 528-533.

[91]

Ladd, D.R., Silverman, K.E.A., Tolkmitt, F., Bergmann, G., Scherer, K.R., 1985. Evidence for the independent function of intonation contour type, voice quality, and F0 range in signaling speaker affect. J. Acoust. Soc. Amer. 78, 435-444.

[92]

Laver, J., 1980. The Phonetic Description of Voice Quality. Cambridge University Press, Cambridge.

[93]

Laver, J., 1991. The Gift of Speech. Edinburgh University Press, Edinburgh.

[94]

Lavner, Y., Gath, I., Rosenhouse, J., 2000. The effects of acoustic modifications on the identification of familiar voices speaking isolated vowels. Speech Communication 30 (1), 9-26.

Digital Library

[95]

Levin, H., Lord, W., 1975. Speech pitch frequency as an emotional state indicator. IEEE Trans. Systems, Man Cybernet. 5, 259-273.

[96]

Lieberman, P., Michaels, S.B., 1962. Some aspects of fundamental frequency and envelope amplitude as related to the emotional content of speech. J. Acoust. Soc. Amer. 34, 922-927.

[97]

Mahl, G.F., Schulze, G., 1964. Psychological research in the extralinguistic area. In: Sebeok, T.A., Hayes, A.S., Bateson, M.C. (Eds.), Approaches to Semiotics. Mouton, The Hague, pp. 51-124.

[98]

Markel, N.N., Bein, M.F., Phillis, J.A., 1973. The relation between words and tone of voice. Lang. Speech 16, 15-21.

[99]

Massaro, D.W., 2000. Multimodal emotion perception: Analogous to speech processes. In: Proc. ISCA Workshop on Speech and Emotion, 5-7 September 2000, Newcastle, Northern Ireland, pp. 114-121.

[100]

Morris, J.S., Scott, S.K., Dolan, R.J., 1999. Saying it with feeling: Neural responses to emotional vocalizations. Neuropsychologia 37 (10), 1155-1163.

[101]

Moses, P., 1954. The Voice of Neurosis. Grune and Stratton, New York.

[102]

Mozziconacci, S.J.L., 1998. Speech variability and emotion: Production and perception. Ph.D. Thesis, Technical University Eindhoven, Eindhoven.

[103]

Morton, J.B., Trehub, S.E., 2001. Children's understanding of emotion in speech. Child Develop. 72 (3), 834-843.

[104]

Murray, I.R., Arnott, J.L., 1993. Toward a simulation of emotion in synthetic speech: A review of the literature on human vocal emotion. J. Acoust. Soc. Amer. 93, 1097-1108.

[105]

Murray, I.R., Arnott, J.L., Rohwer, E.A., 1996. Emotional stress in synthetic speech: Progress and future directions. Speech Communication 20 (1-2), 85-91.

Digital Library

[106]

Niwa, S., 1971. Changes of voice characteristics in urgent situations. Reports of the Aeromedical Laboratory, Japan Air Self-Defense Force, 11, pp. 246-251.

[107]

Ofuka, E., McKeown, J.D., Waterman, M.G., Roach, P.J., 2000. Prosodic cues for rated politeness in Japanese speech. Speech Communication 32 (3), 199-217.

Digital Library

[108]

Ostwald, P.F., 1964. Acoustic manifestations of emotional disturbances. In: Disorders of Communication, Vol. 42. Research Publications, pp. 450-465.

[109]

Paeschke, A., Kienast, M., Sendlmeier, W.F., 1999. F0- contours in emotional speech. In: Proc. ICPhS 99, San Francisco, Vol. 2, pp. 929-933.

[110]

Pear, T.H., 1931. Voice and Personality. Chapman and Hall, London.

[111]

Pell, M.D., 1998. Recognition of prosody following unilateral brain lesion: Influence of functional and structural attributes of prosodic contours. Neuropsychologia 36 (8), 701-715.

[112]

Pereira, C., Watson, C., 1998. Some acoustic characteristics of emotion. In: Proc. ICSLP 98, Sydney, Vol. 3, pp. 927-934.

[113]

Picard, R.W., 1997. Affective Computing. MIT Press, Cambridge, MA.

[114]

Pittenger, R.E., Hockett, C.F., Danehy, J.J., 1960. The First Five Minutes: A Sample of Microscopic Interview Analysis. Martineau, Ithaca, NY.

[115]

Plaikner, D., 1970. Die Veränderung der menschlichen Stimme unter dem Einfluss psychischer Belastung. {Voice changes produced by psychic load}. Ph.D. Thesis, University of Innsbruck, Austria.

[116]

Rank, E., Pirker, H., 1998. Generating emotional speech with a concatenative synthesizer. In: Proc. ICSLP 98, Sydney, Vol. 3, pp. 671-674.

[117]

Roessler, R., Lester, J.W., 1976. Voice predicts affect during psychotherapy. J. Nerv. Mental Disease, 163, 166-176.

[118]

Roessler, R., Lester, J.W., 1979. Vocal pattern in anxiety. In: Fann, W.E., Pokorny, A.D., Koracau, I., Williams, R.L. (Eds.), Phenomenology and Treatment of Anxiety. Spectrum, New York.

[119]

Roseman, I., Smith, C., 2001. Appraisal theory: Overview, assumptions, varieties, controversies. In: Scherer, K.R., Schorr, A., Johnstone, T. (Eds.), Appraisal Processes in Emotion: Theory, Methods, Research. Oxford University Press, New York and Oxford, pp. 3-19.

[120]

Russell, J.A., 1994. Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychol. Bull. 115, 102-141.

[121]

Sangsue, J., Siegwart, H., Grosjean, M., Cosnier, J., Cornut, J., Scherer, K.R., 1997. Developpement d'un questionnaire d'évaluation subjective de la qualité de la voix et de la parole, QEV. Geneva Stud. Emot. Comm. 11 (1), Available from: >http://www.unige.ch/fapse/emotion/genstudies/gen-studies.html<.

[122]

Scherer, K.R., 1974. Voice quality analysis of American and German speakers. J. Psycholing. Res. 3, 281-290.

[123]

Scherer, K.R., 1978. Personality inference from voice quality: The loud voice of extroversion. Europ. J. Soc. Psychol. 8, 467-487.

[124]

Scherer, K.R., 1979. Non-linguistic indicators of emotion and psychopathology. In: Izard, C.E. (Ed.), Emotions in Personality and Psychopathology. Plenum Press, New York, pzzzp. 495-529.

[125]

Scherer, K.R., 1982a. Methods of research on vocal communication: Paradigms and parameters. In: Scherer, K.R., Ekman, P. (Eds.), Handbook of Methods in Nonverbal Behavior Research. Cambridge University Press, Cambridge, UK, pp. 136-198.

[126]

Scherer, K.R. (Ed.), 1982b. Vokale Kommunikation: Nonverbale Aspekte des Sprachverhaltens. {Vocal Communication: Nonverbal Aspects of Speech}. Beltz, Weinheim.

[127]

Scherer, K.R., 1984. On the nature and function of emotion: A component process approach. In: Scherer, K.R., Ekman, P. (Eds.), Approaches to Emotion. Erlbaum, Hillsdale, NJ, pp. 293-318.

[128]

Scherer, K.R., 1985. Vocal affect signalling: A comparative approach. In: Rosenblatt, J., Beer, C., Busnel, M., Slater, P.J.B. (Eds.), Advances in the Study of Behavior. Academic Press, New York, pp. 189-244.

[129]

Scherer, K.R., 1986. Vocal affect expression: A review and a model for future research. Psychol. Bull. 99 (2), 143-165.

[130]

Scherer, K.R., 1989. Vocal correlates of emotion. In: Wagner, H., Manstead, A. (Eds.), Handbook of Psychophysiology: Emotion and Social Behavior. Wiley, London, pp. 165-197.

[131]

Scherer, K.R., 1992a. What does facial expression express? In: Strongman, K. (Ed.), International Review of Studies on Emotion, Vol. 2. Wiley, Chichester, pp. 139-165.

[132]

Scherer, K.R., 1992b. On social representations of emotional experience: Stereotypes, prototypes, or archetypes. In: von Cranach, M., Doise, W., Mugny, G. (Eds.), Social Representations and the Social Bases of Knowledge. Huber, Bern, pp. 30-36.

[133]

Scherer, K.R., 1994a. Affect bursts. In: van Goozen, S.H.M., van de Poll, N.E., Sergeant, J.A. (Eds.), Emotions: Essays on Emotion Theory. Erlbaum, Hillsdale, NJ, pp. 161-196.

[134]

Scherer, K.R., 1994b. Toward a concept of "modal emotions". In: Ekman, P., Davidson, R.J. (Eds.), The Nature of Emotion: Fundamental Questions. Oxford University Press, New York/Oxford, pp. 25-31.

[135]

Scherer, K.R., 1999a. Appraisal theories. In: Dalgleish, T., Power, M. (Eds.), Handbook of Cognition and Emotion. Wiley, Chichester, pp. 637-663.

[136]

Scherer, K.R., 1999b. Universality of emotional expression. In: Levinson, D., Ponzetti, J., Jorgenson, P. (Eds.), Encyclopedia of Human Emotions. Macmillan, New York, pp. 669-674.

[137]

Scherer, K.R., 2000a. Psychological models of emotion. In: Borod, J. (Ed.), The Neuropsychology of Emotion. Oxford University Press, Oxford/New York, pp. 137-162.

[138]

Scherer, K.R., 2000b. Emotional expression: A royal road for the study of behavior control. In: Perrig, W., Grob, A. (Eds.), Control of Human Behavior, Mental Processes, and Consciousness. Lawrence Erlbaum Associates, Hillsdale, NJ, pp. 227-244.

[139]

Scherer, K.R., Oshinsky, J.S., 1977. Cue utilization in emotion attribution from auditory stimuli. Motiv. Emot. 1, 331-346.

[140]

Scherer, K.R., Koivumaki, J., Rosenthal, R., 1972a. Minimal cues in the vocal communication of affect: Judging emotions from content-masked speech. J. Psycholinguist. Res. 1, 269-285.

[141]

Scherer, K.R., Rosenthal, R., Koivumaki, J., 1972b. Mediating interpersonal expectancies via vocal cues: Different speech intensity as a means of social influence. Europ. J. Soc. Psychol. 2, 163-176.

[142]

Scherer, K.R., London, H., Wolf, J., 1973. The voice of confidence: Paralinguistic cues and audience evaluation. J. Res. Person. 7, 31-44.

[143]

Scherer, K.R., Ladd, D.R., Silverman, K.E.A., 1984. Vocal cues to speaker affect: Testing two models. J. Acoust. Soc. Amer. 76, 1346-1356.

[144]

Scherer, K.R., Feldstein, S., Bond, R.N., Rosenthal, R., 1985. Vocal cues to deception: A comparative channel approach. J. Psycholinguist. Res. 14, 409-425.

[145]

Scherer, K.R., Banse, R., Wallbott, H.G., Goldbeck, T., 1991. Vocal cues in emotion encoding and decoding. Motiv. Emot. 15, 123-148.

[146]

Scherer, K.R., Johnstone, T., Sangsue, J., 1998. L'état émotionnel du locuteur: facteur négligé mais non négligeable pour la technologie de la parole. Actes des XXIIèmes Journées d'Etudes sur la Parole, Martigny.

[147]

Scherer, K.R., Johnstone, T., Klasmeyer, G, Bänziger, T., 2000. Can automatic speaker verification be improved by training the algorithms on emotional speech? In: Proc. ICSLP2000, Beijing, China.

[148]

Scherer, K.R., Banse, R., Wallbott, H.G., 2001a. Emotion inferences from vocal expression correlate across languages and cultures. J. Cross-Cult. Psychol. 32 (1), 76-92.

[149]

Scherer, K.R., Schorr, A., Johnstone, T. (Eds.), 2001b. Appraisal Processes in Emotion: Theory, Methods, Research. Oxford University Press, New York and Oxford.

[150]

Scherer, K.R., Johnstone, T., Klasmeyer, G., in press. Vocal expression of emotion. In: Davidson, R.J., Goldsmith, H., Scherer, K.R. (Eds.), Handbook of the affective sciences, Oxford University Press, New York and Oxford.

[151]

Scherer, T., 2000. Stimme, Emotion land Psyche: Untersuchungen zur emotionalen Qualität der Stimme {Voice, emotion, and psyche: Studies on the emotional quality of voice}. Ph.D. Thesis, University of Marburg, Germany.

[152]

Scripture, E.W., 1921. A study of emotions by speech transcription. Vox 31, 179-183.

[153]

Sedlacek, K., Sychra, A., 1963. Die Melodie als Faktor des emotionellen Ausdrucks {Speech melody as a factor of emotional expression}. Folia Phonatrica 15, 89-98.

[154]

Shipley-Brown, F., Dingwall, W., Berlin, C., Yeni-Komshian, G., Gordon-Salant, S., 1988. Hemispheric processing of affective and linguistic intonation contours in normal subjects. Brain Lang. 33, 16-26.

[155]

Siegman, A.W., 1987a. The telltale voice: Nonverbal messages of verbal communication. In: Siegman, A.W., Feldstein, S. (Eds.), Nonverbal Behavior and Communication, second ed. Erlbaum, Hillsdale, NJ, pp. 351-433.

[156]

Siegman, A.W., 1987b. The pacing of speech in depression. In: Maser, J.D. (Ed.), Depression and Expressive Behavior. Erlbaum, Hillsdale, NJ, pp. 83-102.

[157]

Simonov, P.V., Frolov, M.V., 1973. Utilization of human voice for estimation of man's emotional stress and state of attention. Aerospace Md. 44, 256-258.

[158]

Skinner, E.R., 1935. A calibrated recording and analysis of the pitch, force and quality of vocal tones expressing happiness and sadness. Speech Monogr. 2, 81-137.

[159]

Sobin, C., Alpert, M., 1999. Emotion in speech: The acoustic attributes of fear, anger, sadness, and joy. J. Psycholinguist. Res. 28 (4), 347-365.

[160]

Starkweather, J.A., 1956. Content-free speech as a source of information about the speaker. J. Pers. Soc. Psychol. 35, 345-350.

[161]

Steinhauer, K., Alter, K., Friederici, A., 1999. Brain potentials indicate immediate use of prosodic cues in natural speech processing. Nature Neurosci. 2, 191-196.

[162]

Stemmler, G., Heldmann, M., Pauls, C.A., Scherer, T., 2001. Constraints for emotion specificity in fear and anger: The context counts. Psychophysiology 38 (2), 275-291.

[163]

Sulc, J., 1977. To the problem of emotional changes in the human voice. Activitas Nervosa Superior 19, 215-216.

[164]

Tischer, B., 1993. Die vokale Kommunikation von Gefühlen {The Vocal Communication of Emotions}. Psychologie-Verlags-Union, Weinheim.

[165]

Titze, I., 1992. Acoustic interpretation of the voice range profile (Phonetogram). J. Speech Hear. Res. 35, 21-34.

[166]

Tolkmitt, F.J., Scherer, K.R., 1986. Effects of experimentally induced stress on vocal parameters. J. Exp. Psychol.: Hum. Percept. Perform. 12, 302-313.

[167]

Tolkmitt, F., Bergmann, G., Goldbeck, Th., Scherer, K.R., 1988. Experimental studies on vocal communication. In: Scherer, K.R. (Ed.), Facets of Emotion: Recent Research. Erlbaum, Hillsdale, NJ, pp. 119-138.

[168]

Tomkins, S.S., 1962. Affect, Imagery, Consciousness. The Positive Affects, Vol. 1. Springer, New York.

[169]

Trager, G.L., 1958. Paralanguage: A first approximation. Stud. Linguist. 13, 1-12.

[170]

Tusing, K.J., Dillard, J.P., 2000. The sounds of dominance: Vocal precursors of perceived dominance during interpersonal influence. Hum. Comm. Res. 26 (1), 148-171.

[171]

Utsuki, N., Okamura, N., 1976. Relationship between emotional state and fundamental frequency of speech. Reports of the Aeromedical Laboratory, Japan Air Self-Defense Force, 16, 179-188.

[172]

van Bezooijen, R., 1984. The Characteristics and Recognizability of Vocal Expression of Emotions. Foris, Dordrecht, The Netherlands.

[173]

van Bezooijen, R., Otto, S., Heenan, T.A., 1983. Recognition of vocal expressions of emotions: A three-nation study to identify universal characteristics. J. Cross-Cult. Psychol. 14, 387-406.

[174]

Wagner, H.L., 1993. On measuring performance in category judgment studies on nonverbal behavior. J. Nonverb. Behav. 17 (1), 3-28.

[175]

Wallbott, H.G., Scherer, K.R., 1986. Cues and channels in emotion recognition. J. Pers. Soc. Psychol. 51, 690-699.

[176]

Wehrle, T., Kaiser, S., Schmidt, S., Scherer, K.R, 2000. Studying dynamic models of facial expression of emotion using synthetic animated faces. J. Pers. Soc. Psychol. 78 (1), 105-119.

[177]

Westermann, R., Spies, K., Stahl, G., Hesse, F.W., 1996. Relative effectiveness and validity of mood induction procedures: A meta-analysis. Europ. J. Soc. Psychol. 26 (4), 557-580.

[178]

Whiteside, S.P., 1999. Acoustic characteristics of vocal emotions simulated by actors. Perc. Mot. Skills 89 (3, Part 2), 1195-1208.

[179]

Williams, C.E., Stevens, K.N., 1969. On determinating the emotional state of pilots during flight: An exploratory study. Aerospace Md. 40, 1369-1372.

[180]

Williams, C.E., Stevens, K.N., 1972. Emotions and speech: Some acoustical correlates. J. Acoust. Soc. Amer. 52, 1238-1250.

[181]

Wundt, W., 1874/1905. Grundzüge der physiologischen Psychologic, {Fundamentals of physiological psychology, orig. pub. 1874} fifth ed. Engelmann, Leipzig.

[182]

Zuberbier, E., 1957. Zur Schreib- und Sprechmotorik der Depressiven. {On the motor aspect of speaking and writing in depressives}. Z. Psychother. Medizin. Psychol. 7, 239-249.

[183]

Zwirner, E., 1930. Beitrag zur Sprache des Depressiven. {Contribution on the speech of depressives}. Phonometrie III. Spezielle Anwendungen I. Karger, Basel, pp. 171-187.

[184]

Zwicker, E., 1982. Psychoacoustics. Springer, New York.

Cited By

Meler AChua TNgo CKumar RLauw HKa-Wei Lee R(2024)In The Beginning, Let There Be The Word: Challenges and Insights in Applying Sentiment Analysis to Social ResearchCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651264(1214-1217)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3651264
Beccaro WArjona Ramírez MLiaw WGuimarães H(2024)Analysis of Oral Exams With Speaker Diarization and Speech Emotion Recognition: A Case StudyIEEE Transactions on Education10.1109/TE.2023.332115567:1(74-86)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1109/TE.2023.3321155
Iben Nasr LMasmoudi AHadrich Belguith L(2024)Survey on Arabic speech emotion recognitionInternational Journal of Speech Technology10.1007/s10772-024-10088-727:1(53-68)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s10772-024-10088-7
Show More Cited By

Index Terms

Vocal communication of emotion: a review of research paradigms

Recommendations

Vocal markers of emotion: Comparing induction and acting elicitation

The search for vocal markers of emotion has been hampered by the difficulty of obtaining access to speech samples that represent authentic expressions of the speaker's felt emotions. The recent trend to privilege real-life, naturalistic speech tokens, ...
Speech Emotion Perception by Human and Machine
Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction

The human speech contains and reflects information about the emotional state of the speaker. The importance of research of emotions is increasing in telematics, information technologies and even in health services. The research of the mean acoustical ...
Automatic refinement of an expressive speech corpus assembling subjective perception and automatic classification

This paper presents an automatic system able to enhance expressiveness in speech corpora recorded from acted or stimulated speech. The system is trained with the results of a subjective evaluation carried out on a reduced set of the original corpus. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Speech Communication

Speech Communication Volume 40, Issue 1-2

Special issue on speech and emotion

April 2003

249 pages

ISSN:0167-6393

Issue’s Table of Contents

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 April 2003

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

140
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Meler AChua TNgo CKumar RLauw HKa-Wei Lee R(2024)In The Beginning, Let There Be The Word: Challenges and Insights in Applying Sentiment Analysis to Social ResearchCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651264(1214-1217)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3651264
Beccaro WArjona Ramírez MLiaw WGuimarães H(2024)Analysis of Oral Exams With Speaker Diarization and Speech Emotion Recognition: A Case StudyIEEE Transactions on Education10.1109/TE.2023.332115567:1(74-86)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1109/TE.2023.3321155
Iben Nasr LMasmoudi AHadrich Belguith L(2024)Survey on Arabic speech emotion recognitionInternational Journal of Speech Technology10.1007/s10772-024-10088-727:1(53-68)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s10772-024-10088-7
Bhangale KKothandaraman M(2024)Speech Emotion Recognition Using Generative Adversarial Network and Deep Convolutional Neural NetworkCircuits, Systems, and Signal Processing10.1007/s00034-023-02562-543:4(2341-2384)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s00034-023-02562-5
Koçak TDibek BPolat EKafesçioğlu NDemiroğlu C(2023)Automatic detection of attachment style in married couples through conversation analysisEURASIP Journal on Audio, Speech, and Music Processing10.1186/s13636-023-00291-w2023:1Online publication date: 31-May-2023
https://dl.acm.org/doi/10.1186/s13636-023-00291-w
Ferrari AHuichapa TSpoletini PNovielli NFucci DGirardi D(2023)Using Voice and Biofeedback to Predict User Engagement during Product Feedback InterviewsACM Transactions on Software Engineering and Methodology10.1145/363571233:4(1-36)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3635712
Zeng KLiu G(2023)Emotion recognition based on millimeter wave radarProceedings of the 2023 3rd International Conference on Bioinformatics and Intelligent Computing10.1145/3592686.3592728(232-236)Online publication date: 10-Feb-2023
https://dl.acm.org/doi/10.1145/3592686.3592728
Seaborn KRogers KNam SKojima M(2023)Kawaii Game Vocalics: A Preliminary ModelCompanion Proceedings of the Annual Symposium on Computer-Human Interaction in Play10.1145/3573382.3616099(202-208)Online publication date: 6-Oct-2023
https://dl.acm.org/doi/10.1145/3573382.3616099
Zhou KSisman BRana RSchuller BLi H(2023)Emotion Intensity and its Control for Emotional Voice ConversionIEEE Transactions on Affective Computing10.1109/TAFFC.2022.317557814:1(31-48)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TAFFC.2022.3175578
Aldeneh ZProvost E(2023)You're Not You When You're Angry: Robust Emotion Features Emerge by Recognizing SpeakersIEEE Transactions on Affective Computing10.1109/TAFFC.2021.308605014:2(1351-1362)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TAFFC.2021.3086050
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents