The Audio Uncanny Valley: Sound, Fear and The Horror Game.
The Audio Uncanny Valley: Sound, Fear and The Horror Game.
The Audio Uncanny Valley: Sound, Fear and The Horror Game.
2009
This Conference Paper is brought to you for free and open access by the School of Games Computing and Creative Technologies at UBIR: University
of Bolton Institutional Repository. It has been accepted for inclusion in Games Computing and Creative Technologies: Conference Papers (Peer-
Reviewed) by an authorized administrator of UBIR: University of Bolton Institutional Repository. For more information, please contact
ubir@bolton.ac.uk.
The audio Uncanny Valley: Sound, fear and the horror game
Mark Grimshaw
School of Games Computing & Creative Technologies
University of Bolton
mng1@bolton.ac.uk
Abstract. The 1970 proposition that there is an Uncanny Valley which man-made characters inhabit as their human-likeness (both
appearance and movement) increases has been a growing topic of debate in the fields of robotics, animation and computer games
particularly since the turn of the century. However, what the theory and subsequent related writings do not account for is the role of
sound in creating perceptions of uncanniness and fear, a particularly useful attribute in computer game genres such as survival horror.
This paper has a dual purpose: to explore diverse writings on the uncanny as they relate to sound and to prepare the groundwork for
future work investigating the possible relationship between sound and the Uncanny Valley.
The paper comprises, in large part, a survey of selected works on the uncanny and the Uncanny Valley from a variety of disciplines.
It emphasizes the link between uncanniness and negative emotions, such as fear and apprehension, and discusses the genesis of the
term uncanny in early psychoanalytical writings, relating this to more modern theories on human emotion. Writings on the uncanny,
or related emotional states, from psychoacoustics, textiles research, films and computer games are assessed as to their validity and
potential application to the fostering of an aural climate of fear in computer games and, where such writings do not explicitly deal
with sound, attempts are made to apply the ideas contained within to sound as it exists within computer games. In dealing with the
theory of the Uncanny Valley, the paper points out the theorys focus on appearance and movement to the exclusion of sound and
suggests that there is an uncanny in sound that might, in future, be used to modify the Uncanny Valley theory. Throughout, there is
the suggestion that the uncanny (and any future theory of an audio or audiovisual Uncanny Valley) can be harnassed to the design of
horror computer games.
Ultimately, it is hoped, such work will be of use to computer game sound designers who wish to create a greater perception of fear
and apprehension through the canny use of uncanny sound. Some of the design tips presented at the end of the discussion are already
used instinctively by sound designers across a range of media, including computer games, whereas others are less obvious in their
origin and affect. Recently published empirical data is provided to strengthen the case for the latter. In some cases, the design tips
must await the coming of procedural audio to computer games.
-1-
The audio Uncanny Valley: Sound, fear and the horror game
design. What almost all studies dealing with the Uncanny work suggests this physiological arousal and associated emotion
Valley share, though, is a concentration on the image. Whether leads to player engagement and immersion in the 3-dimensional
still or moving, such writings invariably deal with the environments of first-person shooter games.[11] For a general
appearance, and or motion, of the human-like character; there is overview of threat and associated emotions in computer games,
a visual bias to the study of the uncanny. see Perron.[12]
This paper asks: Is it possible to apply the Uncanny Valley Interestingly, most research and writing on emotion and sound
theory to the emotions aroused by sound and thus to include in virtual reality and computer games deals with the negative
sound as a factor in the Uncanny Valley? Although it does not emotions terror, fear and apprehension (using Plutchiks
attempt to fully answer that question (such an answer must await terminology) and their semantic variants (using others
more empirical research), the paper does seek to identify terminologies); this paper continues that tradition. Outside of
attributes of sound that contribute to the uncanny with the 3D-worlds, there is a wider survey of the emotions associated
assumption that, should this identification be achieved, it will with sound but, nevertheless, research is patchy. Owren and
then be possible to codify aspects of computer game sound Bachorowski, studying primate vocalizations, suggest that
design that elicit or block negative emotions such as fear and its primates use some sounds not to convey representational
variants as the game genre requires. (In some future survival information to a listener but to directly or indirectly affect and
horror game, the author imagines a less fear more fear sound arouse particular emotional states within the listener at a
FX slider in the game set-up interface, or a continual fundamental cognitive level.[13] To do this, they manipulate
physiological monitoring of the player during gameplay that, the parameters of the sound directly and the authors hypothesize
through real-time sound synthesis and audio processing, keeps that this is how some forms of human laughter work and calls
the player on a particular emotional roller-coaster.) Thus, where for attention from babies, for example. Edworthy et al.
many visual modellers attempt to cross the Uncanny Valley, conducted an experiment on the perception of urgency as
viewing it as an obstacle to overcome, this paper agrees with various parameters of sound and harmonic patterns of audio
authors such as MacDorman and Hoeger et al. that the trough of alarms were altered.[14] Their observations include: sounds
the Uncanny Valley is, in some game genres, to be welcomed. with a fast onset and offset of 20msecs. or less are perceived as
Additionally, though, it also further supports earlier suggestions more urgent than sounds with a longer onset than offset which
that, despite claims to the contrary, the obstacle that is the are themselves perceived as more urgent than sounds with a
Uncanny Valley cannot, in fact, be overcome.[7] shorter onset than offset; and the more random the harmonic
pattern of the sounds, the more urgent the perception of those
2. Sound, the Uncanny and the Valley sounds. The interesting result that sounds with a longer onset
than offset are perceived as more pressing than the reverse is
In addition to psychoanalytical work, popular literature also explained by the authors with the suggestion that the former
contains descriptions of the uncanny and, in some cases, class of sounds has the characteristic amplitude envelope of
descriptions of uncanny sound. Some quotations from The approaching sound sources whereas the latter class of sounds
Beasts of Tarzan demonstrate this: From the lips of the ape- has more of the character of receding sound sources. Although
man came a weird, uncanny sound [] strange, uncanny notes no explanation is offered for the effect of melodic randomness,
that the girl could not ascribe to any particular night prowler it may be that the perception of urgency is related to the
more terrible because of their mystery [] he was afraid of the uncertainty (in the West at least) arising from less tonally-
jungle; uncanny noises that were indeed frightful came forth centred music and the consequent difficulty of processing the
from its recesses.[8] Such uncanny sound is typically tones and identifying a melodic pattern. This association of
associated with negative emotions such as Plutchiks basic uncertainty (a lack of fluency in the processing of sound) with
emotion of terror and its less intense outgrowths fear and urgency and apprehension is developed further by authors
apprehension.[9] (There are several theories of emotion but discussing fear and sound in horror computer games, a
Plutchiks is an interesting one to use in this context because of discussion detailed below.
its psychoevolutionary basis and the claim that emotions aid in
the survival of the organism in the environment interesting not Alarms presage previously unseen threats and threats, according
merely because of similar claims made by writers on the to Plutchik are the stimulus events leading to feelings of terror,
uncanny, as illustrated below, but also because the exemplars fear and apprehension. Threat sounds are used to great effect in
used in many articles on fear in computer games tend to be first- the computer game Left 4 Dead particularly where the actions of
person shooters or horror games where the player operates in a a player alert the swarm of zombies.[15] In this case, a wolf-
hostile environment.) These emotions, according to Plutchek, like howl heralds the swarms attack and it is the predatory
derive from the threat stimulus event which itself, as the Tarzan denotation and lycanthropic connotation that is designed to send
stories show, is often associated with the unknown, the a chill up the spine. Paralleling this, Halpern et al., analyzing
unfamiliar, the darkness and the night. Such emotions and the nerve-jarring sound of fingernails scraping across a
scenarios are the basic ingredients of the horror genre in blackboard, suggest aversion to such a sound either might be
literature, film and computer games. because of its similarity to predator sounds or it might be (an
implicit suggestion in the paper) a vestigial response to proto-
There has been surprisingly little work on the association of human warning calls due to its similarity to the macaque
emotions with sound (that is, sound FX as opposed to music or monkeys warning screech (this is acknowledged as pure
speech). In the area of sound design for the horror genre of speculation in the paper and one of the authors has since
cinema (which the parallel genre in computer games closely disowned the conjecture).[16]
follows), and in the absence of any comprehensive and well-
founded methodology, such design proceeds on the basis of Cho et al. conducted an investigation into the parameters of
experience, clich or trial and error. It is usually no less textile sounds (that is, the rustling sounds of a variety of fabrics)
effective for this. Some research in the area of computer games that, it was assumed, were responsible for negative feelings
deals broadly with sound as a means to increase physiological about the textile.[17] Their results suggest that increasing
and, therefore, emotional arousal in the player [10] and other loudness and sharpness of timbre (the lowest sharpness acum
-2-
The audio Uncanny Valley: Sound, fear and the horror game
value for any of the fabrics was 2.38, equivalent to a band of Ekman and Kajastila, rather than investigating the parameters of
high frequency energy centred on approximately 5kHz) produce sound contributing to the feeling of being scared, conducted a
physiological reactions associated with negative emotions. In small-scale, subjective study to determine the perceptual effect
other words, loud, sharp sounds are not generally pleasant. of localization on sounds already pre-judged to be scary.[24]
Compare this to Halpern et al. who, when discussing parameters Importantly, sounds were played back to participants in the
of sound leading to unpleasant feelings, accept loudness as a absence of any contextualizing image. Sounds comprised those
factor but discount high frequencies instead pointing to low-mid made by large predators, which motivates the importance of
frequencies as the cause (a high-pass filter, attenuating localizing the threat. The results support the authors
frequencies below 2kHz, decreased the subjective hypothesis that the scariness of a (scary) sound is causally
unpleasantness rating). This suggests that negative affect related to how well it affords localizing a potentially harmful
responses may well be provoked by the presence of certain source and this, as the authors suggest, probably has its root
frequencies but there are probably additional factors involved as cause in the evolutionary link between fear and survival. The
well. Context, connotation or something more subconscious are inability to localize a sound is generalized to a lack of ease or
suggested by the reference to predatory sounds or macaque fluency in processing sound; thus, according to the authors,
monkeys and their warning calls; more unpleasant physical [t]he less information available, the more threatening the
associations might be suggested by the sound of crackling static situation should be. This seems a surprisingly broad
electricity close to the skin. That there might be instinctive assessment. It is unlikely that de-localizing all types of sound
negative emotional responses to unexpected loud sounds in will promote fear in the listener. Low-frequency sine waves,
general is supported by the Moro Reflex found through a limited and similar natural sounds such as whale song, are difficult to
period in pre- and post-natal babies a reflex response to sound localize yet are not necessarily threatening because of that
(or falling) which, it has been suggested, is founded upon the recordings of whale song are often used for relaxation purposes.
one unlearned and innate human fear. The same could be said for the general hum of traffic outside my
office or the 50-60Hz mains hum in a house. In the case of the
Moncrieff et al., in a study assessing the possibility of authors study, a predatory sound (already judged to be scary) is
automatically classifying horror and thriller films by their audio made more scary by removing the ability to localize it
content, analyzed the frequency of sound energy and affect generalizing this to all sounds is perhaps a step too far.
events (these latter are an intentional emotional inflexion of the
events visually portrayed).[18] Affect events are indexical, A sustaining thread in Ekman and Kajastilas argument is the
there being a high level of semantic association between the impact of uncertainty on the perceived level of scariness of a
sound energy and affect events where detected sound energy sound. The concept of uncertainty also appears in Kromands
patterns correspond to an affect event (such as an alarm or sense study of sound in the survival horror game genre; specifically,
of apprehension), they affirm the affect event and so the a framework of uncertainty that constantly holds the player
detection of many such patterns, according to the authors, can be between knowledge and ignorance.[25] According to
used to classify film according to either horror or thriller genre. Kromand, the soundscapes of survival horror computer games
Given the typical intensity of the sound energy associated with purposefully mislead by making it unclear whether the sounds
the horror-type affect event, it would be interesting to see if the heard derive from within the game diegesis or without: this
authors classification method (tested on Western, Hollywood- collapse of the barrier between the diegetic and non-diegetic
style cinema) works for the cinema of other cultures. As Mala soundscape is a strategy to build a horror atmosphere. The
has stated: Asian horror is often rooted in vision.[19] This removal of causality, an understanding of and awareness of the
contention is supported by Ringu [20] director Nakata: Other source of the sound, and its unsettling result is something that
people tend to use different sounds altogether to express horror, has been described by Chion in the context of cinema.[26] For
but I can increase the perception of it to the maximum by sounds having no visible source on screen, Chion appropriates
utilizing a very quiet sound.[21] The manifestation of threat the electro-acoustic term acousmatic: A sound or voice that
stimulus events for fear and apprehension may well contain remains acousmatic creates a mystery of the nature of its sound
features that do not function uniformly across the human race source, its properties and its powers. In film, the decision to
but, instead, are culturally specific in their threat and meaning. unveil the mystery or not belongs solely to the director; in
computer games, as Stockburger makes clear, such unveiling is
In a comparison of the uncanny in Ringu and the American more dynamic and sound sources may be unmasked by scripted
remake The Ring [22], Ball provides a short section on aural events designed into the game (equivalent to the decisions of the
uncanniness which, in Ringu, is exemplified for the author by cinematic director) or by the kinaesthetic intervention of the
the audio processing applied to the familiar sound of a ringing player.[27] In the cases that Kromand identifies, the sounds are
digital telephone.[23] Here, the uncanny is created through the destined to remain acousmatic and thus they are, as Parker and
process of making the familiar strange (the ringing heard is a Heerama state, instinctively threatening.[28]
combination of multiple telephone rings slightly processed to
match the films theme of water). According to Ball, it is this Brenton et al. review a number of theories on presence, realism
defamiliarization of a mundane sound the distortion of a sound and the Uncanny Valley from which they derive five hypotheses
that yet retains its broadly recognizable original form and on the relationship of the Uncanny Valley to presence in virtual
purpose that leads to the uncanny. This is, perhaps, too broad worlds.[29] The Gestalt-derived theory of presence suggests the
and all-encompassing an explanation. There are many varieties brain chooses one of a set of hypotheses relating either to what
of telephone ring, each approximating the classic and iconic we perceive of a virtual world or to where we physically are.
telephone bell (both analogue and digital) but altering it in some Engaging in the arcana of a virtual world yet still aware of the
way; there is no suggestion that any of these sounds are uncanny mundanity of reality (the weighty effect of gravity or the
despite being defamiliarizing distortions of the original, familiar physical environment around the computer monitor, for
ring. Instead, the context of the ring plays an important role; not example), the brain will pick one or the other hypothesis. The
only is it framed within the horror film genre, it is signalled hypothesis chosen dictates where we feel present; in virtuality or
early as an apprehensive aural cue, a threat stimulus event, reality. A switch from virtual hypothesis to reality hypothesis is
through the films plot and narrative. a break in presence. As a conjecture, Brenton et al. theorize
-3-
The audio Uncanny Valley: Sound, fear and the horror game
-4-
The audio Uncanny Valley: Sound, fear and the horror game
combination. Above all, though, the sound designer should be uncanny sound or whether that sound follows its own uncanny
aware of the omnipotence of the framing context: logic.
-5-
The audio Uncanny Valley: Sound, fear and the horror game
[19] Mala, E., The sound of horror, Newsweek, (2008) [29] Brenton, H., Gillies, M., Ballin, D., & Chatting DJ., The
[20] Nakata, H., Ringu, (1998) uncanny valley: Does it exist? Proceedings of the Human-
[21] Naito, T., Interview with Hideo Nakata, Specter Director, Animated Characters Interaction, HCI 2005: The Bigger Picture,
Kateigaho, (2005) (2008)
[22] Verbinski, G., The Ring, (2002) [30] Laurel, B., Computers as theatre. New York, Addison-
[23] Ball, S. K. V. M., The uncanny in Japanese and American Wesley, (1993)
horror film: Hideo Nakata's Ringu and Gore Verbinski's Ring, [31] Tinwell, A. & Grimshaw, M., Survival horror games - An
Unpublished master's thesis, North Carolina State University, uncanny modality, Thinking After Dark, Montreal, (2009)
Raleigh, NC, (2006) [32] Acoyauh (trans.), The creation,
[24] Ekman, I., & Kajastila, R., Localisation cues affect http://www.geocities.com/Athens/Academy/3088/pv-
emotional judgements: Results from a user study on scary creation.html [accessed 2 May 2009]
sound, AES 35th International Conference, London, (2009) [33] Grimshaw, M., The acoustic ecology of the first-person
[25] Kromand, D., Sound and the diegesis in survival-horror shooter: The player experience of sound in the first-person
games, Audio Mostly 2008, Pite, Sweden, (2008) shooter computer game, Saarbrcken, VDM Verlag, (2008)
[26] Chion, M., Audio-vision: Sound on screen (C. Gorbman, [34] Pollick, F. E., In search of the uncanny valley, In K.
Trans.), New York, Columbia University Press, (1994) Grammer, & A. Juett (Eds.), Analog communication: Evolution,
[27] Stockburger, A., The rendered arena: Modalities of space brain mechanisms, dynamics, simulation, Cambridge, MA: MIT
in video and computer games, Unpublished PhD thesis, Press, The Vienna Series in Theoretical Biology, (in press)
University of the Arts, London, (2006) [35] Minato, T., Shimda, M., Ishiguro, H., & Itakura, S.,
[28] Parker, J. R., & Heerama, J., Audio interaction in computer Development of an android robot for studying human-robot
mediated games, International Journal of Computer Games interaction, In Orchard, R., Yang, C and Ali, M., (Eds),
Technology, 2008, Innovations in Applied Artificial Intelligence, Volume 3029,
http://www.hindawi.com/GetArticle.aspx?doi=10.1155/2008/17 424-434 Berlin: Springer (2004)
8923 [accessed: 27 December 2007], (2008) [36] Tinwell, A. & Grimshaw, M., Bridging the uncanny: An
impossible traverse?, Mindtrek, Tampere, (2009)
-6-