Aural Limbo: Space As A Sonic Interactive Interface
Aural Limbo: Space As A Sonic Interactive Interface
Aural Limbo: Space As A Sonic Interactive Interface
Mateo Zlatar
2003
1
Acknowledgements
2
Table of Contents
Preliminaries
Title
Abstract
Acknowledgements
Table of Contents
Chapter 1. Introduction
1.1. Motivations
1.2. Overview of the Thesis
1.3. Contribution of this Thesis
Chapter 2. Background
2.1 Introduction
2.2 Spaces for sound
2.3 The Aural Perception of Spaces
2.3.1 Space as Sound Points
2.3.2. Sound as Feedback of Space
2.3.3. Spaces with aural directional
messages
2.3.4. Dislocated Perception
2.3.5. Sound as Inner Space
2.4 Sound as Space
2.5 Patterns in Noise
Chapter 3. Methodology
3.1. Introduction
3.2. Preliminary Experiments
3.2.1. Wearable Synth (Fall 2002)
3.2.2. Trigger Space (Spring 2003)
3.2.3. Sonic Arena (Fall 2002)
3.3. Thesis Prototypes
3.3.1. System Overview
3.3.2. Traffic Report (Spring 2003)
3.3.3. Spatial Scrub (Spring 2003)
3
3.3.4. Accumulation
3.3.5. Panning Sound
3.4. Summary of Implementations
3.5. Thesis Installation
3.5.1 Revealing Patterns in Noise.
3.5.2. The Context of the Public Space.
3.5.3. The users experience
Chapter 5. Conclusion
5.1. Conclusions
5.2. Future Directions
Chapter 6. Bibliography
Appendices
Colophon
4
Chapter 1. Introduction
1.1. Motivations
6
is far more suggestive than the final result.
7
is exposed through documentation on the design and
programming fronts, as design sketches, flowcharts and
mapping diagrams, explaining functionality, interaction
goals and expectations, that finally are summarized and
evaluated.
Chapter 4 is dedicated to discussion and analysis of the
work outcome, evaluating at all its development stages.
Chapter 5, Conclusions, summarizes the Thesis
contributions, pointing unfulfilled desires and future
directions.
Finally, I provide two technical appendixes, one showing
visual programming diagrams and other for supplementary
sketches.
8
Art into the realm of computer multimedia.
9
Chapter 2. Background
2.1 Introduction
10
2.2 Spaces for sound
11
creating percussive patterns by the reverberation of its
sounds. A friend of mine, who brought me pictures of the
building, described the experience as an unexpected gift,
because she just walk through the space without knowing
what was going to happen. As she discovered these
rhythmical patterns, she also noticed how other people
stayed in the space to play with those sounds, as if the
building were a large-scale sound instrument. A building
that is a sonic instrument.
14
“The Soundcube is an instrument for producing space with
sound. It has a grid of loudspeakers on each of its six walls.
It is visually speaking “neutral”, i.e., without any specific
spatial message. The sound is programmed to travel from
loudspeaker to loudspeaker. The dimensions of the cube
depend on the particular situation. An infinite number of
spaces or spatial sensations can be created. The Soundcube
is a laboratory for studies in the definition and character
of space and for investigation into the relationship between
motions of sound and their audio-physical experience. At
the same time it is a place for demonstrations to the public.”
(Leitner, 18.)
Soundcube and Spatial Grid were never built, although
Leitner designed the sound movement specifications in
custom scores, and punch-card programming.
These instruments served as hypothetical situations were
to develop his theoretical framework. The hypothesis he
was trying to depict, was “Rhythm as Space” referring to
accentuation of individual points in space by the pulsing
of individual speaker units across the six walls. A sequence
of pulses describes direction, creating the illusion of lines
and circles of sound traveling in space, in which intensity,
tempo and duration became critically significant.
18
dynamic sound layer, which is crossed by the body in motion
in any given direction, perceiving the dimension of the
gates physically (fig.12) Leitner made a sketch of how he
sees a sequence of gates implemented in a public space
(passageway of hallway) A directional message in this case
will have a polarized meaning; one encouraging the body
movement if walks in the same direction, and the other
moving against your direction which may be analogized to
Fig.12 Sound Gate, Leitner 1971.
Non-directional program. the experience of walking against the “wind direction”,
which is not wrong or right, but simply adds this sense of
impulse.
20
breathing falls in synch with them, which can be thought
as an expansion of the body in space through sound, similar
as we may found in the ancestral practice of mantra praying,
a form of meditation where sound leads the ritual as its
“score”.
Sound tends to create a feeling of awareness of an inner
space, which goes beyond the body space. Leitner was
aware of this as he recalls in an interview during the eighties:
“Hearing experiences not only enable us to have a special
spatial experience but also an internal space” (Bernhard
Leitner interviewed by Wolfgang Pehnt during Documenta
1984, Cologne.)
I believe that sound has a strong influence at the spiritual
level, independent of individual religious believes or ritual
practices. David Rokeby, multimedia artist creator of Very
Nervous System, a video tracking software and performance
instrument, comments about this issue in the context of
his interactive sound installation “VNS” (1986-1991).
“The diffuse, parallel nature of the sound interaction and
Fig.17 David Rokeby performing with his
Very Nervous System. the intensity of the feedback loop can produce a state that
is almost shamanistic. The self expands, and loses itself to
fill the space and by implication, the world.” (Rokeby)
23
Fontana transmits sounds from one location to another
creating a re-presentation of a distant space, which in the
change of context suffers of an intriguing permutation,
which evokes another physical space. An example of this
is "Sound Island" a sound intervention made by Fontana
at Arc de Triomphe, Paris in commemoration of the 50th
anniversary of the D-day. In this work loudspeakers were
placed in the four façades of the monument, which
transmitted live sounds from the Normandy Coast,
transforming the visual and aural experience of the constant
traffic around the Arc. The harmonic complexity of the
natural sounds of the ocean and crushing waves has the
psycho-acoustic ability to mask other sounds, directing our
attention to them over the overwhelming noise of traffic.
It is interesting to notice how an intervention like this, with
no visual or physical alteration, has the power to completely
transform the notion of a space.
25
object he found, which can be reflected in his “prepared
pianos”. From them it can be said that Cage wanted to
incorporate musical meaning to commonly unsemanticized
sounds, which in Bill Fontana’s words, is the ultimate
contribution of Sound Art.
“The semantic ambiguity of sound will change when society
develops a capacity to perceive patterns or qualities that
are recognizable as part of a context of meaning, such as
the sound vocabularies of contemporary music and acoustic
art […] The task of acoustic art and acoustic design is to
fundamentally challenge all of the old historical definitions
of noise and the resulting preconceptions that most people
have about the sounds they live with”. (Fontana, 3)
For Cage and Fontana the problem of noise may be
understood as a lack of listening awareness; in the case of
Fontana by creating works that uses the natural environment
as living source of musical information. He assumes that
“at any given moment there will be something meaningful
to hear” and “that music - in the sense of meaningful sound
patterns - is a natural process that is going on constantly.”
26
Chapter 3. Methodology
3.1. Introduction
Timer Chip
setup
3.2.2. Trigger Space (Spring 2003)
An experiment similar to Wearable Synth was done later in
time, using video input instead of sensors attached to the
body.
Fig.24 Sensors positions and chip
location in Wearable Synth.
The video-input is processed in the computer using Cyclops,
28
and MAX/MSP as the computer vision and sound processing
software respectively. Cyclops allows for the specification
of a grid of custom rows and columns over the image,
outputting integer numbers according to the occlusion of
any of the cells in the grid.
I assigned a different sound to each cell. The visual field of
the camera is converted in a kind of "minefield", in that
way the body can trigger those sounds by just crossing a
cell.
33
From the video image I analyzed the amount of overall
movement occurring in the image frame by frame. I
connected this value to the speed of the sampled sound;
as a result there was a proportional increase in pitch and
speed of sound according to the traffic cycle. I thought that
this translation can be implemented as a simple feedback
a public building gives to its surrounding environment.
Fig.33 The pattern of the traffic cycle.
The avenue in a side of the Library has four lanes and many
people wait for several minutes to access the Children’s
section entrance, which may give them time enough to
make the association of traffic and sound if loudspeakers
are mounted in the lateral façade.
3.3.4. Accumulation
In this prototype I wanted to experiment with a feedback
whose changes can only be perceived in a long timeframe.
I placed a video camera observing the Brooklyn Library
main hall from above, (almost plain vertical), so I could see
a large space and the traffic of people happening in it.
The sound used was the ambient sound of the same space.
The video image was analyzed in terms of amount of
Fig.35 The interior hall of the Brooklyn Library
movement happening each 15 frames. The number obtained
from that reading was used to control a “delay feedback”
in the sound. A delay feedback in sound is similar to a
“video feedback” as we may find in early video works of
Nam Jun Paik, where the camera points to a monitor
displaying the same image being captured, creating a
spiraled image because of the closed circuit between input
and output. In sound such effect can be accomplished by
reinserting a portion of the input to the output, with a slight
delay. The effect is a constant tone emerging from the
original sound, which changes also according to the input.
The amount of movement in the space was connected to
the amount of feedback in the sound, reflecting the physical
situation. In this way, we can obtain an aural “portrait” of
the space in a given moment. If we compare snapshots
along a day, we would see significant changes in the sound.
This experiment used an extremely long “exposure”, so the
Fig.36 The increases in the delay are changes of sound may not be perceived instantly, but along
implemented in a logarithmic curve, making the
feedback more “sensible” to small changes in the course of a day.
the environment, and more “stable” at the peak.
35
3.3.5. Panning Sound
This prototype was an experiment on tracking positions of
multiple users in space, and also the amount of movement
within zones of the space. Tracking people’s positions
requires certain conditions of light in the space in order to
differentiate people from the background. Also as the
lighting conditions may change along the day, there is the
need for periodic revision of the thresholds that determine
people’s presence. This task is hard, so I spent a great
amount of time figuring out a way to dynamically take
periodic measures of lighting conditions to readjust the
thresholds. There is also the problem of occlusion. When
two people are close to each other, it is almost impossible
for the computer program to differentiate them as two
individuals. Nevertheless, the position and distribution of
users in space is valuable information that can be used in
a variety of ways, for example the panning of sounds
between output channels. As I am currently working with
two output channels, Left and Right, I divided the video
input field in 16 vertical zones, each one reporting movement
in its own space, I assigned to each one of these zones a
numerical value from 0 t o16, each extreme representing
an output channel. The numbers in between represent
intermediate values in the amplitude of each channel. In
this way, if one person is present in the space, its location
pans the sound to the opposite side of which he/she is. As
the user moves around, sound moves accordingly.
36
In the case of a greater number of users, their positions
are averaged for determine the movement of sound. The
movement of sound in space would give a dynamic response
to peoples displacement within the space, creating an
attraction–repulsion relationship between sound and
listeners.
I decided to move sounds in the opposite direction because
this makes the movement more evident, because mapping
to the same position can be easily confused with sounds
being louder by being closer to a channel side, which
happens naturally in the proximity of a sound source.
37
3.4. Summary of Implementations
Physical Scale
Image Analysis
Mapping to Sound
Trigger Samples Pointer to file time Panning / Reverb Speed of soundtrack Feedback Delay
Amount
Required Temporality
38
3.5. Thesis Installation
39
3.5.2. The Context of the Public Space
41
Chapter 4. Discussion and Analysis
Chapter 5. Conclusion
5.1. Conclusions
46
Chapter 6. Bibliography
Books
47
Leitner, Bernhard . Sound:Space. New York University Press.
New York, 1978.
48
Kruger, Myron . Essays.
49
Appendices
Stereo Delay
Feedback Pacth
50
A.2. Program Interface for Aural Limbo
Audio Router Pan Process Effects Mixer Process Mix / Pan Monitor
Control Motion
Random
Track Picker
51
A.3. Program Interface for Traffic Report
52
B. Suplemmentary Sketch
53
Colophon
54
Mateo Zlatar
mateo@parsons.edu
2003