Mini Savior Opt. (2017) is a twenty-five minute live electronic performance which demonstrates an... more Mini Savior Opt. (2017) is a twenty-five minute live electronic performance which demonstrates an enactive and embodied approach to interactive and improvisational music systems. The piece was formed out of a playful exploration of my most recent hybrid analogue/digital performance system. An excessive number of components mutually affect each other through an ecological network of sound analysis and digital signal processing (DSP). Engaging with different parts of the instrument through tangible and haptic controllers, I bring a sense of immediacy into my hands: the slightest movement may trigger a drastic change in sound, which in turn may activate other processes within the network. Through the physical struggle, the performer, vulnerable to the fragile instabilities that have been potentialised, attempts to navigate the performance space.
A collaboratively devised performance of Morton Subotnik's Sidewinder takes a recording as th... more A collaboratively devised performance of Morton Subotnik's Sidewinder takes a recording as the jumping off point from which to develop a collective instrument with a fixed sonic outcome in mind. Work on Christian Wolff's For 1, 2, or 3 People features negotiations between text and technology in pursuit of fluent, timely exchange between players. Sean Williams' Electronic Skank, on the other hand, takes the techne of Jamaican dub practice as a basis for organising bodies, technologies and sounds. Each of these projects addressed practical and aesthetic problematics of collective electronic musicking in a distinct way. We explore these through the optic of the performance ecosystem (Waters, 2007): specifically the idea that, in practice, distinctions between performers, instruments and environments are dynamic and contingent. By unpacking the notion of "environment" somewhat, we trace particular valances that arose in these projects between the bodily and technically immediate, and various planes of social, temporal and technical environmental influence (Born, 2010). In this way, we aim to contextualise a discussion of concrete practical responses to musical issues into a wider frame that affords reflection on the social, cultural and historical specificity of the environment in which these issues were encountered. Our hope is that this embellishment of Waters' framework provides a possible approach for reflective practice-as-research that can accommodate musical-technical particularity and historically situated critical awareness.
Discussions of pedagogical approaches to computer music are often rooted within the realm of high... more Discussions of pedagogical approaches to computer music are often rooted within the realm of higher education alone. This paper describes Sound, Electronics and Music, a large-scale project in which tutelage was provided on various topics related to sound and music technology to around nine hundred school children in Scotland in 2014 and 2015. Sixteen schools were involved, including two additional support needs schools. The project engaged several expert musicians and researchers to deliver the different areas of the course. A particular emphasis was placed on providing a form of music education that would engender creative practice that was available to all, regardless of both musical ability and background. The findings and outcomes of the project suggest that we should not be restricting the discussion of how to continue to educate future generations in the practices surrounding computer music to the university level. We may be failing to engage an age group that is growing read...
HCI International 2020 – Late Breaking Papers, 2020
This research develops a novel way of rethinking cultural and social behavior using computational... more This research develops a novel way of rethinking cultural and social behavior using computationally augmented artifacts. These 'instruments' provide various types of auditory feedback when manipulated by certain actions within social contexts, such as a bar or dining space. They foster affective social engagement through the habitual and explorative actions that they afford in everyday contexts, and their resulting auditory feedback. The goal is not only to observe how social interactions are affected by the manipulation of augmented artifacts, but also to observe how the sounds and manipulations affect psycho-sociological [1] changes towards more collaborative social relations during the processes of participatory sense-making [2]. In this paper, we present: a) a study of dynamic social interaction and how we instrumented tangible artifacts to reflect and induce engagement, b) a literature review that provides background for our design methodology, c) 'vocal prototyping'-a responsive media technique for developing action-sonic mappings, d) our experimental prototype based on this design methodology.
Nearly two decades after its inception as a workshop at the Association for Computing Machinery C... more Nearly two decades after its inception as a workshop at the Association for Computing Machinery Conference on Human Factors in Computing Systems, New Interfaces for Musical Expression (NIME) exists as an established international conference significantly distinct from its precursor. Although this origin story is often noted, the implications of NIME's history as emerging from a field predominantly dealing with human–computer interaction (HCI) have rarely been discussed. In this paper we highlight many of the recent—and some not so recent—challenges that have been brought upon the NIME community as it attempts to maintain and expand its identity as a platform for multidisciplinary research into HCI, interface design, and electronic and computer music. We discuss the relationship between the market demands of the neoliberal university—which have underpinned academia's drive for innovation—and the quantification and economization of research performance that have facilitated ce...
Access and Mediation: Transdisciplinary Perspectives on Attention, 2022
In this chapter we consider attention from an enactive-ecological perspective in which the organi... more In this chapter we consider attention from an enactive-ecological perspective in which the organism-with-environment interdependencies that emerge in the process of living are fundamental and necessary for understanding cognition (Thompson 2010). While technological advances have often provided an impetus for empirical studies of attention, we propose, moreover, that such developments have enabled the facilitation of new highly participatory forms of musicking. In these types of technologically-mediated musical interactions, there is fertile ground for exploring attention within the types of organizational dynamics that emerge over time between group members within musical situations, especially those related to improvisation.
We propose three enactive-ecological themes that we develop in terms of three ranges of timescales of attention modulation. The purpose of the three themes and associated timescales is to provide a way of making distinctions and disentangling processes while doing justice to the complex interdependencies of organizational dynamics (Gahrn-Andersen et al. 2019). Firstly, we consider the notion of habit which affirms the interdependencies between organism and environment as a fundamentally embodied process of identity generation (James and Loaiza 2020). Secondly, we consider the role of attention within social interactivity, whereby attention can be said to emerge within the processes of participatory sense-making (De Jaegher and Di Paolo 2007). Finally, we propose that while attention has often been articulated through ocularcentric metaphors of focus or illumination, a more fruitful approach might involve articulating the speed, strength, or amplification of such dynamics as a processual, non-static paradigm. Technologically-mediated musical practices, particularly ones which involve improvisational modes of playing, offer not only an environment in which these ideas can be studied, but also provide participatory and experiential platforms for interdisciplinary research (Hayes 2019).
Nearly two decades after its inception as a workshop at the ACM Conference on Human Factors in Co... more Nearly two decades after its inception as a workshop at the ACM Conference on Human Factors in Computing Systems , NIME exists as an established international conference significantly distinct from its precursor. While this origin story is often noted, the implications of NIME's history as emerging from a field predominantly dealing with human-computer interaction have rarely been discussed. In this paper we highlight many of the recent-and some not so recent-challenges that have been brought upon the NIME community as it attempts to maintain and expand its identity as a platform for multidisciplinary research into HCI, interface design, and electronic and computer music. We discuss the relationship between the market demands of the neoliberal university-which have underpinned academia's drive for innovation-and the quantification and economisa-tion of research performance which have facilitated certain disciplinary and social frictions to emerge within NIME-related research and practice. Drawing on work that engages with feminist theory and cultural studies, we suggest that critical reflection and moreover mediation is necessary in order to address burgeoning concerns which have been raised within the NIME discourse in relation to method-ological approaches, 'diversity and inclusion', 'accessibility', and the fostering of rigorous interdisciplinary research.
The paradigm of enactive music cognition offers an anti-representational framework for understand... more The paradigm of enactive music cognition offers an anti-representational framework for understanding musical activity as both corporeal and culturally-situated. In this paper, I discuss live electronic musical improvisation as an exemplary model for the enactive framework in its ability to demonstrate the importance of participatory, relational, emergent, and embodied musical activities and processes. Following Gallagher, I argue that the Dreyfus model of skill acquisition, where performers develop from novices to experts who may eventually achieve a state of ‘mindless flow’, does not adequately account for what can happen during various forms of musical play. A critical study of improvisation reveals that a more generous conception of meaningful musical activity is needed, particularly in terms of who is able to take part as an improviser. I contextualise these ideas from the position of being an improviser of live electronic music performed on self-built, hybrid analogue/digital instruments, my background in creating expressive musical systems for people with profound and complex learning difficulties, and through my recent explorations of both pedagogical and research approaches to interdisciplinary improvisation.
Cochlear implant (CI) users' poor speech recognition in noise and music perception may be both du... more Cochlear implant (CI) users' poor speech recognition in noise and music perception may be both due to their limited access to pitch cues such as the fundamental frequency (F0). Recent studies showed that similar to residual low-frequency acoustic hearing, vibrotactile presentation of the F0 significantly improved speech recognition in noise of CI users. The present study tested whether F0-based vibrotactile stimulation can improve melodic contour identification (MCI) of normal-hearing listeners with acoustically simulated CI processing. Each melodic contour consisted of five musical notes with one of nine contour patterns (rising, falling, or flat in each half of the contour). The F0 of the middle note was 220 or 880 Hz, and the frequency intervals between adjacent notes were 1, 3, or 5 semitones. The F0 of each note was extracted in real time and transposed to a vibration frequency centered around 110 Hz at the right forearm top. MCI was tested in five experimental conditions (with a 4-or 8-channel CI simulation alone, vibrotactile stimulation alone, and 4-or 8-channel CI simulation plus vibrotactile stimulation), each after the same amount of brief training was provided. Results showed that discrimination of vibrotactile stimuli significantly improved from chance to near perfect as the vibration frequency interval increased from 0.25 to 3 semitones. The MCI performance with vibrotactile stimulation alone was similar to that with the 4-channel CI simulation alone, but was significantly worse than that with the 8-channel CI simulation alone. Significant improvement in MCI performance with the addition of vibrotactile stimulation was only found with the 4-channel CI simulation when the middle F0 was 880 Hz and when the frequency intervals were 3 or 5 semitones. The improvement in MCI performance with than without vibrotactile stimulation was significantly correlated with the baseline MCI performance with 4-channel CI simulation alone or with the MCI performance difference between vibrotactile stimulation and 8-channel CI simulation. Therefore, when the simulated or real CI performance is relatively poor, vibrotactile stimulation based on the F0 may improve MCI with acoustic CI simulations and perhaps in real CI users as well.
This paper discusses the ongoing research project Sounding Out Spaces which explores technologica... more This paper discusses the ongoing research project Sounding Out Spaces which explores technologically-mediated sonic responses to site through human, material, and environmental considerations. Informed by theories of self-organisation and reflexivity, the project attempts to build a methodology for developing portable sound-systems using microcontroller technologies in which sonic entities emerge over time through mutually affecting relationships with the environments in which they are situated. I assess this work with reference to Hayles' discussion of second-order cybernetics and its implications for conceptualising musical systems as sets of relationships between living things, machines, and the environment. I expound these ideas through two case studies of the latest iteration of the tools-hardware and software systems-developed for this work: firstly, a large-scale installation which was presented in the Sonoran Desert of Arizona, United States. The second case study took place as a series of experiments at the Ars Bioarctica residency in the sub-arctic tundra at the Kilpisjärvi Biological Station, University of Helsinki, Finland. Keywords-Sonic interaction design, sonic art, ecosystems, emergence, autopoiesis 1. Introduction Sounding Out Spaces is a project which approaches artwork creation as a process in which sonic activity is brought forth through the organisation and movement of things within a medium. These things might include sensing technologies, microcomputers, humans, and other organisms (see Figure 1). Air is the routine medium, but experiments with water and ice have also been undertaken. The project eschews traditional human-computer interaction (HCI) paradigms that are pervasively drawn from in digital musical instrument (DMI) design. In this model, a performer will typically use a variety of input gestures that are sensed by a controller or tracking mechanism, in order to affect parameters within digital signal processing (DSP), resulting in changes in sonic output. Rather, Sounding Out Spaces decentralises-but does not forget-the human, drawing on the application of second-order cybernetics in order to produce sonic structures that continuously evolve over time. Specifically, it explores the notion of structurally determined-or autopoietic (Maturana and Varela, 1973/1980)-systems. These are systems in which behaviour can be triggered by changes in environmental factors, but only the structure of the system itself is responsible for how these external perturbations are dealt with temporally.
This paper explores the idea of using virtual textural terrains as a means of generating haptic p... more This paper explores the idea of using virtual textural terrains as a means of generating haptic profiles for force-feedback controllers. This approach breaks from the paradigm established within audio-haptic research over the last few decades where physical models within virtual environments are designed to transduce gesture into sonic output. We outline a method for generating multimodal terrains using basis functions, which are rendered into monochro-matic visual representations for inspection. This visual terrain is traversed using a haptic controller, the NovInt Falcon , which in turn receives force information based on the grayscale value of its location in this virtual space. As the image is traversed by a performer the levels of resistance vary, and the image is realized as a physical terrain. We discuss the potential of this approach to afford engaging musical experiences for both the performer and the audience as iterated through numerous performances.
Advancements in embedded computer platforms have allowed data to be collected and shared between ... more Advancements in embedded computer platforms have allowed data to be collected and shared between objects—or smart devices—in a network. While this has resulted in highly functional outcomes in fields such as automation and monitoring, there are also implications for artistic and expressive systems. In this paper we present a pluralistic approach to incorporating environmental factors within the field of site-responsive sonic art using embedded audio and data processing techniques. In particular, we focus on the role of such systems within an ecosystemic framework, both in terms of incorporating systems of living organisms, as well as sonic interaction design. We describe the implementation of such a system within a large-scale site-responsive sonic art installation that took place in the subtropical desert climate of Arizona in 2017.
In this paper we explore an interdisciplinary approach towards an aesthetics of touch. Research i... more In this paper we explore an interdisciplinary approach towards an aesthetics of touch. Research into the role of the body has become increasingly prevalent in elds ranging from philosophy of mind to human-computer interaction. At the same time, haptic technology has becoming ubiquitous within personalized devices and wearables. Despite this, touch remains largely under-explored within contemporary aesthetics. We rstly outline what might be gained from artistic practices that acknowledge a multisensory model of perception. Secondly, we discuss the diiculties of arriving at a standardized taxonomy for touch-based aesthetics and why this endeavor may not be fruitful. Finally, we outline an approach based on rst-person felt experiences, drawing on creative practice research involving computational technology within the elds of somatics, dance, and music.
Discussions of pedagogical approaches to computer music are often rooted within the realm of high... more Discussions of pedagogical approaches to computer music are often rooted within the realm of higher education alone. This paper describes Sound, Electronics and Music , a large-scale project in which tutelage was provided on various topics related to sound and music technology to around nine hundred school children in Scotland in 2014 and 2015. Sixteen schools were involved, including two additional support needs schools. The project engaged several expert musicians and researchers to deliver the different areas of the course. A particular emphasis was placed on providing a form of music education that would engender creative practice that was available to all, regardless of both musical ability and background. The findings and outcomes of the project suggest that we should not be restricting the discussion of how to continue to educate future generations in the practices surrounding computer music to the university level. We may be failing to engage an age group that is growing readily familiar with the skills and vocabulary surrounding new technologies.
Skin Music (2012) is a musical composition that is experienced as a private, multisensory install... more Skin Music (2012) is a musical composition that is experienced as a private, multisensory installation by one person at a time. By lying on a piece of bespoke furniture, the listener perceives the music both through the usual auditory channels, as well as by different types of haptic sensation, through their body. The piece addresses the shared perceptual experiences of sonic and haptic sensation through an exploration of vibra-tional feedback.
Live music making can be understood as an enactive process , whereby musical experiences are crea... more Live music making can be understood as an enactive process , whereby musical experiences are created through human action. This suggests that musical worlds coevolve with their agents through repeated sensorimotor interactions with the environment (where the music is being created), and at the same time cannot be separated from their sociocultural contexts. This paper investigates this claim by exploring ways in which technology, physiology, and context are bound up within two different musical scenarios: live electronic musical performance; and person-centred arts applications of NIMEs. In this paper I outline an ethnographic and phenomeno-logical enquiry into my experiences as both a performer of live electronic and electro-instrumental music, as well as my extensive background in working with new technologies in various therapeutic and person-centred artistic situations. This is in order to explore the sociocultural and technological contexts in which these activities take place. I propose that by understanding creative musical participation as a highly contextualised practice, we may discover that the greatest impact of rapidly developing technological resources is their ability to afford richly diverse, personalised, and embodied forms of music making. I argue that this is applicable over a wide range of musical communities.
One of the most vital feedback systems that has been embedded in musicians for centuries is that ... more One of the most vital feedback systems that has been embedded in musicians for centuries is that of physical response. In the same way that auditory information is available and used throughout a performance, a musician will continuously reassess their playing by making use of not only their specialised sensorimotor skills, but also the tangible feedback that is relayed to them through the body of the instrument. This paper discusses approaches to the development of an augmented instrument, namely the hybrid piano, which focuses on the notion of performance as perceptually guided action. While the acoustic component of the sound energy of the augmented instrument is created within the real-world interactions between hammers, resonating strings, and the soundboard, the digital sonic events cannot be located in a similar palpable source. By exploring notions of multimodality and haptic feedback, the ongoing processes of human action and perception within instrumental performance can be maintained for the player, whilst arguably, also enhancing the experience for the listener.
While digital technologies offer a host of new sonic pos- sibilities, we are no longer dealing wi... more While digital technologies offer a host of new sonic pos- sibilities, we are no longer dealing with the physical vi- brations of strings, tubes and solid bodies as the sound source, but rather with the impalpable numerical streams of digital signal processing (DSP). As a result, when we perform with digital musical instruments (DMIs), we can no longer make use of haptic feedback provided through the body of the instrument itself. Furthermore, many DMIs are derived from interfaces designed for effortlessly smooth human-computer interaction. Here, however, the struggle afforded by the resistance and physical forces of acoustic instruments, which I will argue is integral to musical performance, is all but lost.
This paper discusses the musical outcomes of an ex- ploration into the use of a haptic interface as an instrument for the performance of digital music. I will argue that it is the reintroduction of these tangible forces that is cru- cial for the articulation and effectuation of sonic ideas. In particular, the instrument will be discussed in relation to the work Running Backwards, Uphill (2011) for piano trio and live electronics, where a potentially high level of sophistication of expression was required that would allow the laptop performer to embody the musical intentions of the piece.
This paper describes the implementation of NeVIS, a local network system that establishes communi... more This paper describes the implementation of NeVIS, a local network system that establishes communication between individual performers, as well as between laptop and performers. Specifically, this is achieved by making use of vibrotactile feedback as a signalling tool within an improvisational setting. A discussion of the current developments regarding the use of networks within improvisation is presented, followed by an outline of the benefits of utilising the haptic feedback channel as a further sensory information pathway when performing digital music. We describe a case study of the system within the context of our computer-mediated improvisational duo Můstek, involving piano, percussion and live electronics. Here, a cueing system or framework is imposed over the improvisation and is transmitted directly to the skin of the performers via tiny vibrations. Additionally, performers may make use of simple vibrotactile signals to enhance traditional visual cues that are often employed within performance. A new work, Socks and Ammo, was created using NeVIS, and was presented at various international conferences and festivals. We also tested the system itself within a group of postgraduate researchers and composers. Qualitative evaluation of the musical outcomes as experienced both by the performers and by the listeners at these events is offered, as well as implications about the nature of collaborative music-making.
Mini Savior Opt. (2017) is a twenty-five minute live electronic performance which demonstrates an... more Mini Savior Opt. (2017) is a twenty-five minute live electronic performance which demonstrates an enactive and embodied approach to interactive and improvisational music systems. The piece was formed out of a playful exploration of my most recent hybrid analogue/digital performance system. An excessive number of components mutually affect each other through an ecological network of sound analysis and digital signal processing (DSP). Engaging with different parts of the instrument through tangible and haptic controllers, I bring a sense of immediacy into my hands: the slightest movement may trigger a drastic change in sound, which in turn may activate other processes within the network. Through the physical struggle, the performer, vulnerable to the fragile instabilities that have been potentialised, attempts to navigate the performance space.
A collaboratively devised performance of Morton Subotnik's Sidewinder takes a recording as th... more A collaboratively devised performance of Morton Subotnik's Sidewinder takes a recording as the jumping off point from which to develop a collective instrument with a fixed sonic outcome in mind. Work on Christian Wolff's For 1, 2, or 3 People features negotiations between text and technology in pursuit of fluent, timely exchange between players. Sean Williams' Electronic Skank, on the other hand, takes the techne of Jamaican dub practice as a basis for organising bodies, technologies and sounds. Each of these projects addressed practical and aesthetic problematics of collective electronic musicking in a distinct way. We explore these through the optic of the performance ecosystem (Waters, 2007): specifically the idea that, in practice, distinctions between performers, instruments and environments are dynamic and contingent. By unpacking the notion of "environment" somewhat, we trace particular valances that arose in these projects between the bodily and technically immediate, and various planes of social, temporal and technical environmental influence (Born, 2010). In this way, we aim to contextualise a discussion of concrete practical responses to musical issues into a wider frame that affords reflection on the social, cultural and historical specificity of the environment in which these issues were encountered. Our hope is that this embellishment of Waters' framework provides a possible approach for reflective practice-as-research that can accommodate musical-technical particularity and historically situated critical awareness.
Discussions of pedagogical approaches to computer music are often rooted within the realm of high... more Discussions of pedagogical approaches to computer music are often rooted within the realm of higher education alone. This paper describes Sound, Electronics and Music, a large-scale project in which tutelage was provided on various topics related to sound and music technology to around nine hundred school children in Scotland in 2014 and 2015. Sixteen schools were involved, including two additional support needs schools. The project engaged several expert musicians and researchers to deliver the different areas of the course. A particular emphasis was placed on providing a form of music education that would engender creative practice that was available to all, regardless of both musical ability and background. The findings and outcomes of the project suggest that we should not be restricting the discussion of how to continue to educate future generations in the practices surrounding computer music to the university level. We may be failing to engage an age group that is growing read...
HCI International 2020 – Late Breaking Papers, 2020
This research develops a novel way of rethinking cultural and social behavior using computational... more This research develops a novel way of rethinking cultural and social behavior using computationally augmented artifacts. These 'instruments' provide various types of auditory feedback when manipulated by certain actions within social contexts, such as a bar or dining space. They foster affective social engagement through the habitual and explorative actions that they afford in everyday contexts, and their resulting auditory feedback. The goal is not only to observe how social interactions are affected by the manipulation of augmented artifacts, but also to observe how the sounds and manipulations affect psycho-sociological [1] changes towards more collaborative social relations during the processes of participatory sense-making [2]. In this paper, we present: a) a study of dynamic social interaction and how we instrumented tangible artifacts to reflect and induce engagement, b) a literature review that provides background for our design methodology, c) 'vocal prototyping'-a responsive media technique for developing action-sonic mappings, d) our experimental prototype based on this design methodology.
Nearly two decades after its inception as a workshop at the Association for Computing Machinery C... more Nearly two decades after its inception as a workshop at the Association for Computing Machinery Conference on Human Factors in Computing Systems, New Interfaces for Musical Expression (NIME) exists as an established international conference significantly distinct from its precursor. Although this origin story is often noted, the implications of NIME's history as emerging from a field predominantly dealing with human–computer interaction (HCI) have rarely been discussed. In this paper we highlight many of the recent—and some not so recent—challenges that have been brought upon the NIME community as it attempts to maintain and expand its identity as a platform for multidisciplinary research into HCI, interface design, and electronic and computer music. We discuss the relationship between the market demands of the neoliberal university—which have underpinned academia's drive for innovation—and the quantification and economization of research performance that have facilitated ce...
Access and Mediation: Transdisciplinary Perspectives on Attention, 2022
In this chapter we consider attention from an enactive-ecological perspective in which the organi... more In this chapter we consider attention from an enactive-ecological perspective in which the organism-with-environment interdependencies that emerge in the process of living are fundamental and necessary for understanding cognition (Thompson 2010). While technological advances have often provided an impetus for empirical studies of attention, we propose, moreover, that such developments have enabled the facilitation of new highly participatory forms of musicking. In these types of technologically-mediated musical interactions, there is fertile ground for exploring attention within the types of organizational dynamics that emerge over time between group members within musical situations, especially those related to improvisation.
We propose three enactive-ecological themes that we develop in terms of three ranges of timescales of attention modulation. The purpose of the three themes and associated timescales is to provide a way of making distinctions and disentangling processes while doing justice to the complex interdependencies of organizational dynamics (Gahrn-Andersen et al. 2019). Firstly, we consider the notion of habit which affirms the interdependencies between organism and environment as a fundamentally embodied process of identity generation (James and Loaiza 2020). Secondly, we consider the role of attention within social interactivity, whereby attention can be said to emerge within the processes of participatory sense-making (De Jaegher and Di Paolo 2007). Finally, we propose that while attention has often been articulated through ocularcentric metaphors of focus or illumination, a more fruitful approach might involve articulating the speed, strength, or amplification of such dynamics as a processual, non-static paradigm. Technologically-mediated musical practices, particularly ones which involve improvisational modes of playing, offer not only an environment in which these ideas can be studied, but also provide participatory and experiential platforms for interdisciplinary research (Hayes 2019).
Nearly two decades after its inception as a workshop at the ACM Conference on Human Factors in Co... more Nearly two decades after its inception as a workshop at the ACM Conference on Human Factors in Computing Systems , NIME exists as an established international conference significantly distinct from its precursor. While this origin story is often noted, the implications of NIME's history as emerging from a field predominantly dealing with human-computer interaction have rarely been discussed. In this paper we highlight many of the recent-and some not so recent-challenges that have been brought upon the NIME community as it attempts to maintain and expand its identity as a platform for multidisciplinary research into HCI, interface design, and electronic and computer music. We discuss the relationship between the market demands of the neoliberal university-which have underpinned academia's drive for innovation-and the quantification and economisa-tion of research performance which have facilitated certain disciplinary and social frictions to emerge within NIME-related research and practice. Drawing on work that engages with feminist theory and cultural studies, we suggest that critical reflection and moreover mediation is necessary in order to address burgeoning concerns which have been raised within the NIME discourse in relation to method-ological approaches, 'diversity and inclusion', 'accessibility', and the fostering of rigorous interdisciplinary research.
The paradigm of enactive music cognition offers an anti-representational framework for understand... more The paradigm of enactive music cognition offers an anti-representational framework for understanding musical activity as both corporeal and culturally-situated. In this paper, I discuss live electronic musical improvisation as an exemplary model for the enactive framework in its ability to demonstrate the importance of participatory, relational, emergent, and embodied musical activities and processes. Following Gallagher, I argue that the Dreyfus model of skill acquisition, where performers develop from novices to experts who may eventually achieve a state of ‘mindless flow’, does not adequately account for what can happen during various forms of musical play. A critical study of improvisation reveals that a more generous conception of meaningful musical activity is needed, particularly in terms of who is able to take part as an improviser. I contextualise these ideas from the position of being an improviser of live electronic music performed on self-built, hybrid analogue/digital instruments, my background in creating expressive musical systems for people with profound and complex learning difficulties, and through my recent explorations of both pedagogical and research approaches to interdisciplinary improvisation.
Cochlear implant (CI) users' poor speech recognition in noise and music perception may be both du... more Cochlear implant (CI) users' poor speech recognition in noise and music perception may be both due to their limited access to pitch cues such as the fundamental frequency (F0). Recent studies showed that similar to residual low-frequency acoustic hearing, vibrotactile presentation of the F0 significantly improved speech recognition in noise of CI users. The present study tested whether F0-based vibrotactile stimulation can improve melodic contour identification (MCI) of normal-hearing listeners with acoustically simulated CI processing. Each melodic contour consisted of five musical notes with one of nine contour patterns (rising, falling, or flat in each half of the contour). The F0 of the middle note was 220 or 880 Hz, and the frequency intervals between adjacent notes were 1, 3, or 5 semitones. The F0 of each note was extracted in real time and transposed to a vibration frequency centered around 110 Hz at the right forearm top. MCI was tested in five experimental conditions (with a 4-or 8-channel CI simulation alone, vibrotactile stimulation alone, and 4-or 8-channel CI simulation plus vibrotactile stimulation), each after the same amount of brief training was provided. Results showed that discrimination of vibrotactile stimuli significantly improved from chance to near perfect as the vibration frequency interval increased from 0.25 to 3 semitones. The MCI performance with vibrotactile stimulation alone was similar to that with the 4-channel CI simulation alone, but was significantly worse than that with the 8-channel CI simulation alone. Significant improvement in MCI performance with the addition of vibrotactile stimulation was only found with the 4-channel CI simulation when the middle F0 was 880 Hz and when the frequency intervals were 3 or 5 semitones. The improvement in MCI performance with than without vibrotactile stimulation was significantly correlated with the baseline MCI performance with 4-channel CI simulation alone or with the MCI performance difference between vibrotactile stimulation and 8-channel CI simulation. Therefore, when the simulated or real CI performance is relatively poor, vibrotactile stimulation based on the F0 may improve MCI with acoustic CI simulations and perhaps in real CI users as well.
This paper discusses the ongoing research project Sounding Out Spaces which explores technologica... more This paper discusses the ongoing research project Sounding Out Spaces which explores technologically-mediated sonic responses to site through human, material, and environmental considerations. Informed by theories of self-organisation and reflexivity, the project attempts to build a methodology for developing portable sound-systems using microcontroller technologies in which sonic entities emerge over time through mutually affecting relationships with the environments in which they are situated. I assess this work with reference to Hayles' discussion of second-order cybernetics and its implications for conceptualising musical systems as sets of relationships between living things, machines, and the environment. I expound these ideas through two case studies of the latest iteration of the tools-hardware and software systems-developed for this work: firstly, a large-scale installation which was presented in the Sonoran Desert of Arizona, United States. The second case study took place as a series of experiments at the Ars Bioarctica residency in the sub-arctic tundra at the Kilpisjärvi Biological Station, University of Helsinki, Finland. Keywords-Sonic interaction design, sonic art, ecosystems, emergence, autopoiesis 1. Introduction Sounding Out Spaces is a project which approaches artwork creation as a process in which sonic activity is brought forth through the organisation and movement of things within a medium. These things might include sensing technologies, microcomputers, humans, and other organisms (see Figure 1). Air is the routine medium, but experiments with water and ice have also been undertaken. The project eschews traditional human-computer interaction (HCI) paradigms that are pervasively drawn from in digital musical instrument (DMI) design. In this model, a performer will typically use a variety of input gestures that are sensed by a controller or tracking mechanism, in order to affect parameters within digital signal processing (DSP), resulting in changes in sonic output. Rather, Sounding Out Spaces decentralises-but does not forget-the human, drawing on the application of second-order cybernetics in order to produce sonic structures that continuously evolve over time. Specifically, it explores the notion of structurally determined-or autopoietic (Maturana and Varela, 1973/1980)-systems. These are systems in which behaviour can be triggered by changes in environmental factors, but only the structure of the system itself is responsible for how these external perturbations are dealt with temporally.
This paper explores the idea of using virtual textural terrains as a means of generating haptic p... more This paper explores the idea of using virtual textural terrains as a means of generating haptic profiles for force-feedback controllers. This approach breaks from the paradigm established within audio-haptic research over the last few decades where physical models within virtual environments are designed to transduce gesture into sonic output. We outline a method for generating multimodal terrains using basis functions, which are rendered into monochro-matic visual representations for inspection. This visual terrain is traversed using a haptic controller, the NovInt Falcon , which in turn receives force information based on the grayscale value of its location in this virtual space. As the image is traversed by a performer the levels of resistance vary, and the image is realized as a physical terrain. We discuss the potential of this approach to afford engaging musical experiences for both the performer and the audience as iterated through numerous performances.
Advancements in embedded computer platforms have allowed data to be collected and shared between ... more Advancements in embedded computer platforms have allowed data to be collected and shared between objects—or smart devices—in a network. While this has resulted in highly functional outcomes in fields such as automation and monitoring, there are also implications for artistic and expressive systems. In this paper we present a pluralistic approach to incorporating environmental factors within the field of site-responsive sonic art using embedded audio and data processing techniques. In particular, we focus on the role of such systems within an ecosystemic framework, both in terms of incorporating systems of living organisms, as well as sonic interaction design. We describe the implementation of such a system within a large-scale site-responsive sonic art installation that took place in the subtropical desert climate of Arizona in 2017.
In this paper we explore an interdisciplinary approach towards an aesthetics of touch. Research i... more In this paper we explore an interdisciplinary approach towards an aesthetics of touch. Research into the role of the body has become increasingly prevalent in elds ranging from philosophy of mind to human-computer interaction. At the same time, haptic technology has becoming ubiquitous within personalized devices and wearables. Despite this, touch remains largely under-explored within contemporary aesthetics. We rstly outline what might be gained from artistic practices that acknowledge a multisensory model of perception. Secondly, we discuss the diiculties of arriving at a standardized taxonomy for touch-based aesthetics and why this endeavor may not be fruitful. Finally, we outline an approach based on rst-person felt experiences, drawing on creative practice research involving computational technology within the elds of somatics, dance, and music.
Discussions of pedagogical approaches to computer music are often rooted within the realm of high... more Discussions of pedagogical approaches to computer music are often rooted within the realm of higher education alone. This paper describes Sound, Electronics and Music , a large-scale project in which tutelage was provided on various topics related to sound and music technology to around nine hundred school children in Scotland in 2014 and 2015. Sixteen schools were involved, including two additional support needs schools. The project engaged several expert musicians and researchers to deliver the different areas of the course. A particular emphasis was placed on providing a form of music education that would engender creative practice that was available to all, regardless of both musical ability and background. The findings and outcomes of the project suggest that we should not be restricting the discussion of how to continue to educate future generations in the practices surrounding computer music to the university level. We may be failing to engage an age group that is growing readily familiar with the skills and vocabulary surrounding new technologies.
Skin Music (2012) is a musical composition that is experienced as a private, multisensory install... more Skin Music (2012) is a musical composition that is experienced as a private, multisensory installation by one person at a time. By lying on a piece of bespoke furniture, the listener perceives the music both through the usual auditory channels, as well as by different types of haptic sensation, through their body. The piece addresses the shared perceptual experiences of sonic and haptic sensation through an exploration of vibra-tional feedback.
Live music making can be understood as an enactive process , whereby musical experiences are crea... more Live music making can be understood as an enactive process , whereby musical experiences are created through human action. This suggests that musical worlds coevolve with their agents through repeated sensorimotor interactions with the environment (where the music is being created), and at the same time cannot be separated from their sociocultural contexts. This paper investigates this claim by exploring ways in which technology, physiology, and context are bound up within two different musical scenarios: live electronic musical performance; and person-centred arts applications of NIMEs. In this paper I outline an ethnographic and phenomeno-logical enquiry into my experiences as both a performer of live electronic and electro-instrumental music, as well as my extensive background in working with new technologies in various therapeutic and person-centred artistic situations. This is in order to explore the sociocultural and technological contexts in which these activities take place. I propose that by understanding creative musical participation as a highly contextualised practice, we may discover that the greatest impact of rapidly developing technological resources is their ability to afford richly diverse, personalised, and embodied forms of music making. I argue that this is applicable over a wide range of musical communities.
One of the most vital feedback systems that has been embedded in musicians for centuries is that ... more One of the most vital feedback systems that has been embedded in musicians for centuries is that of physical response. In the same way that auditory information is available and used throughout a performance, a musician will continuously reassess their playing by making use of not only their specialised sensorimotor skills, but also the tangible feedback that is relayed to them through the body of the instrument. This paper discusses approaches to the development of an augmented instrument, namely the hybrid piano, which focuses on the notion of performance as perceptually guided action. While the acoustic component of the sound energy of the augmented instrument is created within the real-world interactions between hammers, resonating strings, and the soundboard, the digital sonic events cannot be located in a similar palpable source. By exploring notions of multimodality and haptic feedback, the ongoing processes of human action and perception within instrumental performance can be maintained for the player, whilst arguably, also enhancing the experience for the listener.
While digital technologies offer a host of new sonic pos- sibilities, we are no longer dealing wi... more While digital technologies offer a host of new sonic pos- sibilities, we are no longer dealing with the physical vi- brations of strings, tubes and solid bodies as the sound source, but rather with the impalpable numerical streams of digital signal processing (DSP). As a result, when we perform with digital musical instruments (DMIs), we can no longer make use of haptic feedback provided through the body of the instrument itself. Furthermore, many DMIs are derived from interfaces designed for effortlessly smooth human-computer interaction. Here, however, the struggle afforded by the resistance and physical forces of acoustic instruments, which I will argue is integral to musical performance, is all but lost.
This paper discusses the musical outcomes of an ex- ploration into the use of a haptic interface as an instrument for the performance of digital music. I will argue that it is the reintroduction of these tangible forces that is cru- cial for the articulation and effectuation of sonic ideas. In particular, the instrument will be discussed in relation to the work Running Backwards, Uphill (2011) for piano trio and live electronics, where a potentially high level of sophistication of expression was required that would allow the laptop performer to embody the musical intentions of the piece.
This paper describes the implementation of NeVIS, a local network system that establishes communi... more This paper describes the implementation of NeVIS, a local network system that establishes communication between individual performers, as well as between laptop and performers. Specifically, this is achieved by making use of vibrotactile feedback as a signalling tool within an improvisational setting. A discussion of the current developments regarding the use of networks within improvisation is presented, followed by an outline of the benefits of utilising the haptic feedback channel as a further sensory information pathway when performing digital music. We describe a case study of the system within the context of our computer-mediated improvisational duo Můstek, involving piano, percussion and live electronics. Here, a cueing system or framework is imposed over the improvisation and is transmitted directly to the skin of the performers via tiny vibrations. Additionally, performers may make use of simple vibrotactile signals to enhance traditional visual cues that are often employed within performance. A new work, Socks and Ammo, was created using NeVIS, and was presented at various international conferences and festivals. We also tested the system itself within a group of postgraduate researchers and composers. Qualitative evaluation of the musical outcomes as experienced both by the performers and by the listeners at these events is offered, as well as implications about the nature of collaborative music-making.
As a performer of firstly acoustic and latterly electronic and electro-instrumental music, I cons... more As a performer of firstly acoustic and latterly electronic and electro-instrumental music, I constantly seek to improve my mode of interaction with the digital realm: that is, to achieve a high level of sensitivity and expression. This thesis illustrates reasons why making use of haptic interfaces—which offer physical feedback and resistance to the performer—may be viewed as an important approach in addressing the shortcomings of some the standard systems used to mediate the performer’s engagement with various sorts of digital musical information. By examining the links between sound and touch, and the performer-instrument relationship, various new compositional and performance strategies start to emerge. I explore these through a portfolio of original musical works, which span the continuum of composition and improvisation, largely based around performance paradigms for piano and live electronics. I implement new haptic technologies, using vibrotactile feedback and resistant interfaces, as well as exploring more metaphorical connections between sound and touch. I demonstrate the impact that the research brings to the creative musical outcomes, along with the implications that these techniques have on the wider field of live electronic musical performance.
Uploads
Papers by Lauren Hayes
We propose three enactive-ecological themes that we develop in terms of three ranges of timescales of attention modulation. The purpose of the three themes and associated timescales is to provide a way of making distinctions and disentangling processes while doing justice to the complex interdependencies of organizational dynamics (Gahrn-Andersen et al. 2019). Firstly, we consider the notion of habit which affirms the interdependencies between organism and environment as a fundamentally embodied process of identity generation (James and Loaiza 2020). Secondly, we consider the role of attention within social interactivity, whereby attention can be said to emerge within the processes of participatory sense-making (De Jaegher and Di Paolo 2007). Finally, we propose that while attention has often been articulated through ocularcentric metaphors of focus or illumination, a more fruitful approach might involve articulating the speed, strength, or amplification of such dynamics as a processual, non-static paradigm. Technologically-mediated musical practices, particularly ones which involve improvisational modes of playing, offer not only an environment in which these ideas can be studied, but also provide participatory and experiential platforms for interdisciplinary research (Hayes 2019).
This paper discusses the musical outcomes of an ex- ploration into the use of a haptic interface as an instrument for the performance of digital music. I will argue that it is the reintroduction of these tangible forces that is cru- cial for the articulation and effectuation of sonic ideas. In particular, the instrument will be discussed in relation to the work Running Backwards, Uphill (2011) for piano trio and live electronics, where a potentially high level of sophistication of expression was required that would allow the laptop performer to embody the musical intentions of the piece.
We propose three enactive-ecological themes that we develop in terms of three ranges of timescales of attention modulation. The purpose of the three themes and associated timescales is to provide a way of making distinctions and disentangling processes while doing justice to the complex interdependencies of organizational dynamics (Gahrn-Andersen et al. 2019). Firstly, we consider the notion of habit which affirms the interdependencies between organism and environment as a fundamentally embodied process of identity generation (James and Loaiza 2020). Secondly, we consider the role of attention within social interactivity, whereby attention can be said to emerge within the processes of participatory sense-making (De Jaegher and Di Paolo 2007). Finally, we propose that while attention has often been articulated through ocularcentric metaphors of focus or illumination, a more fruitful approach might involve articulating the speed, strength, or amplification of such dynamics as a processual, non-static paradigm. Technologically-mediated musical practices, particularly ones which involve improvisational modes of playing, offer not only an environment in which these ideas can be studied, but also provide participatory and experiential platforms for interdisciplinary research (Hayes 2019).
This paper discusses the musical outcomes of an ex- ploration into the use of a haptic interface as an instrument for the performance of digital music. I will argue that it is the reintroduction of these tangible forces that is cru- cial for the articulation and effectuation of sonic ideas. In particular, the instrument will be discussed in relation to the work Running Backwards, Uphill (2011) for piano trio and live electronics, where a potentially high level of sophistication of expression was required that would allow the laptop performer to embody the musical intentions of the piece.