short-paper

Human-Sound Interaction: Towards a Human-Centred Sonic Interaction Design approach

Authors:

Balandino Di Donato,

Christopher Dewey,

Tychonas MichailidisAuthors Info & Claims

MOCO '20: Proceedings of the 7th International Conference on Movement and Computing

Article No.: 14, Pages 1 - 4

https://doi.org/10.1145/3401956.3404233

Published: 15 July 2020 Publication History

Abstract

In this paper, we explore human-centered interaction design aspects that determine the realisation and appreciation of musical works (installations, composition and performance), interfaces for sound design and musical expression, augmented instruments, sonic aspects of virtual environments and interactive audiovisual performances. In this first work, with the human at the centre of the design, we started sketching modes of interaction with sound that could result direct, engaging, natural and embodied in a collaborative, interactive, inclusive and diverse music environment. We define this as Human-Sound Interaction (HSI). To facilitate the exploration of HSIs, we prototyped SoundSculpt, a cross-modal audio, holographic projection and mid-air haptic feedback system. During an informal half-day workshop, we observed that HSIs through SoundSculpt have the potential to foster new ways of interaction with sound and to make them accessible to diverse musicians, sound artists and audience.

References

[1]

Laura Bianchini, Michelangelo Lupone, Silvia Lanzalone, and Alessio Gabriele. 2019. Augmented Instruments at CRM - Centro Ricerche Musicali of Rome: Feed-Drum, SkinAct, WindBack and ResoFlute. In Proceedings of the International Computer Music Conference (ICMC '19). New York, NY, USA.

[2]

Jamie Bullock and Balandino Di Donato. 2016. Approaches to Visualising the Spatial Position of "Sound-Objects". In Proceedings of the Conference on Electronic Visualisation and the Arts (EVA '16). BCS Learning Development Ltd., London, United Kingdom, 15--22. https://doi.org/10.14236/ewic/EVA2016.4

Digital Library

[3]

Baptiste Caramiaux, Patrick Susini, Tommaso Bianco, et al. 2011. Gestural Embodiment of Environmental Sounds: an Experimental Study. In Proc. of the International Conference on New Interaces for Musical Expression (NIME '11). Oslo, Norway, 144--148.

[4]

Tom Carter, Sue Ann Seah, Benjamin Long, Bruce Drinkwater, and Sriram Subramanian. 2013. UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In Proc. of the 26th annual ACM symposium on User interface software and technology. ACM, 505--514.

Digital Library

[5]

Perry R. Cook. 2001. Principles for Designing Computer Music Controllers. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME'01). Seattle, WA, 3--6.

[6]

Alice-Ann Darrow and Julie Novak. 2007. The Effect of Vision and Hearing Loss on Listeners' Perception of Referential Meaning in Music. Journal of Music Therapy 44, 1 (03 2007), 57--73. https://doi.org/10.1093/jmt/44.1.57

[7]

Christopher Dewey and Jonathan P. Wakefield. 2019. Exploring the Container Metaphor for Equalisation Manipulation. In Proceedings of the International Conference on New Interfaces for Musical Expression. UFRGS, Porto Alegre, Brazil, 128--129.

[8]

Paul Dourish. 2001. Where The Action Is: The Foundations of Embodied Interaction. MIT Press.

[9]

John Levack Drever. 2019. 'Primacy of the Ear' - But Whose Ear?: The case for auraldiversity in sonic arts practice and discourse. Organised Sound 24, 1 (2019), 85--95. https://doi.org/10.1017/S1355771819000086

[10]

Karmen Franinović and Stefania Serafin. 2013. Sonic interaction design. Mit Press.

[11]

Alessio Gabriele, Michelangelo Lupone, Paola Inverardi, and Patrizio Pelliccione. 2010. Ad-opera: music-inspired self-adaptive systems. In Proceedings of the FSE/SDP workshop on Future of software engineering research. 119--124.

Digital Library

[12]

William W. Gaver. 1991. Technology Affordances. In Proc. of the SIGCHI Conference on Human Factors in Computing Systems (CHI '91). ACM, New York, NY, USA, 79--84. https://doi.org/10.1145/108844.108856

[13]

Steven Gelineck, Dannie Korsgaard, and Morten Büchert. 2015. Stage- vs. Channel-strip Metaphor - Comparing Performance when Adjusting Volume and Panning of a Single Channel in a Stereo Mix. In Proceedings of the International Conference on New Interfaces for Musical Expression. Baton Rouge, Louisiana, USA, 343--346.

Digital Library

[14]

Steven Gelineck and Anders Kirk Uhrenholt. 2016. Exploring visualisation of channel activity, levels and EQ for user interfaces implementing the stage metaphor for music mixing. In 2nd AES Workshop on Intelligent Music Production.

[15]

Rolf Inge Godøy. 2010. Gestural affordances of musical sound. In Musical Gestures: Sound, Movement, and Meaning, Rolf Inge Godøy and Marc Leman (Eds.). Routledge, New York, New York, 115--137.

[16]

Rolf Inge Godøy, Egil Haga, and Alexander Refsum Jensenius. 2006. Exploring Music-Related Gestures by Sound-Tracing: A Preliminary Study. In Proc. of the 2nd International Symposium on Gesture Interfaces for Multimedia Systems (GIMS2006). Leeds, United Kingdom.

[17]

Sukeshini A Grandhi, Gina Joue, and Irene Mittelberg. 2011. Understanding naturalness and intuitiveness in gesture production. In Proc. of the SIGCHI Conference on Human Factors in Computing Systems (CHI). Vancouver, BC, Canada, 821--824.

Digital Library

[18]

Rex Hartson. 2003. Cognitive, physical, sensory, and functional affordances in interaction design. Behaviour & Information Technology 22, 5 (2003), 315--338. https://doi.org/10.1080/01449290310001592587

[19]

Andrew Hugill. 2019. Aural Diversity. In Given at the Equality, Diversity and Inclusion in Music Higher Education conference. City University, London, United Kingdom.

[20]

Takayuki Iwamoto, Mari Tatezono, and Hiroyuki Shinoda. 2008. Non-contact method for producing tactile sensation using airborne ultrasound. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 504--513.

Digital Library

[21]

Michelangelo Lupone, Silvia Lanzalone, Alessio Gabriele, and Martina De Luca. 2015. Forme Immateriali by Michelangelo Lupone: structure, creation, interaction, evolution of a permanent adaptive music work. In Proceedings of the Electroacoustic Winds 2015 Conference. Aveiro, Portugal.

[22]

Donald A Norman and Stephen W Draper. 1986. User Centered System Design: New Perspectives on Human-computer Interaction. Lawrence Erlbaum Associates, Hillsdale, New Jersey.

Digital Library

[23]

Ian O'connell and James Rock. 2011. Projection apparatus and method for pepper's ghost illusion. (Feb. 8 2011). US Patent 7,883,212.

[24]

Lidia Oshlyansky, Harold Thimbleby, and Paul Cairns. 2004. Breaking Affordance: Culture as Context. In Proceedings of the Third Nordic Conference on Human-Computer Interaction (NordiCHI '04). Association for Computing Machinery, New York, NY, USA, 81--84. https://doi.org/10.1145/1028014.1028025

Digital Library

[25]

Stefano Papetti and Charalampos Saitis. 2018. Musical Haptics. Springer, Cham.

[26]

Linda Pring, Katherine Woolf, and Valerie Tadic. 2008. Melody and Pitch Processing in Five Musical Savants with Congenital Blindness. Perception 37, 2 (2008), 290--307. https://doi.org/10.1068/p5718

[27]

Davide Rocchesso, Stefania Serafin, Frauke Behrendt, Nicola Bernardini, Roberto Bresin, Gerhard Eckel, Karmen Franinovic, Thomas Hermann, Sandra Pauletto, Patrick Susini, and Yon Visell. 2008. Sonic Interaction Design: Sound, Information and Experience. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (CHI EA '08). Association for Computing Machinery, New York, NY, USA, 3969--3972. https://doi.org/10.1145/1358628.1358969

[28]

Charalampos Saitis, Hanna Järveläinen, and Claudia Fritz. 2018. The Role of Haptic Cues in Musical Instrument Quality Perception. Springer International Publishing, Cham, 73--93. https://doi.org/10.1007/978-3-319-58316-7_5

[29]

Suranga Chandima Sanayakkara, Lonce Wyse, S. H. Ong, and Elizabeth A. Taylor. 2013. Enhancing Musical Experience for the Hearing-Impaired Using Visual and Haptic Displays. Human-Computer Interaction 28, 2 (2013), 115--160. https://doi.org/10.1080/07370024.2012.697006

[30]

Michael Schutz and Scott Lipscomb. 2007. Hearing Gestures, Seeing Music: Vision Influences Perceived Tone Duration. Perception 36, 6 (2007), 888--897. https://doi.org/10.1068/p5635

[31]

B.R. Shelton and C.L Searle. 1980. The influence of vision on the absolute identification of sound-source position. Perception & Psychophysics 28, 6 (1980), 589--596. https://doi.org/10.3758/BF03198830

[32]

Dag Svanæs. 2000. Understanding interactivity: steps to a phenomenology of human-computer interaction. Norges teknisk-naturvitenskapelige universitet.

[33]

Atau Tanaka, Alessandro Altavilla, and Neil Spowage. 2012. Gestural Musical Affordances. In Proceedings of the 9th International Conference on Sound and Music Computing. Copenhangen, Denmark.

[34]

Atau Tanaka and Adam Parkinson. 2016. Haptic Wave: A Cross-Modal Interface for Visually Impaired Audio Producers. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). Association for Computing Machinery, New York, NY, USA, 2150--2161. https://doi.org/10.1145/2858036.2858304

Digital Library

[35]

Oliver Vitouch, Sandra Sovdat, and Norman Holler. 2006. Audio-vision: Visual input drives perceived music tempo. In Proceedings of the 9th International Conference on Music Perception & Cognition. Bologna, Italy, 86--89.

[36]

Mads Walther-Hansen. 2017. New and Old User Interface Metaphors in Music Production. Journal on the Art of Record Production (JARP) 11 (2017).

Cited By

Davis TPocius KCusson VGentili-Morin MPasquier P(2024)Embodied eTube Gestures and AgencyProceedings of the 9th International Conference on Movement and Computing10.1145/3658852.3659084(1-6)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3658852.3659084
Jeong JKim DJeong J(2024)EchoTap: Non-Verbal Sound Interaction with Knock and Tap GesturesInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2348837(1-22)Online publication date: 3-Jun-2024
https://doi.org/10.1080/10447318.2024.2348837
Karpodini C(2023)Evaluating Haptic Technology in Accessibility of Digital Audio Workstations for Visually Impaired CreativesACM SIGACCESS Accessibility and Computing10.1145/3584732.3584738(1-1)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1145/3584732.3584738
Show More Cited By

Index Terms

Human-Sound Interaction: Towards a Human-Centred Sonic Interaction Design approach
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction devices
      1. Haptic devices
      2. Sound-based input / output

Recommendations

Form Follows Sound: Designing Interactions from Sonic Memories
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Sonic interaction is the continuous relationship between user actions and sound, mediated by some technology. Because interaction with sound may be task oriented or experience-based it is important to understand the nature of action-sound relationships ...
Music and Human-Computer Interaction
Robot-human interaction with an anthropomorphic percussionist
CHI '06: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

The paper presents our approach for human-machine interaction with an anthropomorphic mechanical percussionist that can listen to live players, analyze perceptual musical aspects in real-time, and use the product of this analysis to play along in a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

MOCO '20: Proceedings of the 7th International Conference on Movement and Computing

July 2020

205 pages

ISBN:9781450375054

DOI:10.1145/3401956

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

Rutgers University: Rutgers University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

MOCO '20

MOCO '20: 7th International Conference on Movement and Computing

July 15 - 17, 2020

NJ, Jersey City/Virtual, USA

Acceptance Rates

Overall Acceptance Rate 85 of 185 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
412
Total Downloads

Downloads (Last 12 months)59
Downloads (Last 6 weeks)7

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Davis TPocius KCusson VGentili-Morin MPasquier P(2024)Embodied eTube Gestures and AgencyProceedings of the 9th International Conference on Movement and Computing10.1145/3658852.3659084(1-6)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3658852.3659084
Jeong JKim DJeong J(2024)EchoTap: Non-Verbal Sound Interaction with Knock and Tap GesturesInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2348837(1-22)Online publication date: 3-Jun-2024
https://doi.org/10.1080/10447318.2024.2348837
Karpodini C(2023)Evaluating Haptic Technology in Accessibility of Digital Audio Workstations for Visually Impaired CreativesACM SIGACCESS Accessibility and Computing10.1145/3584732.3584738(1-1)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1145/3584732.3584738
Karpodini C(2022)Evaluating haptic technology in accessibility of digital audio workstations for visual impaired creatives.Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3550414(1-5)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3550414
Olpindo JVan Nort D(2022)Towards Inclusive and Interactive Spaces for BreakdancingArtsIT, Interactivity and Game Creation10.1007/978-3-030-95531-1_14(189-206)Online publication date: 10-Feb-2022
https://doi.org/10.1007/978-3-030-95531-1_14

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents