demonstration

Phantom Undulations: Remote Physiological Sensing in Abstract Installation Works

Authors:

Dingding Zheng,

Kirill Ragozin,

George Chernyshov,

Kouta MinamizawaAuthors Info & Claims

AHs '23: Proceedings of the Augmented Humans International Conference 2023

Pages 367 - 370

https://doi.org/10.1145/3582700.3583710

Published: 14 March 2023 Publication History

Abstract

Phantom Undulations is a mixed-media work in which an artist’s physiological data is being used remotely to manipulate the sounds and visuals of an abstract artifact in a gallery setting. This work relies heavily on the concept of showing the artist’s presence or liveness in an abstract and remote manner through changes in the harmony, rhythm, and timbre of a loosely structured soundscape as well as the physical appearance of the artifact. We propose a method of utilizing real time physiological sensing data through a custom built sensing wristband and accompanying software. This system reads the physiological data of the artist and sends it to the Internet, where it can be received by the artifact anywhere on Earth. In addition to the artist’s physiological data, we also offer a way for the audience to incorporate their own data into the work via several sensing wristbands which will accompany the artifact. Through this collaborative process, we wish to invite the audience to join the artist in manipulating the sonic and visual characteristics of this artifact and create a contrapuntally fluid and responsive musical experience.

References

[1]

John L Andreassi. 2010. Psychophysiology: Human behavior and physiological response. Psychology Press.

[2]

Mathias Benedek and Christian Kaernbach. 2010. A continuous measure of phasic electrodermal activity. Journal of neuroscience methods 190, 1 (2010), 80–91.

[3]

Wolfram Boucsein. 2012. Electrodermal activity: Springer Science & Business Media. Broek, EL vd, Schut, MH, Westerink, JHDM, Herk, J. v., & Tuinenbreijer, K (2012).

[4]

Russell Burton. 2015. The elements of music: what are they, and who cares. In Music: Educating for Life. ASME XXth National Conference Proceedings. Australian Society for Music Education Parkvill, VIC, Australia, 22–28.

[5]

Kwang-Ho Choi, Junbeom Kim, O Sang Kwon, Min Ji Kim, Yeon Hee Ryu, and Ji-Eun Park. 2017. Is heart rate variability (HRV) an adequate tool for evaluating human emotions?–A focus on the use of the International Affective Picture System (IAPS). Psychiatry research 251(2017), 192–196.

[6]

Stefan Koelsch and Lutz Jäncke. 2015. Music and the heart. European heart journal 36, 44 (2015), 3043–3049.

[7]

Roland Uwe Krabs, Ronny Enk, Niels Teich, and Stefan Koelsch. 2015. Autonomic effects of music in health and Crohn’s disease: the impact of isochronicity, emotional valence, and tempo. PloS one 10, 5 (2015), e0126224.

[8]

Carol L Krumhansl. 1997. An exploratory study of musical emotions and psychophysiology.Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale 51, 4 (1997), 336.

[9]

Ernst Kurth and Kurth Ernst. 1991. Ernst Kurth: selected writings. Number 2. Cambridge University Press.

[10]

Peter J Lang, Mark K Greenwald, Margaret M Bradley, and Alfons O Hamm. 1993. Looking at pictures: Affective, facial, visceral, and behavioral reactions. Psychophysiology 30, 3 (1993), 261–273.

[11]

Qianqian Mu, George Chernyshov, Ziyue Wang, Danny Hynds, Dingding Zheng, Kouta Minamizawa, Dunya Chen, Atsuro Ueki, Masa Inakage, and Kai Kunze. 2022. ImageFlowing-Enhance Emotional Expression by Reproducing the Vital Signs of the Photographer. In ACM SIGGRAPH 2022 Emerging Technologies. 1–2.

Digital Library

[12]

Lutz Neugebauer and David Aldridge. 1998. Communication, heart rate and the musical dialogue. British Journal of Music Therapy 12, 2 (1998), 46–52.

[13]

Valorie N Salimpoor, Mitchel Benovoy, Gregory Longo, Jeremy R Cooperstock, and Robert J Zatorre. 2009. The rewarding aspects of music listening are related to degree of emotional arousal. PloS one 4, 10 (2009), e7487.

[14]

Fred Shaffer and Jay P Ginsberg. 2017. An overview of heart rate variability metrics and norms. Frontiers in public health(2017), 258.

[15]

Moe Sugawa, Taichi Furukawa, George Chernyshov, Danny Hynds, Jiawen Han, Marcelo Padovani, Dingding Zheng, Karola Marky, Kai Kunze, and Kouta Minamizawa. 2021. Boiling Mind: Amplifying the Audience-Performer Connection through Sonification and Visualization of Heart and Electrodermal Activities. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction. 1–10.

Digital Library

[16]

Marieke van Dooren, Joris H Janssen, 2012. Emotional sweating across the body: Comparing 16 different skin conductance measurement locations. Physiology & behavior 106, 2 (2012), 298–304.

[17]

Jonathan Weinel, Stuart Cunningham, Nathan Roberts, Shaun Roberts, and Darryl Griffiths. 2014. EEG as a controller for psychedelic visual music in an immersive dome environment. EVA London 2014: Electronic Visualisation & the Arts (2014).

Cited By

Sparrow LGalwey CLoveridge BGlasser SOsborne MKelly R(2024)Heart and Soul: The Ethics of Biometric Capture in Immersive Artistic PerformanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642309(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642309

Index Terms

Phantom Undulations: Remote Physiological Sensing in Abstract Installation Works
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
    2. Interactive systems and tools

Recommendations

Innermost Echoes: Integrating Real-Time Physiology into Live Music Performances
TEI '24: Proceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction

In this paper, we propose a method for utilizing musical artifacts and physiological data as a means for creating a new form of live music experience that is rooted in the physiology of the performers and audience members. By utilizing physiological data ...
Variations: an interactive musical sculpture
C&C '05: Proceedings of the 5th conference on Creativity & cognition

Variations is an ongoing exploration of multi-channel and interactive music experienced through a sculptural interface. The interface functions as a means by which participants can create their own mix of the music by interacting with elements of a ...
Learning Visual Programming by Creating a Walkable Interactive Installation
AM '15: Proceedings of the Audio Mostly 2015 on Interaction With Sound

In this paper we explore the question of how to teach visual programming languages, such as Max/MSP, Pure Data, vvvv, to experienced media and sound artists. A pedagogical challenge with these students is to keep them motivated after the initial phase ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

AHs '23: Proceedings of the Augmented Humans International Conference 2023

March 2023

395 pages

ISBN:9781450399845

DOI:10.1145/3582700

General Chairs:
Jamie A. Ward
Goldsmiths, University of London, United Kingdom
,
Mark McGill
University of Glasgow, United Kingdom
,
Karola Marky
Ruhr-University Bochum, Germany

Copyright © 2023 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 March 2023

Check for updates

Author Tags

Qualifiers

Demonstration
Research
Refereed limited

Conference

AHs '23

AHs '23: Augmented Humans Conference

March 12 - 14, 2023

Glasgow, United Kingdom

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
70
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)5

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sparrow LGalwey CLoveridge BGlasser SOsborne MKelly R(2024)Heart and Soul: The Ethics of Biometric Capture in Immersive Artistic PerformanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642309(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642309

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents