NEXUS: A Tangible Multi-User Sensor-Based Telematic Novel Mixing-Interface for Multimedial Exploration
Authors: 
Hannah Walter, Eric Larrieux, Robert Torche, Cedric Spindler, Patrick MüllerAuthors Info & Claims
Pages 245 - 259
Abstract
This paper explores the realm of telematic performance practice, interconnecting geographically disparate locations and performers through telecommunication technologies. Specifically, we focus on networked multimedia mixing and advance the field within the framework of third-wave human-computer interaction, emphasizing embodied interaction. Along with a detailed introduction of our Novel Interface for Multimedial Exploration, the NEXUS, we provide insights into our first use-case scenario, employing the NEXUS-NIME within a telematic performance context. Adapting and expanding on existing examples of dimension spaces, we introduce a novel Dimension Space for Phenomenologically Situated Interaction, which accounts for situated embodiment. It is employed in a dimension space analysis comparing the NEXUS with historical interfaces for networked musical and multimedial expression that incorporate multimodal interactions through tangible user interfaces. In doing so we demonstrate the NEXUS’s role as a transformative multi-user tool and bundle of NIMEs, shifting sound engineering practices from one of transparency and social fidelity to one of hypermediacy, risk and respons-ability. Drawing on concepts from feminist new materialism and posthumanism, the NEXUS opens new avenues for rendering-capable, for exploring complex interdependencies between human and non-human actors becoming-with in telematic ecosystems.
References
[1]
Bettina Bildhauer. 2021. Putting the Net Back into the Internet: The Materiality of Networks from a Premodern Perspective. Digital Philology: A Journal of Medieval Cultures 10, 1 (2021), 95–110. https://doi.org/10.1353/dph.2021.0004
[2]
Jay David Bolter and Richard Grusin. 1999. Remediation. Understanding New Media. MIT Press, Cambridge.
[3]
Albertus Jacobus Bongers. 2006. Interactivation. Towards an e-cology of people, our technological environment, and the arts. Ph.D. Dissertation. Vrije Universiteit Amsterdam.
[4]
Bert Bongers. 2000. Physical interfaces in the electronic arts. Interaction theory and interfacing techniques for real-time performance. In Trends in Gestural Control of Music, Marcelo M Wanderley and M Battier (Eds.). Ircam - Centre Pompidou, Paris, 41–70.
[5]
Bert Bongers. 2007. Electronic Musical Instruments: Experiences of a New Luthier. Leonardo Music Journal 17 (Dec. 2007), 9–16. https://doi.org/10.1162/lmj.2007.17.9
[6]
Jonas Braasch, Chris Chafe, Pauline Oliveros, and Doug Van Nort. 2009. Mixing-Console Design Considerations for Telematic Music Applications(127th). Rensselaer Polytechnic Institute, Troy, NY, USA, CCRMA, Stanford University, Stanford, CA, USA, 1–17. https://aes2.org/publications/elibrary-page/?id=15136
[7]
Hartmut Böhme. 2004. Einführung Netzwerke. Zur Theorie und Geschichte einer Konstruktion. In Netzwerke: eine Kulturtechnik der Moderne, Jürgen Barkhoff (Ed.). Number 29 in Literatur - Kultur - Geschlecht Große Reihe. Böhlau, Köln, 17–36.
[8]
Susanne Bødker. 2006. When second wave HCI meets third wave challenges, Vol. 189. 1–8. https://doi.org/10.1145/1182475.1182476
[9]
Juan P. Carrascal and Sergi Jordà. 2011. Multitouch Interface For Audio Mixing. In Proceedings of the International Conference on New Interfaces for Musical Expression. Oslo, Norway, 100–103. https://doi.org/10.5281/ZENODO.1177983
[10]
Vinciane Despret. 2004. The Body We Care for: Figures of Anthropo-zoo-genesis. Body & Society 10, 2-3 (June 2004), 111–134. https://doi.org/10.1177/1357034X04042938
[11]
Vinciane Despret. 2008. The Becomings of Subjectivity in Animal Worlds. Subjectivity 23, 1 (July 2008), 123–139. https://doi.org/10.1057/sub.2008.15
[12]
Paul Dourish. 2001. Where the Action Is: The Foundations of Embodied Interaction. The MIT Press. https://direct.mit.edu/books/book/3875/Where-the-Action-IsThe-Foundations-of-Embodied https://doi.org/10.7551/mitpress/7221.001.0001.
[13]
Kenneth P. Fishkin. 2004. A taxonomy for and analysis of tangible interfaces. Personal and Ubiquitous Computing 8, 5 (Sept. 2004). https://doi.org/10.1007/s00779-004-0297-4
[14]
Martin Fröhlich, Roman Haefeli, and Patrick Müller. 2024. Telemersive Toolkit: Exchange Multi Media Streams for Distributed Networked Performance Installations. In Proceedings of the 2024 ACM International Conference on Interactive Media Experiences(IMX ’24). Association for Computing Machinery, Stockholm, Sweden, 354–360. https://doi.org/10.1145/3639701.3663637
[15]
Alexander R. Galloway. 2004. Protocol: How Control Exists after Decentralization. The MIT Press. https://doi.org/10.7551/mitpress/5658.001.0001
[16]
Steven Gelineck, Morten Büchert, and Jesper Andersen. 2013. Towards a more flexible and creative music mixing interface(CHI EA ’13). Association for Computing Machinery, Paris, France, 733–738. https://doi.org/10.1145/2468356.2468487
[17]
David Gibson. 1997. The art of mixing: a visual guide to recording, engineering, and production. MixBooks, Vallejo, Calif.
[18]
Donna J. Haraway. 2016. Staying with the Trouble: Making Kin in the Chthulucene. Duke University Press, Durham and London. https://doi.org/10.1215/9780822373780
[19]
Steve Harrison, Phoebe Sengers, and Deborah Tatar. 2011. Making epistemological trouble: Third-paradigm HCI as successor science. Interacting with Computers 23, 5 (Sept. 2011), 385–392. https://doi.org/10.1016/j.intcom.2011.03.005
[20]
Steve Harrison, Deborah Tatar, and Phoebe Sengers. 2007. The Three Paradigms of HCI. In Alt.Chi. Session at the SIGCHI. CHI, San Jose, California, USA, 1–18.
[21]
Martin Heidegger. 1967. Sein und Zeit (11 ed.). Max Niemeyer Verlag, Tübingen. http://archive.org/details/HeideggerMartinSeinUndZeit
[22]
Sergi Jordà, Günter Geiger, Marcos Alonso, and Martin Kaltenbrunner. 2007. The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In Proceedings of the 1st international conference on Tangible and embedded interaction(TEI ’07). Association for Computing Machinery, New York, NY, USA, 139–146. https://doi.org/10.1145/1226969.1226998
[23]
Scott R. Klemmer, Björn Hartmann, and Leila Takayama. 2006. How bodies matter: five themes for interaction design. In Proceedings of the 6th conference on Designing Interactive systems. ACM, University Park PA USA, 140–149. https://doi.org/10.1145/1142405.1142429
[24]
Volker Krefeld and Michel Waisvisz. 1990. The Hand in the Web: An Interview with Michel Waisvisz. Computer Music Journal 14, 2 (1990), 28–33. https://doi.org/10.2307/3679709 Publisher: The MIT Press.
[25]
Sybille Krämer. 2015. Medium, Messenger, Transmission: An Approach to Media Philosophy. Amsterdam University Press, Amsterdam. https://www.jstor.org/stable/j.ctt19630z0
[26]
Eric Larrieux and Mélia Roger. 2023. The Air Listening Station: Bridging the gap between Sound Art and Sonification. In Proceedings of the 18th International Audio Mostly Conference. ACM, Edinburgh United Kingdom, 85–92. https://doi.org/10.1145/3616195.3616203
[27]
Noah Liebman, Michael Nagara, Jacek Spiewla, and Erin Zolkosky. 2010. Cuebert: A New Mixing Board Concept for Musical Theatre. In Proceedings of the International Conference on New Interfaces for Musical Expression. Sydney, Australia, 51–56. https://doi.org/10.5281/zenodo.1177833
[28]
Maurice Merleau-Ponty. 1945. Phénoménologie de la perception. Gallimard, Paris.
[29]
Maurice Merleau-Ponty. 2002. Phenomenology of perception: an introduction. Routledge, London. OCLC: 80428978.
[30]
Patrick Müller, Matthias Ziegler, and Johannes Schütt. 2019. Towards a Telematic Dimension Space. In Proceedings of the International Conference on New Interfaces for Musical Expression, Marcelo Queiroz and Anna Xambó Sedó (Eds.). UFRGS, Porto Alegre, Brazil, 393–400. https://doi.org/10.5281/zenodo.3673007
[31]
Pippa Norris. 2001. Digital Divide: Civic Engagement, Information Poverty, and the Internet Worldwide (1 ed.). Cambridge University Press. https://doi.org/10.1017/CBO9781139164887
[32]
Jeff Pressing. 1997. Some Perspectives on Performed Sound and Music in Virtual Environments. Presence: Teleoperators and Virtual Environments 6, 4 (Aug. 1997), 482–503. https://doi.org/10.1162/pres.1997.6.4.482
[33]
Yvonne Rogers. 2012. HCI Theory: Classical, Modern, and Contemporary. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-031-02197-8
[34]
Whitney Jesse Slaten. 2018. Doing Sound: An Ethnography of Fidelity, Temporality, and Labor Among Live Sound Engineers. Ph.D. Dissertation. Columbia University, New York. https://doi.org/10.7916/D87S94VD
[35]
Lucy Suchman. 2006. Human-Machine Reconfigurations: Plans and Situated Actions (2 ed.). Cambridge University Press, Cambridge. https://doi.org/10.1017/CBO9780511808418
[36]
Dag Svanæs. 2013. Interaction design for and with the lived body: Some implications of merleau-ponty’s phenomenology. ACM Trans. Comput.-Hum. Interact. 20, 1 (April 2013), 8:1–8:30. https://doi.org/10.1145/2442106.2442114
[37]
Atau Tanaka. 2000. Musical Performance Practice on Sensor-based Instruments. In Trends in Gestural Control of Music, Marcelo M Wanderley and M Battier (Eds.). Ircam - Centre Pompidou, Paris, 389–406.
[38]
HannaH Vyborg Walter, Robert Torche, Cedric Spindler, Eric Larrieux, and Patrick Müller. 2024. kompost*klang*küche*: Telematic WORLDing with Glitch. Journal of Network Music and Arts 6, 1 (July 2024), 1–26. https://commons.library.stonybrook.edu/jonma/vol6/iss1/4
[39]
Anna Xambó. 2017. Embodied Music Interaction: Creative Design Synergies Between Music Performance and HCI. In Digital bodies: creativity and technology in the arts and humanities, Susan Broadhurst and Sara Price (Eds.). Palgrave Macmillan, London, 207–220. OCLC: ocn981117201.
[40]
Anna Xambó Sedó. 2014. Tabletop Tangible Interfaces for Music Performance: Design and Evaluation. Ph.D. Dissertation. The Open University. https://oro.open.ac.uk/42473/
[41]
Dominik Ziliotis and Christiane Reiser. 2015. Livingdome. FORUM für Kunsttherapien 2, Häuser und Räume (2015), 6–8. https://www.gpk.ch/angebot/magazin.
Index Terms
- NEXUS: A Tangible Multi-User Sensor-Based Telematic Novel Mixing-Interface for Multimedial Exploration
Recommendations
Embodied metaphors in tangible interaction design
For centuries, learning and development has been supported by physical activity and manipulating physical objects. With the introduction of embedded technologies, opportunities for employing tangible or embodied interaction for learning and development ...
Zound: An interactive live electronics
AM '17: Proceedings of the 12th International Audio Mostly Conference on Augmented and Participatory Sound and Music ExperiencesAn interactive live electronics, completely controlled real-time by the mouth harp's (Jew's harp, Jaw harp) acoustic signal has been created, using Max (™ Cycling '74). The system interacts according to the various timbre's features of the instrument, ...
Tangible bots: interaction with active tangibles in tabletop interfaces
CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsWe present interaction techniques for tangible tabletop interfaces that use active, motorized tangibles, what we call Tangible Bots. Tangible Bots can reflect changes in the digital model and assist users by haptic feedback, by correcting errors, by ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2024
565 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 September 2024
Check for updates
Author Tags
- Human-Computer Interaction
- New Interfaces for Musical Expression
- Telematic performance
- digital augmentation
- digital musical instrument
- dimension space analysis
- embedded systems
- embodied interaction
- extended reality
- feminist new materialism.
- mixed reality
- networked multimedia mixing
- posthumanism
- responsive environments
- sonic interaction design
- tangible user interfaces
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Swiss National Science Foundation
Conference
AM '24
Acceptance Rates
Overall Acceptance Rate 177 of 275 submissions, 64%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 120Total Downloads
- Downloads (Last 12 months)120
- Downloads (Last 6 weeks)41
Reflects downloads up to 14 Feb 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in