research-article

Intelligent Music Interfaces: When Interactive Assistance and Augmentation Meet Musical Instruments

Authors:

Jordan Aiko Deja,

Snehesh Shrestha,

Matthias Hoppe,

Andrii Matviienko,

Karola MarkyAuthors Info & Claims

AHs '23: Proceedings of the Augmented Humans International Conference 2023

Pages 379 - 383

https://doi.org/10.1145/3582700.3582731

Published: 14 March 2023 Publication History

Abstract

The interactive augmentation of musical instruments to foster self-expressiveness and learning has a rich history. Over the past decades, the incorporation of interactive technologies into musical instruments emerged into a new research field requiring strong collaboration between different disciplines. The workshop "Intelligent Music Interfaces" covers a wide range of musical research subjects and directions, including (a) current challenges in musical learning, (b) prototyping for improvements, (c) new means of musical expression, and (d) evaluation of the solutions.

References

[1]

Jordan Aiko Deja, Nuwan T Attygale, Klen Čopič Pucihar, and Matjaž Kljun. 2020. Sound 2121: Cross-Reality Transitions Between Real and Augmented Sound Landscapes. In International Workshop on Cross-Reality (XR) Interaction @ ACM ISS 2020. CEUR Workshop Proceedings, Virtual and Lisbon Portugal, 1–4.

[2]

Jordan Aiko Deja, Sven Mayer, Klen Čopič Pucihar, and Matjaž Kljun. 2022. A Survey of Augmented Piano Prototypes: Has Augmentation Improved Learning Experiences?Proceedings of the ACM on Human-Computer Interaction 6, ISS(2022), 226–253.

[3]

Christiane Glatz, Jonas Ditz, Thomas Kosch, Albrecht Schmidt, Marie Lahmer, and Lewis L. Chuang. 2017. Reading the Mobile Brain: From Laboratory to Real-World Electroencephalography. In Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia(Stuttgart, Germany) (MUM ’17). Association for Computing Machinery, New York, NY, USA, 573–579. https://doi.org/10.1145/3152832.3156560

Digital Library

[4]

Jiawen Han, George Chernyshov, Moe Sugawa, Dingding Zheng, Danny Hynds, Taichi Furukawa, Marcelo Padovani, Kouta Minamizawa, Karola Marky, Jamie A Ward, and Kai Kunze. 2022. Linking Audience Physiology to Choreography. ACM Trans. Comput.-Hum. Interact. 29 (aug 2022), 1–32. https://doi.org/10.1145/3557887 Just Accepted.

Digital Library

[5]

Jacob Harrison, Alan Chamberlain, and Andrew P. McPherson. 2019. Accessible Instruments in the Wild: Engaging with a Community of Learning-Disabled Musicians. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems(CHI EA ’19). ACM, New York, NY, USA, Article LBW0247, 6 pages. https://doi.org/10.1145/3290607.3313037

Digital Library

[6]

Jakob Karolus, Annika Kilian, Thomas Kosch, Albrecht Schmidt, and Paweł W. Wozniak. 2020. Hit the Thumb Jack! Using Electromyography to Augment the Piano Keyboard. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (Eindhoven, Netherlands) (DIS ’20). Association for Computing Machinery, New York, NY, USA, 429–440. https://doi.org/10.1145/3357236.3395500

Digital Library

[7]

Jakob Karolus, Hendrik Schuff, Thomas Kosch, Paweł W. Wozniak, and Albrecht Schmidt. 2018. EMGuitar: Assisting Guitar Playing with Electromyography. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 651–655. https://doi.org/10.1145/3196709.3196803

Digital Library

[8]

Joseph R. Keebler, Travis J. Wiltshire, Dustin C. Smith, and Stephen M. Fiore. 2013. Picking Up STEAM: Educational Implications for Teaching With an Augmented Reality Guitar Learning System. In Proceedings of the International Conference on Virtual, Augmented and Mixed Reality(VAMR). Springer, Cham, Switzerland, 170–178. https://doi.org/10.1007/978-3-642-39420-1_19

[9]

Chutisant Kerdvibulvech and Hideo Saito. 2007. Real-Time Guitar Chord Estimation by Stereo Cameras for Supporting Guitarists. In Proceedings of the 10th International Workshop on Advanced Image Technology(IWAIT). The Institute of Electronics, Information and Communication Engineers, Bangkok, Thailand, 256–261.

[10]

Annika Kilian, Jakob Karolus, Thomas Kosch, Albrecht Schmidt, and Paweł W. Paweł. 2021. EMPiano: Electromyographic Pitch Control on the Piano Keyboard. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, Article 196, 4 pages. https://doi.org/10.1145/3411763.3451556

Digital Library

[11]

Markus Löchtefeld, Sven Gehring, Ralf Jung, and Antonio Krüger. 2011. GuitAR: Supporting Guitar Learning through Mobile Projection. In CHI ’11 Extended Abstracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI EA ’11). Association for Computing Machinery, New York, NY, USA, 1447–1452. https://doi.org/10.1145/1979742.1979789

Digital Library

[12]

Andreas Löcken, Mark Colley, Andrii Matviienko, Kai Holländer, Debargha Dey, Azra Habibovic, Andrew L Kun, Susanne Boll, and Andreas Riener. 2021. WeCARe: Workshop on Inclusive Communication between Automated Vehicles and Vulnerable Road Users. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (Oldenburg, Germany) (MobileHCI ’20). Association for Computing Machinery, New York, NY, USA, Article 43, 5 pages. https://doi.org/10.1145/3406324.3424587

Digital Library

[13]

Andreas Löcken, Andrii Matviienko, Mark Colley, Debargha Dey, Azra Habibovic, Yee Mun Lee, and Andreas Riener. 2022. Accessible Automated Automotive Workshop Series (A3WS): International Perspective on Inclusive External Human-Machine Interfaces. In Adjunct Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Seoul, Republic of Korea) (AutomotiveUI ’22). Association for Computing Machinery, New York, NY, USA, 192–195. https://doi.org/10.1145/3544999.3551347

Digital Library

[14]

Karola Marky, Julian Fischer, Max Mühlhäuser, and Andrii Matviienko. 2021. Investigating Page Turning Methods for Sheet Music during Piano Play. In Adjunct Publication of the 23rd International Conference on Mobile Human-Computer Interaction(Toulouse & Virtual, France) (MobileHCI ’21). Association for Computing Machinery, New York, NY, USA, Article 18, 6 pages. https://doi.org/10.1145/3447527.3474863

Digital Library

[15]

Karola Marky, Annika Kilian, Andreas Weiß, Jakob Karolus, Matthias Hoppe, Pawel W. Wozniak, Max Mühlhäuser, and Thomas Kosch. 2022. Intelligent Music Interfaces: When Interactive Assistance and Augmentation Meet Musical Instruments. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 84, 4 pages. https://doi.org/10.1145/3491101.3503743

Digital Library

[16]

Karola Marky, Andreas Weiß, Andrii Matviienko, Florian Brandherm, Sebastian Wolf, Martin Schmitz, Florian Krell, Florian Müller, Max Mühlhäuser, and Thomas Kosch. 2021. Let’s Frets! Assisting Guitar Students During Practice via Capacitive Sensing. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 746, 12 pages. https://doi.org/10.1145/3411764.3445595

Digital Library

[17]

Karola Marky, Andreas Weiß, Florian Müller, Martin Schmitz, Max Mühlhäuser, and Thomas Kosch. 2021. Let’s Frets! Mastering Guitar Playing with Capacitive Sensing and Visual Guidance. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, Article 169, 4 pages. https://doi.org/10.1145/3411763.3451536

Digital Library

[18]

Andrii Matviienko, Wilko Heuten, Alan Dix, and Susanne CJ Boll. 2021. Interactive Technology for Cycling — Ideate, Make – Remote, Together. In Adjunct Publication of the 23rd International Conference on Mobile Human-Computer Interaction(Toulouse & Virtual, France) (MobileHCI ’21). Association for Computing Machinery, New York, NY, USA, Article 29, 4 pages. https://doi.org/10.1145/3447527.3474870

Digital Library

[19]

Andrew P. McPherson, Adrian Gierakowski, and Adam M. Stark. 2013. The Space between the Notes: Adding Expressive Pitch Control to the Piano Keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 2195–2204. https://doi.org/10.1145/2470654.2481302

Digital Library

[20]

Florian Müller, Dirk Schnelle-Walka, Tobias Grosse-Puppendahl, Sebastian Günther, Markus Funk, Kris Luyten, Oliver Brdiczka, Niloofar Dezfuli, and Max Mühlhäuser. 2018. SmartObjects: Sixth Workshop on Interacting with Smart Objects. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3170427.3170606

Digital Library

[21]

Katja Rogers, Amrei Röhlig, Matthias Weing, Jan Gugenheimer, Bastian Könings, Melina Klepsch, Florian Schaub, Enrico Rukzio, Tina Seufert, and Michael Weber. 2014. P.I.A.N.O.: Faster Piano Learning with Interactive Projection. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces (Dresden, Germany) (ITS ’14). Association for Computing Machinery, New York, NY, USA, 149–158. https://doi.org/10.1145/2669485.2669514

Digital Library

[22]

Hatice Sahin, Heiko Mueller, Shadan Sadeghian, Debargha Dey, Andreas Löcken, Andrii Matviienko, Mark Colley, Azra Habibovic, and Philipp Wintersberger. 2021. Workshop on Prosocial Behavior in Future Mixed Traffic. In 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Leeds, United Kingdom) (AutomotiveUI ’21 Adjunct). Association for Computing Machinery, New York, NY, USA, 167–170. https://doi.org/10.1145/3473682.3477438

Digital Library

[23]

Gian-Luca Savino, Tamara von Sawitzky, Andrii Matviienko, Miriam Sturdee, Paweł W. Woźniak, Markus Löchtefeld, Andrew L. Kun, Andreas Riener, and Jonna Häkkilä. 2021. Cycling@CHI: Towards a Research Agenda for HCI in the Bike Lane. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, Article 107, 5 pages. https://doi.org/10.1145/3411763.3441316

Digital Library

[24]

Martin Schmitz, Sebastian Günther, Karola Marky, Florian Müller, Andrii Matviienko, Alexandra Voit, Roberts Marky, Max Mühlhäuser, and Thomas Kosch. 2022. Rethinking Smart Objects: The International Workshop on Interacting with Smart Objects in Interactive Spaces. In Companion Proceedings of the 2022 Conference on Interactive Surfaces and Spaces (Wellington, New Zealand) (ISS ’22). Association for Computing Machinery, New York, NY, USA, 64–67. https://doi.org/10.1145/3532104.3571470

Digital Library

[25]

Snehesh Shrestha, Cornelia Fermüller, Tianyu Huang, Pyone Thant Win, Adam Zukerman, Chethan M Parameshwara, and Yiannis Aloimonos. 2022. AIMusicGuru: Music Assisted Human Pose Correction.

[26]

Snehesh Shrestha, William Sentosatio, Huiashu Peng, Cornelia Fermuller, and Yiannis Aloimonos. 2023. FEVA: Fast Event Video Annotation Tool.

[27]

Tamara von Sawitzky, Philipp Wintersberger, Andrii Matviienko, Andreas Löcken, Andreas Riener, and Florian Michahelles. 2021. Workshop on Intelligent Cyclist Support Systems and Applications. In Mensch und Computer 2021 - Workshopband, Carolin Wienrich, Philipp Wintersberger, and Benjamin Weyers (Eds.). Gesellschaft für Informatik e.V., Bonn, 1–3. https://doi.org/10.18420/muc2021-mci-ws12-122

Cited By

Kosch TWeiß ADeja JShrestha SHoppe MMatviienko AMarky K(2024)Intelligent Music Interfaces: When Interactive Assistance and Adaptive Augmentation Meet Musical InstrumentsProceedings of the Augmented Humans International Conference 202410.1145/3652920.3653039(327-330)Online publication date: 4-Apr-2024
https://dl.acm.org/doi/10.1145/3652920.3653039
Probst KOckiya D(2024)To Touch, or Not to Touch: Evaluating Manual Page Turning Modalities for Digital Sheet Music During Piano PlayAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680242(1-7)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680242

Index Terms

Intelligent Music Interfaces: When Interactive Assistance and Augmentation Meet Musical Instruments
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Intelligent Music Interfaces: When Interactive Assistance and Augmentation Meet Musical Instruments
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

The interactive augmentation of musical instruments to foster self-expressiveness and learning has a rich history. Over the past decades, the incorporation of interactive technologies into musical instruments emerged into a new research field requiring ...
Intelligent Music Interfaces: When Interactive Assistance and Adaptive Augmentation Meet Musical Instruments
AHs '24: Proceedings of the Augmented Humans International Conference 2024

The interactive augmentation of musical instruments to foster self-expression and learning has a rich history. Over the past decades, incorporating interactive technologies into musical instruments has emerged as a research field requiring strong ...
Post-DMI musical Instruments
AM '16: Proceedings of the Audio Mostly 2016

What are the particularities of developing contemporary instruments? This text points out how the authors observed both a focal shift and persisting elements in the design of contemporary digital musical instruments (DMI's). Based on observations around ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

AHs '23: Proceedings of the Augmented Humans International Conference 2023

March 2023

395 pages

ISBN:9781450399845

DOI:10.1145/3582700

General Chairs:
Jamie A. Ward
Goldsmiths, University of London, United Kingdom
,
Mark McGill
University of Glasgow, United Kingdom
,
Karola Marky
Ruhr-University Bochum, Germany

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 March 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AHs '23

AHs '23: Augmented Humans Conference

March 12 - 14, 2023

Glasgow, United Kingdom

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
133
Total Downloads

Downloads (Last 12 months)62
Downloads (Last 6 weeks)2

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kosch TWeiß ADeja JShrestha SHoppe MMatviienko AMarky K(2024)Intelligent Music Interfaces: When Interactive Assistance and Adaptive Augmentation Meet Musical InstrumentsProceedings of the Augmented Humans International Conference 202410.1145/3652920.3653039(327-330)Online publication date: 4-Apr-2024
https://dl.acm.org/doi/10.1145/3652920.3653039
Probst KOckiya D(2024)To Touch, or Not to Touch: Evaluating Manual Page Turning Modalities for Digital Sheet Music During Piano PlayAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680242(1-7)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680242

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents