research-article

Musical Haptic Wearables for Synchronisation of Visually-impaired Performers: a Co-design Approach

Authors:

Tony StockmanAuthors Info & Claims

IMX '21: Proceedings of the 2021 ACM International Conference on Interactive Media Experiences

Pages 20 - 27

https://doi.org/10.1145/3452918.3458803

Published: 23 June 2021 Publication History

Abstract

The emergence of new technologies is providing opportunities to develop novel solutions that facilitate the integration of visually-impaired people in different activities of our daily life, including collective music making. This paper presents a study conducted with visually-impaired music performers, which involved a participatory approach to the design of accessible technologies for musical communication in group playing. We report on three workshops that were conducted together with members of an established ensemble of solely visually-impaired musicians. The first workshop focused on the identification of the participants’ needs during the activity of playing in groups and how technology could satisfy such needs. The second and third workshops investigated, respectively, the activities of choir singing and instrument playing in ensemble, focusing on the key issue of synchronisation that was identified in the first workshop. The workshops involved prototypes of musical haptic wearables, which were co-designed and evaluated by the participants. Overall, results indicate that wireless tactile communication represents a promising avenue to cater effectively to the needs of visually-impaired performers.

References

[1]

T. Asakawa and N. Kawarazaki. 2012. An electric music baton system using a haptic interface for visually disabled persons. In Proceedings of the SICE Annual Conference. IEEE, 602–607.

[2]

J. Bajo, M. A. Sánchez, V. Alonso, R. Berjón, J. A. Fraile, and J. M. Corchado. 2010. A distributed architecture for facilitating the integration of blind musicians in symphonic orchestras. Expert Systems with Applications 37, 12 (2010), 8508–8515.

Digital Library

[3]

D. Baker, A. Fomukong-Boden, and S. Edwards. 2019. “Don’t follow them, look at me!”: Contemplating a haptic digital prototype to bridge the conductor and visually impaired performer. Music Education Research 21, 3 (2019), 295–314.

[4]

D. Baker and L. Green. 2017. Insights in Sound: Visually Impaired Musicians’ Lives and Learning. Taylor & Francis.

[5]

R. Battello, L. Comanducci, F. Antonacci, A. Sarti, S. Delle Monache, G. Cospito, E. Pietrocola, and F. Berbenni. 2020. An Adaptive Metronome Technique for Mitigating the Impact of Latency in Networked Music Performances. In 2020 27th Conference of Open Innovations Association (FRUCT). IEEE, 10–17.

[6]

A. Bouwer, S. Holland, and M. Dalgleish. 2013. The Haptic Bracelets: learning multi-limb rhythm skills from haptic stimuli while reading. In Music and human-computer interaction. Springer, 101–122.

[7]

V. Braun and V. Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77–101.

[8]

J. Brooke. 1996. SUS–A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4–7.

[9]

J.R. Cauchard, J.L. Cheng, T. Pietrzak, and J.A. Landay. 2016. ActiVibe: design and evaluation of vibrations for progress monitoring. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 3261–3271.

[10]

L. Gabrielli and S. Squartini. 2016. Wireless Networked Music Performance. Springer. https://doi.org/10.1007/978-981-10-0335-6_5

[11]

C. Galea and C. Porter. 2018. Accessible choral ensembles for visually impaired singers. In Proceedings of the 32nd International BCS Human Computer Interaction Conference. 1–10.

[12]

M. Giordano and M.M. Wanderley. 2015. Follow the tactile metronome: Vibrotactile stimulation for tempo synchronization in music performance. In Proceedings of Sound and Music Computing Conference.

[13]

F. Goltz. 2018. Ableton Link–A technology to synchronize music software. In Linux Audio Conference. 39–42.

[14]

E. Gunther and S. O’Modhrain. 2003. Cutaneous grooves: composing for the sense of touch. Journal of New Music Research 32, 4 (2003), 369–381. https://doi.org/10.1076/jnmr.32.4.369.18856

[15]

L. Hayes and C. Michalakos. 2012. Imposing a networked vibrotactile communication system for improvisational suggestion. Organised Sound 17, 01 (2012), 36–44.

Digital Library

[16]

S. Holland, A. Bouwer, and O. Hödl. 2018. Haptics for the development of fundamental rhythm skills, including multi-limb coordination. In Musical Haptics. Springer Series on Touch and Haptic Systems, S. Papettiand C. Saitis (Eds.). Springer International Publishing.

[17]

R. Hupke, L. Beyer, M. Nophut, S. Preihs, and J. Peissig. 2019. Effect of a Global Metronome on Ensemble Accuracy in Networked Music Performance. In Audio Engineering Society Convention 147. Audio Engineering Society.

[18]

N. Kawarazaki, Y. Kaneishi, N. Saito, and T. Asakawa. 2014. A Supporting System of Choral Singing for Visually Impaired Persons Using Depth Image Sensor. Journal of Robotics and Mechatronics 26, 6 (2014), 735–742.

[19]

M.T. Marshall and M.M. Wanderley. 2006. Vibrotactile feedback in digital musical instruments. In Proceedings of the Conference on New Interfaces for Musical Expression. 226–229.

[20]

A. McPherson and V. Zappi. 2015. An Environment for Submillisecond-Latency Audio and Sensor Processing on BeagleBone Black. In Audio Engineering Society Convention 138. Audio Engineering Society. http://www.aes.org/e-lib/browse.cfm?elib=17755

[21]

S. Merchel and M. E. Altinsoy. 2018. Auditory-Tactile Experience of Music. In Musical Haptics, S. Papetti and C. Saitis (Eds.). Springer, 123–148. https://doi.org/10.1007/978-3-319-58316-7_7

[22]

O. Metatla, F. Martin, A. Parkinson, N. Bryan-Kinns, T. Stockman, and A. Tanaka. 2016. Audio-haptic interfaces for digital audio workstations. Journal on Multimodal User Interfaces 10, 3 (2016), 247–258.

[23]

T. Michailidis. 2016. On the hunt for feedback: vibrotactile feedback in interactive electronic music performances. Ph.D. Dissertation. Birmingham City University.

[24]

S. Papetti and C. Saitis (Eds.). 2018. Musical Haptics. Springer, Cham. https://doi.org/10.1007/978-3-319-58316-7

[25]

C. Rottondi, C. Chafe, C. Allocchio, and A. Sarti. 2016. An Overview on Networked Music Performance Technologies. IEEE Access 4(2016), 8823–8843. https://doi.org/10.1109/ACCESS.2016.2628440

[26]

N. G. Sahib, T. Stockman, A. Tombros, and O. Metatla. 2013. Participatory design with blind users: a scenario-based approach. In IFIP Conference on Human-Computer Interaction. Springer, 685–701.

[27]

M. Schumacher, M. Giordano, M.M. Wanderley, and S. Ferguson. 2013. Vibrotactile notification for live electronics performance: A prototype system. In Proceedings of the International Symposium on Computer Music Multidisciplinary Research. 516–525.

[28]

A. Tanaka and A. Parkinson. 2016. Haptic wave: A cross-modal interface for visually impaired audio producers. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2150–2161.

[29]

L. Turchet and M. Barthet. 2019. Co-design of Musical Haptic Wearables for electronic music performer’s communication. IEEE Transactions on Human-Machine Systems 49, 2 (2019), 183–193. https://doi.org/10.1109/THMS.2018.2885408

[30]

L. Turchet, C. Fischione, G. Essl, D. Keller, and M. Barthet. 2018. Internet of Musical Things: Vision and Challenges. IEEE Access 6(2018), 61994–62017. https://doi.org/10.1109/ACCESS.2018.2872625

[31]

L. Turchet and E. Rinaldo. 2021 (accepted). Technical performance assessment of the Ableton Link protocol over Wi-Fi. Journal of the Audio Engineering Society(2021 (accepted)). https://doi.org/10.17743/jaes.2021.0014

[32]

L. Turchet, T. West, and M. M. Wanderley. 2020. Touching the audience: Musical Haptic Wearables for augmented and participatory live music performances. Journal of Personal and Ubiquitous Computing (2020), 1–21. https://doi.org/10.1007/s00779-020-01395-2

Digital Library

[33]

J. Van Der Linden, E. Schoonderwaldt, J. Bird, and R. Johnson. 2011. Musicjacket – combining motion capture and vibrotactile feedback to teach violin bowing. IEEE Transactions on Instrumentation and Measurement 60, 1(2011), 104–113.

[34]

B. Verplank, M. Gurevich, and M. V. Mathews. 2002. THE PLANK: Designing a simple haptic controller. In Proceedings of the International Conference on New Interfaces for Musical Expression. 33–36.

Cited By

Lu LCrispin CGirouard A(2024)"We Musicians Know How to Divide and Conquer": Exploring Multimodal Interactions To Improve Music Reading and Memorization for Blind and Low Vision LearnersProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675604(1-14)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675604
Ueda YWithana ASugiura Y(2024)Tactile Presentation of Orchestral Conductor's Motion Trajectory2024 IEEE/SICE International Symposium on System Integration (SII)10.1109/SII58957.2024.10417570(546-553)Online publication date: 8-Jan-2024
https://doi.org/10.1109/SII58957.2024.10417570
Nijs LTurchet L(2024)Internet of Musical Things for Children2024 IEEE 5th International Symposium on the Internet of Sounds (IS2)10.1109/IS262782.2024.10704214(1-9)Online publication date: 30-Sep-2024
https://doi.org/10.1109/IS262782.2024.10704214
Show More Cited By

Recommendations

Playing with Feeling: Exploring Vibrotactile Feedback and Aesthetic Experiences for Developing Haptic Wearables for Blind and Low Vision Music Learning
ASSETS '23: Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility

Musical haptic wearables (MHWs) that convey information through vibrotactile feedback holds the potential to support the music learning of a blind or low vision (BLV) music learner. Yet, it is unclear how these technologies can give functional support ...
How Blind and Visually Impaired Composers, Producers, and Songwriters Leverage and Adapt Music Technology
ASSETS '20: Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility

Today, music creation software and hardware are central to the workflow of most professional composers, producers, and songwriters. Music is an aural art form, but it is notated graphically, and highly visual mainstream technologies pose significant ...
Smart Mandolin and Musical Haptic Gilet: effects of vibro-tactile stimuli during live music performance
AM '19: Proceedings of the 14th International Audio Mostly Conference: A Journey in Sound

In this paper we investigate the role of haptic stimuli in affecting the perception of live music. We designed a study where a smart mandolin performer played live for audience members wearing a gilet-based musical haptic wearable, which provided vibro-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IMX '21: Proceedings of the 2021 ACM International Conference on Interactive Media Experiences

June 2021

331 pages

ISBN:9781450383899

DOI:10.1145/3452918

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IMX '21

Sponsor:

IMX '21: ACM International Conference on Interactive Media Experiences

June 21 - 23, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 69 of 245 submissions, 28%

Upcoming Conference

IMX '25

Sponsor:
sigchi

ACM International Conference on Interactive Media Experiences

June 3 - 6, 2025

Niter?i , Brazil

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
256
Total Downloads

Downloads (Last 12 months)72
Downloads (Last 6 weeks)5

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lu LCrispin CGirouard A(2024)"We Musicians Know How to Divide and Conquer": Exploring Multimodal Interactions To Improve Music Reading and Memorization for Blind and Low Vision LearnersProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675604(1-14)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675604
Ueda YWithana ASugiura Y(2024)Tactile Presentation of Orchestral Conductor's Motion Trajectory2024 IEEE/SICE International Symposium on System Integration (SII)10.1109/SII58957.2024.10417570(546-553)Online publication date: 8-Jan-2024
https://doi.org/10.1109/SII58957.2024.10417570
Nijs LTurchet L(2024)Internet of Musical Things for Children2024 IEEE 5th International Symposium on the Internet of Sounds (IS2)10.1109/IS262782.2024.10704214(1-9)Online publication date: 30-Sep-2024
https://doi.org/10.1109/IS262782.2024.10704214
Masu RMerendino NRodã ATurchet L(2024)Sustainable Internet of Musical Things: Strategies to Account for Environmental and Social Sustainability in Network-Based Interactive Music SystemsIEEE Access10.1109/ACCESS.2024.339346812(62818-62833)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3393468
Lu LCochrane KKang JGirouard A(2023)“Why are there so many steps?”: Improving Access to Blind and Low Vision Music Learning through Personal Adaptations and Future Design IdeasACM Transactions on Accessible Computing10.1145/361566316:3(1-20)Online publication date: 21-Sep-2023
https://dl.acm.org/doi/10.1145/3615663
Lu LKang JCrispin CGirouard A(2023)Playing with Feeling: Exploring Vibrotactile Feedback and Aesthetic Experiences for Developing Haptic Wearables for Blind and Low Vision Music LearningProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608397(1-16)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3608397
Turchet LLagrange MRottondi CFazekas GPeters NØstergaard JFont FBäckström TFischione C(2023)The Internet of Sounds: Convergent Trends, Insights, and Future DirectionsIEEE Internet of Things Journal10.1109/JIOT.2023.325360210:13(11264-11292)Online publication date: 1-Jul-2023
https://doi.org/10.1109/JIOT.2023.3253602
Ueda YSugiura Y(2022)Demonstration of Trajectory Presentation of Conducting Motions Using Tactile Sensation for Visually Impaired2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00196(901-902)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00196
Turchet LRottondi C(2022)On the relation between the fields of Networked Music Performances, Ubiquitous Music, and Internet of Musical ThingsPersonal and Ubiquitous Computing10.1007/s00779-022-01691-z27:5(1783-1792)Online publication date: 5-Oct-2022
https://dl.acm.org/doi/10.1007/s00779-022-01691-z

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents