short-paper

Air violin: a machine learning approach to fingering gesture recognition

Authors:

David Dalmazzo,

Rafael RamirezAuthors Info & Claims

MIE 2017: Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education

Pages 63 - 66

https://doi.org/10.1145/3139513.3139526

Published: 13 November 2017 Publication History

Get Access

Abstract

We train and evaluate two machine learning models for predicting fingering in violin performances using motion and EMG sensors integrated in the Myo device. Our aim is twofold: first, provide a fingering recognition model in the context of a gamification virtual violin application where we measure both right hand (i.e. bow) and left hand (i.e. fingering) gestures, and second, implement a tracking system for a computer assisted pedagogical tool for self-regulated learners in high-level music education. Our approach is based on the principle of mapping-by-demonstration in which the model is trained by the performer. We evaluated a model based on Decision Trees and compared it with a Hidden Markovian Model.

References

[1]

Frédéric Bevilacqua, Bruno Zamborlin, Anthony Sypniewski, Norbert Schnell, Fabrice Guédy, and Nicolas Rasamimanana. 2009. Continuous realtime gesture following and recognition. In International gesture workshop. Springer, 73–84.

Google Scholar

[2]

Matthew Brand, Nuria Oliver, and Alex Pentland. 1997. Coupled hidden Markov models for complex action recognition. In Computer vision and pattern recognition, 1997. proceedings., 1997 ieee computer society conference on. IEEE, 994–999.

Digital Library

Google Scholar

[3]

Rebecca Fiebrink and Perry R Cook. 2010. The Wekinator: a system for real-time, interactive machine learning in music. In Proceedings of The Eleventh International Society for Music Information Retrieval Conference (ISMIR 2010)(Utrecht).

Google Scholar

[4]

Jules Françoise, Norbert Schnell, Riccardo Borghesi, and Frédéric Bevilacqua. 2014. Probabilistic models for designing motion and sound relationships. In Proceedings of the 2014 international conference on new interfaces for musical expression. 287–292.

Google Scholar

[5]

Ohad Fried and Rebecca Fiebrink. 2013. Cross-modal Sound Mapping Using Deep Learning. In NIME. 531–534.

Google Scholar

[6]

Paul Kolesnik and Marcelo M Wanderley. 2005. Implementation of the Discrete Hidden Markov Model in Max/MSP Environment. In FLAIRS Conference. 68–73.

Google Scholar

[7]

Chad Peiper, David Warden, and Guy Garnett. 2003. An interface for real-time classification of articulations produced by violin bowing. In Proceedings of the 2003 conference on New interfaces for musical expression. National University of Singapore, 192–196.

Digital Library

Google Scholar

[8]

Terence D Sanger. 2007. Bayesian filtering of myoelectric signals. Journal of neurophysiology 97, 2 (2007), 1839–1845.

Crossref

Google Scholar

[9]

Norbert Schnell, Axel Röbel, Diemo Schwarz, Geoffroy Peeters, Riccardo Borghesi, et al. 2009. MuBu and friends–assembling tools for content based real-time interactive audio processing in Max/MSP. In ICMC.

Google Scholar

[10]

Andrew D Wilson and Aaron F Bobick. 2000. Realtime online adaptive gesture recognition. In Pattern Recognition, 2000. Proceedings. 15th International Conference on, Vol. 1. IEEE, 270–275. Abstract 1 Introduction 2 Related Work 3 Materials and Methods 3.1 Motion Sensors 3.2 Models 3.3 Implementation 4 Results 5 Conclusion and Future Work References

Digital Library

Google Scholar

Cited By

View all

ÇÖPGEVEN NÖZKAYA HAYDIN S(2023)Açık ve uzaktan öğrenmede yapay zeka destekli oyunlaştırmaArtificial intelligence based gamification in open and distance learningAçıköğretim Uygulamaları ve Araştırmaları Dergisi10.51948/auad.12037009:1(386-407)Online publication date: 31-Jan-2023
https://doi.org/10.51948/auad.1203700
Volioti CTsagaris ATrigkas DIliou TKatsantonis MMavridis I(2023)HapticSOUND: An Interactive Learning Experience with a Digital Musical InstrumentApplied Sciences10.3390/app1312714913:12(7149)Online publication date: 14-Jun-2023
https://doi.org/10.3390/app13127149
Wilson WGranieri NAli-MacLachlan I(2023)Ubiquitous Multimodality as a Tool in Violin Performance Classification2023 4th International Symposium on the Internet of Sounds10.1109/IEEECONF59510.2023.10335435(1-8)Online publication date: 26-Oct-2023
https://doi.org/10.1109/IEEECONF59510.2023.10335435
Show More Cited By

Index Terms

Air violin: a machine learning approach to fingering gesture recognition
1. Applied computing
  1. Arts and humanities
    1. Sound and music computing
  2. Education
    1. Interactive learning environments
2. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Instance-based learning

Recommendations

Visual analysis of fingering for pedagogical violin transcription
MM '07: Proceedings of the 15th ACM international conference on Multimedia

Automatic music transcription, in spite of decades of research, remains a challenging research problem. The traditional audio-only approach has yet to achieve a satisfactory performance for any computer-aided pedagogical system. Inspired by the high ...
A machine learning approach to gesture detection in violin performance
AM '24: Proceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures

Playing a musical instrument is a highly complex activity. It requires mental and sensorimotor skills, which are learned during a long trajectory. Learning to play an instrument typically involves long periods of practice without teacher supervision. ...
Kinect Based Dynamic Hand Gesture Recognition Algorithm Research
IHMSC '12: Proceedings of the 2012 4th International Conference on Intelligent Human-Machine Systems and Cybernetics - Volume 01

In dynamic gesture track recognition, gesture recognition rate has always been a problem, Because the traditional hand gesture recognition in how to separate the hand type from the surrounding environment, and then determine the palm position has been ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

MIE 2017: Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education

November 2017

75 pages

ISBN:9781450355575

DOI:10.1145/3139513

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ICMI '17

Sponsor:

SIGCHI

ICMI '17: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 13, 2017

Glasgow, UK

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
338
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)8

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

ÇÖPGEVEN NÖZKAYA HAYDIN S(2023)Açık ve uzaktan öğrenmede yapay zeka destekli oyunlaştırmaArtificial intelligence based gamification in open and distance learningAçıköğretim Uygulamaları ve Araştırmaları Dergisi10.51948/auad.12037009:1(386-407)Online publication date: 31-Jan-2023
https://doi.org/10.51948/auad.1203700
Volioti CTsagaris ATrigkas DIliou TKatsantonis MMavridis I(2023)HapticSOUND: An Interactive Learning Experience with a Digital Musical InstrumentApplied Sciences10.3390/app1312714913:12(7149)Online publication date: 14-Jun-2023
https://doi.org/10.3390/app13127149
Wilson WGranieri NAli-MacLachlan I(2023)Ubiquitous Multimodality as a Tool in Violin Performance Classification2023 4th International Symposium on the Internet of Sounds10.1109/IEEECONF59510.2023.10335435(1-8)Online publication date: 26-Oct-2023
https://doi.org/10.1109/IEEECONF59510.2023.10335435
Modi TGochhait S(2023)Impact of Artificial Intelligence on Gamification: Current Applications2023 International Conference on Innovative Data Communication Technologies and Application (ICIDCA)10.1109/ICIDCA56705.2023.10099771(287-290)Online publication date: 14-Mar-2023
https://doi.org/10.1109/ICIDCA56705.2023.10099771
Lo YLi SSun S(2023)Deep Violin: Deep Learning-Based Violin Bowing Training System2023 International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan)10.1109/ICCE-Taiwan58799.2023.10227040(425-426)Online publication date: 17-Jul-2023
https://doi.org/10.1109/ICCE-Taiwan58799.2023.10227040
Al-Saedi ABoeva VCasalicchio EExner P(2022)Context-Aware Edge-Based AI Models for Wireless Sensor Networks—An OverviewSensors10.3390/s2215554422:15(5544)Online publication date: 25-Jul-2022
https://doi.org/10.3390/s22155544
Kim JLee HCho Y(2022)Learning design to support student-AI collaboration: perspectives of leading teachers for AI in educationEducation and Information Technologies10.1007/s10639-021-10831-627:5(6069-6104)Online publication date: 28-Jan-2022
https://doi.org/10.1007/s10639-021-10831-6
Rhodes CAllmendinger RCliment R(2022)Classifying Biometric Data for Musical Interaction Within Virtual RealityArtificial Intelligence in Music, Sound, Art and Design10.1007/978-3-031-03789-4_25(385-400)Online publication date: 15-Apr-2022
https://doi.org/10.1007/978-3-031-03789-4_25
Marky KFischer JMühlhäuser MMatviienko A(2021)Investigating Page Turning Methods for Sheet Music during Piano PlayAdjunct Publication of the 23rd International Conference on Mobile Human-Computer Interaction10.1145/3447527.3474863(1-6)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3447527.3474863
Marky KWeiß AMatviienko ABrandherm FWolf SSchmitz MKrell FMüller FMühlhäuser MKosch TKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Let’s Frets! Assisting Guitar Students During Practice via Capacitive SensingProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445595(1-12)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445595
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Visual analysis of fingering for pedagogical violin transcription

A machine learning approach to gesture detection in violin performance

Kinect Based Dynamic Hand Gesture Recognition Algorithm Research