Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/1497308.1497421acmconferencesArticle/Chapter ViewAbstractPublication PagesiiwasConference Proceedingsconference-collections
research-article

Computer vision method for pianist's fingers information retrieval

Published: 24 November 2008 Publication History

Abstract

Voice, text or music recognition systems have quite a lot of elements in common with gesture recognition systems. They recognize "something" moving in orbit in space and in time. Hidden Markov Models (HMM) is an advisable mathematic theory managing stochastic processes such as human voice and gestures. This paper proposes a system retrieving music information from the right hand of a pianist using computer vision technology. Finger movement recognition is included in a general procedure, the gesture recognition. The results from a tentative analysis led to the use of a low cost video camera. The study on the pianist fingering retrieval has been effectuated on four aspects: preparation and capture of video signal, finger segmentation and tracking, feature vector exportation and HMM classifier. The recognition has not been effectuated in real time. Video signal processing (masks, filters, threshold etc) as well as the application of HMM was done with the use of suitable MATLAB toolboxes.

References

[1]
http://www.mathworks.com/
[2]
Arshia Cont. 2004. Improvement of observation modeling for score following. Dea atiam. University of Paris 6, IRCAM, Paris,.
[3]
Arshia Cont. 2006. Realtime audio to score alignment for polyphonic music instruments using sparse non-negative constraints and hierarchical hmms. In Proceedings of the IEEE ICASSP, (Toulouse, France, May 2006).
[4]
D. Schwarz, S. Britton, R. Cahen, T. Goepfer. 2007. Musical Applications of Real-Time Corpus-Based Concatenative Synthesis. International Computer Music Conference (ICMC) (Copenhagen, Denmark 2007).
[5]
D. Schwarz, A. Cont, N. Orio. 2006. Score Following at Ircam. In Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR), (Victoria, USA 2006)
[6]
D. Schwarz, N. Orio, N. Schnell. 2004. Robust Polyphonic Midi Score Following with Hidden Markov Models. In Proceedings of the International Computer Music Conference (ICMC) (Miami, USA 2004).
[7]
Quam D. L. 1990. Gesture recognition with a DataGlove, in Aerospace and Electronics Conference. In Proceedings of the IEEE National, vol. 2, pp. 755--760.
[8]
D. Radicioni, L. Anselma and V. Lombardo. 2004. A segmentation-based prototype to compute string instruments fingering. In Proceedings of the Conference on Interdisciplinary Musicology (Graz, Austria, 2004).
[9]
C. Traube and J. O. Smith III. 2000. Estimating the plucking point on a guitar string. In Proceedings of the COST G-6 Conference on Digital Audio Effects (Verona, Italy 2000).
[10]
J. A. Verner. 1995. Midi guitar synthesis yesterday, today and tomorrow. An overview of the whole fingerpicking thing. Recording Magazine, 8 (9): 52--57.

Cited By

View all
  • (2018)A real-time system for online learning-based visual transcription of piano musicMultimedia Tools and Applications10.1007/s11042-018-5803-177:19(25513-25535)Online publication date: 1-Oct-2018
  • (2015)Real-Time Piano Music Transcription Based on Computer VisionIEEE Transactions on Multimedia10.1109/TMM.2015.247370217:12(2113-2121)Online publication date: Dec-2015
  • (2013)Key detection for a virtual piano teacher2013 28th International Conference on Image and Vision Computing New Zealand (IVCNZ 2013)10.1109/IVCNZ.2013.6727030(282-287)Online publication date: Nov-2013
  • Show More Cited By
  1. Computer vision method for pianist's fingers information retrieval

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    iiWAS '08: Proceedings of the 10th International Conference on Information Integration and Web-based Applications & Services
    November 2008
    703 pages
    ISBN:9781605583495
    DOI:10.1145/1497308
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 24 November 2008

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. HCI
    2. HMM
    3. computer vision
    4. gesture recognition

    Qualifiers

    • Research-article

    Conference

    iiWAS08
    Sponsor:

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)1
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2018)A real-time system for online learning-based visual transcription of piano musicMultimedia Tools and Applications10.1007/s11042-018-5803-177:19(25513-25535)Online publication date: 1-Oct-2018
    • (2015)Real-Time Piano Music Transcription Based on Computer VisionIEEE Transactions on Multimedia10.1109/TMM.2015.247370217:12(2113-2121)Online publication date: Dec-2015
    • (2013)Key detection for a virtual piano teacher2013 28th International Conference on Image and Vision Computing New Zealand (IVCNZ 2013)10.1109/IVCNZ.2013.6727030(282-287)Online publication date: Nov-2013
    • (2010)Retrieval of guitarist fingering information using computer vision2010 25th International Conference of Image and Vision Computing New Zealand10.1109/IVCNZ.2010.6148852(1-7)Online publication date: Nov-2010

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media