Toward DMI Evaluation Using Crowd-Sourced Tagging Techniques

Few formal methods exist for evaluating digital musical instruments (DMIs) .This paper proposes a novel method of DMI evaluation using crowd-sourcedtagging. One of the challenges in devising such methods is that the evaluationof a musical instrument is an inherently qualitative task. While previouslyproposed methods have focused on quantitative methods and largely ignored thequalitative aspects of the task, tagging is well-suited to this and is alreadyused to classify things such as websites and musical genres. These, like DMIs,do not lend themselves to simple categorization or parameterization. Using the social tagging method, participating individuals assign descriptivelabels, or tags, to a DMI. A DMI can then be evaluated by analyzing the tagsassociated with it. Metrics can be generated from the tags assigned to theinstrument, and comparisons made to other instruments. This can give thedesigner valuable insight into the where the strengths of the design lie andwhere improvements may be needed. A prototype system for testing the method is proposed in the paper and iscurrently being implemented as part of an ongoing DMI evaluation project. It isexpected that results from the prototype will be available to report by thetime of the conference in May.