Abstract
This paper presents Query-by-Dancing, a dance music retrieval system that enables a user to retrieve music using dance motions. When dancers search for music to play when dancing, they sometimes find it by referring to online dance videos in which the dancers use motions similar to their own dance. However, previous music retrieval systems could not support retrieval specialized for dancing because they do not accept dance motions as a query. Therefore, we developed our Query-by-Dancing system, which uses a video of a dancer (user) as the input query to search a database of dance videos. The query video is recorded using an ordinary RGB camera that does not obtain depth information, like a smartphone camera. The poses and motions in the query are then analyzed and used to retrieve dance videos with similar poses and motions. The system then enables the user to browse the music attached to the videos it retrieves so that the user can find a piece that is appropriate for their dancing. An interesting problem here is that a simple search for the most similar videos based on dance motions sometimes includes results that do not match the intended dance genre. We solved this by using a novel measure similar to tf-idf to weight the importance of dance motions when retrieving videos. We conducted comparative experiments with 4 dance genres and confirmed that the system gained an average of 3 or more evaluation points for 3 dance genres (waack, pop, break) and that our proposed method was able to deal with different dance genres.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cao, Z., Simon, T., Wei, S., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: The 2017 IEEE Conference on Computer Vision and Pattern Recognition (2017)
Casey, M.A., Veltkamp, R.C., Goto, M., Leman, M., Rhodes, C., Slaney, M.: Content-based music information retrieval: current directions and future challenges. Proc. IEEE 96(4), 668–696 (2008)
Chen, J., Chen, A.: Query by rhythm: an approach for song retrieval in music databases. In: Proceedings of the 8th International Workshop on Research Issues in Data Engineering: Continuous-Media Databases and Applications, pp. 139–146 (1998)
Ghias, A., Logan, J., Chamberlin, D., Smith, B.C.: Query by humming - musical information retrieval in an audio database. In: Proceedings of ACM Multimedia 1995, pp. 231–236 (1995)
Jang, J.-S.R., Lee, H.-R., Yeh, C.-H.: Query by tapping: a new paradigm for content-based music retrieval from acoustic input. In: Shum, H.-Y., Liao, M., Chang, S.-F. (eds.) PCM 2001. LNCS, vol. 2195, pp. 590–597. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45453-5_76
Maezawa, A., Goto, M., Okuno, H.G.: Query-by-conducting: an interface to retrieve classical-music interpretations by real-time tempo input. In: The 11th International Society of Music Information Retrieval, pp. 477–482 (2010)
Müller, M.: Fundamentals of Music Processing - Audio, Analysis, Algorithms, Applications. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21945-5
Salvador, S., Chan, P.: Toward accurate dynamic time warping in linear time and space. J. Intell. Data Anal. 11(5), 561–580 (2007)
Schedl, M., Gómez, E., Urbano, J.: Music information retrieval: recent developments and applications. Found. Trends Inf. Retr. 8(2–3), 127–261 (2014)
Smiraglia, R.P.: Musical works as information retrieval entities: epistemological perspectives. In: The 2nd International Society of Music Information Retrieval, pp. 85–91 (2001)
Turnbull, D., Barrington, L., Torres, D., Lanckriet, G.: Semantic annotation and retrieval of music and sound effects. IEEE Trans. Audio, Speech, Lang. Process. 16(2), 467–476 (2008)
Acknowledgments
This work was supported in part by JST ACCEL Grant Number JPMJAC1602, Japan.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Tsuchida, S., Fukayama, S., Goto, M. (2019). Query-by-Dancing: A Dance Music Retrieval System Based on Body-Motion Similarity. In: Kompatsiaris, I., Huet, B., Mezaris, V., Gurrin, C., Cheng, WH., Vrochidis, S. (eds) MultiMedia Modeling. MMM 2019. Lecture Notes in Computer Science(), vol 11295. Springer, Cham. https://doi.org/10.1007/978-3-030-05710-7_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-05710-7_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05709-1
Online ISBN: 978-3-030-05710-7
eBook Packages: Computer ScienceComputer Science (R0)