Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article
Open access

A survey of music similarity and recommendation from music context data

Published: 27 December 2013 Publication History

Abstract

In this survey article, we give an overview of methods for music similarity estimation and music recommendation based on music context data. Unlike approaches that rely on music content and have been researched for almost two decades, music-context-based (or contextual) approaches to music retrieval are a quite recent field of research within music information retrieval (MIR). Contextual data refers to all music-relevant information that is not included in the audio signal itself. In this article, we focus on contextual aspects of music primarily accessible through web technology. We discuss different sources of context-based data for individual music pieces and for music artists. We summarize various approaches for constructing similarity measures based on the collaborative or cultural knowledge incorporated into these data sources. In particular, we identify and review three main types of context-based similarity approaches: text-retrieval-based approaches (relying on web-texts, tags, or lyrics), co-occurrence-based approaches (relying on playlists, page counts, microblogs, or peer-to-peer-networks), and approaches based on user ratings or listening habits. This article elaborates the characteristics of the presented context-based measures and discusses their strengths as well as their weaknesses.

References

[1]
Aizenberg, N., Koren, Y., and Somekh, O. 2012. Build your own music recommender by modeling internet radio streams. In Proceedings of the 21st International Conference on World Wide Web. ACM, New York, 1--10.
[2]
Aucouturier, J.-J. and Pachet, F. 2002. Scaling up music playlist generation. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'02). 105--108.
[3]
Aucouturier, J.-J., Pachet, F., Roy, P., and Beurivé, A. 2007. Signal + Context = Better Classification. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).
[4]
Aucouturier, J.-J., Pachet, F., and Sandler, M. 2005. “The Way It Sounds”: Timbre models for analysis and retrieval of music signals. IEEE Trans. Multimed. 7, 6, 1028--1035.
[5]
Baccigalupo, C., Plaza, E., and Donaldson, J. 2008. Uncovering affinity of artists to multiple genres from social behaviour data. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).
[6]
Barrington, L., Turnbull, D., Yazdani, M., and Lanckriet, G. 2009. Combining audio content and social context for semantic music discovery. In Proceedings of the 32nd ACM SIGIR.
[7]
Baumann, S. and Hummel, O. 2003. Using cultural metadata for artist recommendation. In Proceedings of the 3rd International Conference on Web Delivering of Music (WEDELMUSIC'03).
[8]
Bell, R. M. and Koren, Y. 2007. Lessons from the Netflix prize challenge. SIGKDD Explorat. 9, 2, 75--79.
[9]
Berenzweig, A., Logan, B., Ellis, D. P., and Whitman, B. 2003. A large-scale evaluation of acoustic and subjective music similarity measures. In Proceedings of the 4th International Conference on Music Information Retrieval (ISMIR'03).
[10]
Brill, E. 1992. A simple rule-based part of speech tagger. In Proceedings of the 3rd Conference on Applied Natural Language Processing. 152--155.
[11]
Brochu, E., de Freitas, N., and Bao, K. 2003. The sound of an album cover: Probabilistic multimedia and IR. In Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics.
[12]
Cano, P. and Koppenberger, M. 2004. The emergence of complex network patterns in music artist networks. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04). 466--469.
[13]
Casey, M. A., Veltkamp, R., Goto, M., Leman, M., Rhodes, C., and Slaney, M. 2008. Content-based music information retrieval: Current directions and future challenges. Proc. IEEE 96, 668--696.
[14]
Celma, O. 2008. Music recommendation and discovery in the long tail. Ph.D. thesis, Universitat Pompeu Fabra, Barcelona, Spain.
[15]
Celma, O., Cano, P., and Herrera, P. 2006. SearchSounds: An audio crawler focused on weblogs. In Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR'06).
[16]
Celma, O. and Lamere, P. 2007. ISMIR 2007 tutorial: Music recommendation. http://www.dtic.upf.edu/˜ocelma/Music RecommendationTutorial-ISMIR2007/ (last accessed: April 2013).
[17]
Charniak, E. 1997. Statistical techniques for natural language parsing. AI Magazine 18, 33--44.
[18]
Chen, S., Moore, J., Turnbull, D., and Joachims, T. 2012. Playlist prediction via metric embedding. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 714--722.
[19]
Cohen, W. W. and Fan, W. 2000. Web-collaborative filtering: Recommending music by crawling the web. Computer Netw. 33, 1--6, 685--698.
[20]
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Harshman, R. 1990. Indexing by latent semantic analysis. J. ASIS 41, 391--407.
[21]
Dror, G., Koenigstein, N., and Koren, Y. 2011a. Yahoo! music recommendations: Modeling music ratings with temporal dynamics and item taxonomy. In Proceedings of the 5th ACM Conference on Recommender Systems. ACM. 165--172.
[22]
Dror, G., Koenigstein, N., Koren, Y., and Weimer, M. 2011b. The Yahoo! music dataset and KDD-Cup'11. J. Mach. Learn. Res. 8, 3--18.
[23]
Eck, D., Lamere, P., Bertin-Mahieux, T., and Green, S. 2008. Automatic generation of social tags for music recommendation. In Advances in Neural Information Processing Systems 20 (NIPS'07). MIT Press.
[24]
Ellis, D. P., Whitman, B., Berenzweig, A., and Lawrence, S. 2002. The quest for ground truth in musical artist similarity. In Proceedings of 3rd International Conference on Music Information Retrieval (ISMIR'02).
[25]
Fields, B., Casey, M., Jacobson, K., and Sandler, M. 2008. Do you sound like your friends? Exploring artist similarity via artist social network relationships and audio signal processing. In Proceedings of the International Computer Music Conference (ICMC'08).
[26]
Geleijnse, G., Schedl, M., and Knees, P. 2007. The quest for ground truth in musical artist tagging in the social web era. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).
[27]
Hofmann, T. 1999. Probabilistic latent semantic analysis. In Proceedings of Uncertainty in Artificial Intelligence (UAI).
[28]
Hu, X., Downie, J. S., and Ehmann, A. F. 2009. Lyric text mining in music mood classification. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).
[29]
Hu, X., Downie, J. S., West, K., and Ehmann, A. 2005. Mining music reviews: Promising preliminary results. In Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR'05).
[30]
Jacobson, K., Fields, B., and Sandler, M. 2008. Using audio analysis and network structure to identify communities in on-line social networks of artists. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).
[31]
Kim, J. H., Tomasik, B., and Turnbull, D. 2009. Using artist similarity to propagate semantic information. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).
[32]
Kleedorfer, F., Knees, P., and Pohle, T. 2008. Oh Oh Oh Whoah! Towards automatic topic detection in song lyrics. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08). 287--292.
[33]
Knees, P., Pampalk, E., and Widmer, G. 2004. Artist classification with web-based data. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04). 517--524.
[34]
Knees, P., Pohle, T., Schedl, M., and Widmer, G. 2006a. Combining audio-based similarity with web-based data to accelerate automatic music playlist generation. In Proceedings of the 8th ACM SIGMM International Workshop on Multimedia Information Retrieval (MIR'06) (Santa Barbara, CA).
[35]
Knees, P., Pohle, T., Schedl, M., and Widmer, G. 2007. A music search engine built upon audio-based and web-based similarity measures. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'07).
[36]
Knees, P., Schedl, M., and Pohle, T. 2008. A deeper look into web-based classification of music artists. In Proceedings of 2nd Workshop on Learning the Semantics of Audio Signals (LSAS'08).
[37]
Knees, P., Schedl, M., Pohle, T., and Widmer, G. 2006b. An innovative three-dimensional user interface for exploring music collections enriched with meta-information from the web. In Proceedings of the 14th ACM International Conference on Multimedia (MM'06).
[38]
Laurier, C., Grivolla, J., and Herrera, P. 2008. Multimodal music mood classification using audio and lyrics. In Proceedings of the International Conference on Machine Learning and Applications.
[39]
Law, E., von Ahn, L., Dannenberg, R., and Crawford, M. 2007. Tagatune: A game for music and sound annotation. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).
[40]
Levy, M. and Sandler, M. 2007. A semantic space for music derived from social tags. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).
[41]
L&iamcr;beks, J. and Turnbull, D. 2011. You can judge an artist by an album cover: Using images for music annotation. IEEE MultiMedia, 18, 4, 30--37.
[42]
Linden, G., Smith, B., and York, J. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 4, 1.
[43]
Logan, B., Ellis, D. P., and Berenzweig, A. 2003. Toward evaluation techniques for music similarity. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'03): Workshop on the Evaluation of Music Information Retrieval Systems.
[44]
Logan, B., Kositsky, A., and Moreno, P. 2004. Semantic analysis of song lyrics. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'04).
[45]
MacQueen, J. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, L. M. L. Cam and J. Neyman, Eds., Statistics Series, vol. I, University of California Press, Berkeley and Los Angeles, CA, 281--297.
[46]
Mahedero, J. P. G., Martínez, A., Cano, P., Koppenberger, M., and Gouyon, F. 2005. Natural language processing of lyrics. In Proceedings of the 13th ACM International Conference on Multimedia (MM'05). 475--478.
[47]
Mandel, M. I. and Ellis, D. P. 2007. A web-based game for collecting music metadata. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).
[48]
Mayer, R., Neumayer, R., and Rauber, A. 2008. Rhyme and style features for musical genre classification by song lyrics. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).
[49]
McFee, B. and Lanckriet, G. 2009. Heterogeneous embedding for subjective artist similarity. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).
[50]
Mesnage, C. S., Rafiq, A., Dixon, S., and Brixtel, R. P. 2011. Music discovery with social networks. In Proceedings of the 2nd Workshop on Music Recommendation and Discovery (WOMRAD'11). 7--12.
[51]
Nanopoulos, A., Rafailidis, D., Symeonidis, P., and Manolopoulos, Y. 2010. Musicbox: Personalized music recommendation based on cubic analysis of social tags. IEEE Trans. Audio, Speech, Lang. Process. 18, 2, 407--412.
[52]
Pachet, F., Westerman, G., and Laigre, D. 2001. Musical data mining for electronic music distribution. In Proceedings of the 1st International Conference on Web Delivering of Music (WEDELMUSIC'01).
[53]
Pampalk, E., Flexer, A., and Widmer, G. 2005. Hierarchical organization and description of music collections at the artist level. In Proceedings of the 9th European Conference on Research and Advanced Technology for Digital Libraries (ECDL'05).
[54]
Pampalk, E. and Goto, M. 2007. MusicSun: A new approach to artist recommendation. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).
[55]
Pohle, T., Knees, P., Schedl, M., Pampalk, E., and Widmer, G. 2007. “Reinventing the Wheel”: A novel approach to music player interfaces. IEEE Trans. Multimed. 9, 567--575.
[56]
Pohle, T., Knees, P., Schedl, M., and Widmer, G. 2007. Building an interactive next-generation artist recommender based on automatically derived high-level concepts. In Proceedings of the 5th International Workshop on Content-Based Multimedia Indexing (CBMI'07).
[57]
Sarwar, B., Karypis, G., Konstan, J., and Reidl, J. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of 10th International Conference on World Wide Web (WWW'01). 285--295.
[58]
Schedl, M. 2008. Automatically extracting, analyzing, and visualizing information on music artists from the world wide web. Ph.D. thesis, Johannes Kepler University, Linz, Austria.
[59]
Schedl, M. and Hauger, D. 2012. Mining microblogs to infer music artist similarity and cultural listening patterns. In Proceedings of the 21st International World Wide Web Conference (WWW'12): 4th International Workshop on Advances in Music Information Research (AdMIRe'12).
[60]
Schedl, M., Hauger, D., and Schnitzer, D. 2012. A model for serendipitous music retrieval. In Proceedings of the 16th International Conference on Intelligent User Interfaces (IUI'12): 2nd International Workshop on Context-awareness in Retrieval and Recommendation (CaRR'12).
[61]
Schedl, M. and Knees, P. 2009. Context-based music similarity estimation. In Proceedings of the 3rd International Workshop on Learning the Semantics of Audio Signals (LSAS'09).
[62]
Schedl, M. and Knees, P. 2011. Personalization in multimodal music retrieval. In Proceedings of the 9th Workshop on Adaptive Multimedia Retrieval (AMR'11).
[63]
Schedl, M., Knees, P., Pohle, T., and Widmer, G. 2006. Towards automatic retrieval of album covers. In Proceedings of the 28th European Conference on Information Retrieval (ECIR'06).
[64]
Schedl, M., Knees, P., and Widmer, G. 2005. A web-based approach to assessing artist similarity using co-occurrences. In Proceedings of the 4th International Workshop on Content-Based Multimedia Indexing (CBMI'05).
[65]
Schedl, M., Pohle, T., Knees, P., and Widmer, G. 2011. Exploring the music similarity space on the web. ACM Trans. Info. Syst. 29, 3.
[66]
Serra, X. 2012. Data gathering for a culture specific approach in MIR. In Proceedings of the 21st International World Wide Web Conference (WWW'12): 4th International Workshop on Advances in Music Information Research (AdMIRe'12).
[67]
Shavitt, Y. and Weinsberg, U. 2009. Songs clustering using peer-to-peer co-occurrences. In Proceedings of the IEEE International Symposium on Multimedia (ISM'09): International Workshop on Advances in Music Information Research (AdMIRe'09).
[68]
Shen, J., Meng, W., Yan, S., Pang, H., and Hua, X. 2010. Effective music tagging through advanced statistical modeling. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 635--642.
[69]
Slaney, M. 2011. Web-scale multimedia analysis: Does content matter? IEEE MultiMedia 18, 2, 12--15.
[70]
Slaney, M. and White, W. 2007. Similarity based on rating data. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).
[71]
Sordo, M., Laurier, C., and Celma, O. 2007. Annotating music collections: How content-based similarity helps to propagate labels. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07). 531--534.
[72]
Turnbull, D., Barrington, L., and Lanckriet, G. 2008. Five approaches to collecting tags for music. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).
[73]
Turnbull, D., Liu, R., Barrington, L., and Lanckriet, G. 2007. A game-based approach for collecting semantic annotations of music. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).
[74]
Wang, X., Rosenblum, D., and Wang, Y. 2012. Context-aware mobile music recommendation for daily activities. In Proceedings of the 20th ACM International Conference on Multimedia.
[75]
Whitman, B. and Lawrence, S. 2002. Inferring descriptions and similarity for music from community metadata. In Proceedings of the 2002 International Computer Music Conference (ICMC'02). 591--598.
[76]
Yang, D., Chen, T., Zhang, W., Lu, Q., and Yu, Y. 2012. Local implicit feedback mining for music recommendation. In Proceedings of the 6th ACM Conference on Recommender Systems. 91--98.
[77]
Zadel, M. and Fujinaga, I. 2004. Web services for music information retrieval. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04).
[78]
Zangerle, E., Gassler, W., and Specht, G. 2012. Exploiting Twitter's collective knowledge for music recommendations. In Proceedings of the 21st International World Wide Web Conference (WWW'12): Making Sense of Microposts (#MSM'12). 14--17.
[79]
Zhang, B., Xiang, Q., Lu, H., Shen, J., and Wang, Y. 2009. Comprehensive query-dependent fusion using regression-on-folksonomies: A case study of multimodal music search. In Proceedings of the 17th ACM International Conference on Multimedia. 213--222.
[80]
Zhao, Z., Wang, X., Xiang, Q., Sarroff, A., Li, Z., and Wang, Y. 2010. Large-scale music tag recommendation with explicit multiple attributes. In Proceedings of the 18th ACM International Conference on Multimedia. 401--410.
[81]
Zobel, J. and Moffat, A. 1998. Exploring the Similarity Space. ACM SIGIR Forum 32, 1, 18--34.

Cited By

View all
  • (2024)Estudo de caso sobre canções compostas em musicoterapia com pessoas com uso prejudicial de substânciasCase study on songs composed in music therapy sessions with people with substance use disordersEstudio de caso sobre canciones compuestas musicoterapia con personas con uso de sustancias nocivasBrazilian Journal of Music Therapy10.51914/brjmt.35.2023.417(23-49)Online publication date: 27-Aug-2024
  • (2024)Enhancing Sequential Music Recommendation with Negative Feedback-informed Contrastive LearningProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688188(1028-1032)Online publication date: 8-Oct-2024
  • (2024)Artist Similarity Based on Heterogeneous Graph Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing10.1109/TASLP.2024.343717032(3717-3729)Online publication date: 2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications
ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 10, Issue 1
December 2013
166 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/2559928
Issue’s Table of Contents
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 December 2013
Accepted: 01 April 2013
Revised: 01 January 2013
Received: 01 September 2012
Published in TOMM Volume 10, Issue 1

Check for updates

Author Tags

  1. Music information retrieval
  2. music context
  3. music recommendation
  4. music similarity
  5. survey

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)634
  • Downloads (Last 6 weeks)125
Reflects downloads up to 23 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Estudo de caso sobre canções compostas em musicoterapia com pessoas com uso prejudicial de substânciasCase study on songs composed in music therapy sessions with people with substance use disordersEstudio de caso sobre canciones compuestas musicoterapia con personas con uso de sustancias nocivasBrazilian Journal of Music Therapy10.51914/brjmt.35.2023.417(23-49)Online publication date: 27-Aug-2024
  • (2024)Enhancing Sequential Music Recommendation with Negative Feedback-informed Contrastive LearningProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688188(1028-1032)Online publication date: 8-Oct-2024
  • (2024)Artist Similarity Based on Heterogeneous Graph Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing10.1109/TASLP.2024.343717032(3717-3729)Online publication date: 2024
  • (2024)Playlist Continuation of Cold-Start Songs2024 IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR62202.2024.00029(141-147)Online publication date: 7-Aug-2024
  • (2024)Dynamic artist-based embeddings with application to playlist generationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107604129(107604)Online publication date: Mar-2024
  • (2024)Content-driven music recommendation: Evolution, state of the art, and challengesComputer Science Review10.1016/j.cosrev.2024.10061851(100618)Online publication date: Feb-2024
  • (2023)Music Similarity Estimation based on Feature Extraction and Deep LearningHighlights in Science, Engineering and Technology10.54097/hset.v39i.651439(138-143)Online publication date: 1-Apr-2023
  • (2023)The Role of Serendipity in User-Curated Music PlaylistsProceedings of the 12th Knowledge Capture Conference 202310.1145/3587259.3627552(140-147)Online publication date: 5-Dec-2023
  • (2023)Smart Recommendation of Cloud Music Services Based on Contextualized Mentality Modeling and Beyond2023 38th Youth Academic Annual Conference of Chinese Association of Automation (YAC)10.1109/YAC59482.2023.10401789(928-935)Online publication date: 27-Aug-2023
  • (2023)MUSE: Visual Analysis of Musical Semantic SequenceIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.317536429:9(4015-4030)Online publication date: 1-Sep-2023
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media