research-article

PopMash: an automatic musical-mashup system using computation of musical and lyrical agreement for transitions

Authors:

Shouqian SunAuthors Info & Claims

Multimedia Tools and Applications, Volume 79, Issue 29-30

Pages 21841 - 21871

https://doi.org/10.1007/s11042-020-08934-2

Published: 01 August 2020 Publication History

Abstract

Musical-mashup is a popular form of music re-creation, aiming at combining multiple pieces of music to create new music artworks. Presently, it is also a challenge in the field of music information study. In this work, an effective framework for harmonious musical-mashup generation is provided. In the experiment, lyrics, melody, and rhythm of music were synthetically analyzed. The “harmony” of mashup transition was evaluated in view of the similarity scores of rhythm, melody and lyrics rhyme. “Mashupable” song segments were selected based on the transition harmony evaluation. Then, the musical-mashup output was carried out by adjusting the rhythm, tone and loudness of each segment. Finally, we created PopMash based on the proposed method, an automatic musical-mashup system that can make smooth and harmony transitions from multiple perspectives, which can efficiently reduce the manual work of music recreation.

References

[1]

Basu S (2004) Mixing with Mozart. In: Proc. ICMC, Miami, USA

[2]

Brown S, Parsons D (1975) The directory of tunes and musical themes

[3]

Catarina M, Rodrigues A, Bernardes G et al (2018) MixMash: a visualisation system for musical Mashup creation. In: International conference information visualisation. IEEE Computer Society

[4]

Chang S and Lee K Lyrics-to-audio alignment by unsupervised discovery of repetitive patterns in vowel acoustics IEEE Access 2017 5 16635-16648

[5]

Davies MEP, Hamel P, and Yoshii K AutoMashUpper: automatic creation of multi-song musical-mashups IEEE/ACM Transactions on Audio, Speech, and Language Processing 2014 22 12 1726-1737

[6]

Diehl N Object - oriented motion estimation and segmentation in image sequences Journal of Signal Processing image Communication 1991 3 1 23-56

[7]

Downie S, Nelson M (2000) Evaluation of a simple and effective music for information retrieval method. Proceedings of International Acm Sigir conference on research & development in information retrieval, ACM, pp 73–80

[8]

Francu C and Nevill-Manning C Distance metrics and indexing strategies for a digital library of popular music IEEE International Conference on. Multimedia and Expo 2000 2 889-892

[9]

Gilleland M (2009) Levenshtein distance, in three flavors. Merriam Park Software

[10]

Goto M (1988) A chorus-section detection method for musical audio signals. In: International conference on acoustics, speech, and signal processing

[11]

Goto M (2012) Grand challenges in music information research. Multimodal music processing. Dagstuhl Publishing, pp 217–225

[12]

Griffin G, Kim YE, Turnbull D (2010) Beat-Sync-Mash-Coder: A web application for real-time creation of beat-synchronous musical-mashups. In: IEEE International Conference on Speech and Signal Processing (ICASSP). IEEE

[13]

Hofmann-Engl L (2002) Melodic similarity-a conceptual framework. In: Proceedings on 2nd international conference on understanding and creating music, Naples

[14]

Hofmann-Engl L (2002) Rhythmic similarity: a theoretical and empirical approach. In: Proceedings of the seventh international conference on music perception and cognition, pp 564–567

[15]

Ishizaki H, Hoashi K, Takishima Y (2009) Full-automatic DJ mixing system with optimal tempo adjustment based on measurement function of user discomfort. In: Proc. Of ISMIR, Kobe, Japan, pp 135–140

[16]

Jehan T Creating music by listening 2005 Massachusetts Institute of Technology PhD Dissertation

[17]

Jennings HD, Ivanov PC, Martins ADM, et al. Variance fluctuations in nonstationary time series: a comparative study of music genres Physica A Statistical Mechanics & Its Applications 2010 336 3 585-594

[18]

Kim YE, Chai W, Garcia R et al (2001) Analysis of a contour based representation for melody. In: Proc International

[19]

Langford S (2011) The remix manual. Focal Press

[20]

Lerdahl F and Jackendo R A generative going of tonal music 1983 Cambridge MIT Press

[21]

Levy M, Sandler M Structural segmentation of himself to audio by constrained clustering. IEEE the Transactions on Audio researched & Language Processing 16(2):318–326

[22]

Li Y, Wang D (2005) Detecting pitch of singing voice in polyphonic audio. In: International conference on acoustics, vol 3

[23]

Li Y, Wu Y, and Liu B A new approximate melody matching method and its application in the hum retrieval system Computer research and development 2003 40 11 1554-1560

[24]

Lin HY, Lin YT, Tien MC, Wu JL (2009) Music Paste: concatenating music clips based on chroma and rhythm features. In: Proc. Of ISMIR, Kobe, Japan

[25]

Lin YT, Lee CL, Jang JS et al (2015) Bridging music via sound effects. In: IEEE International Symposium on Multimedia. IEEE

[26]

Liu IT, Lin YT, Wu JL (2013) Music cut and paste: a personalized musical medley generating system. In: Proc. ISMIR, Curitiba, PR, Brazil

[27]

Luca T, Carlo F, Georg E, Damian K, and Mathieu B Internet of musical things: vision and challenges IEEE Access 2018 6 61994-62017

[28]

Maddage NC, Xu C, Kankanhalli MS (2004) Content-based music structure analysis with applications to music semantics understanding. In: ACM international conference on multimedia

[29]

Maidin DO (1998) A geometrical algorithm for melodic difference. In: Computing in musicology a directory of research, pp 65–72

[30]

Mckinney MF, Moelants D (2004) Extracting the perceptual tempo from music. In: Proc International Conference on Music Information Retrieval ISMIR, pp 146–149

[31]

Miller SM and Snyder B Music and memory: an introduction Comput Music J 2002 26 2 98-100

[32]

Moulines E and Laroche J Non-parametric techniques for pitch-scale and timescale modification of speech Speech Comm 1995 16 2 175-205

[33]

Nguyen DTD, Saini M, Nguyen VT et al (2013) Jiku director: a mobile video mashup system. Proceedings of the 21st ACM international conference on multimedia. ACM

[34]

Ohya H, Morishima S (2013) Automatic mash up music video generation system by remixing existing video content. In: Proceedings of international conference on culture & computing. IEEE

[35]

Pan Y, He F, and Yu H A correlative denoising autoencoder to model social influence for top-N recommender system Front Comp Sci 2020 14 3 1-13

[36]

Phaye SSR, Mehta L, Saini M (2018) The one man show. IEEE International Symposium on Multimedia

[37]

Quan Q, He F, Li H (2020) A multi-phase blending method with incremental intensity for training detection networks. Vis Comput. 10.1007/s00371-020-01796-7

[38]

Rand W, Birmingham W (2001) Statistical analysis in music information retrieval. Journal of Indiana University

[39]

Roman BG, Matthew EPD, and Bernhard US Psychoacoustic approaches for harmonic music mixing Appl Sci 2016 6 123 1-22

[40]

Ross M, Shaffer H, Cohen A, Freudberg R, and Manley H Average amount difference function pitch extractor IEEE Trans Acoust Speech Signal Process 1974 22 5 353-362

[41]

Saini MK, Gadde R, Yan S et al (2012) MoViMash: online mobile video mashup. In: Proceedings of the 20th ACM international conference on multimedia. ACM

[42]

Shrestha P, With PHND, Weda H et al (2010) Automatic mashup generation from multiple-camera concert recordings. In: Proceedings of the 18th international conference on Multimedea, Firenze, Italy, October 25–29

[43]

Tokui N (2008) Massh! - A Web-based Collective Musical-mashup System, Massh!: a web-based collective musical-mashup system. In: Proceedings of the Third International Conference on Digital Interactive Media in Entertainment and Arts, September 2008, Athens, Greece. ACM, pp 10–12

[44]

Yu H, He F, Pan Y (2020) A scalable region-based level set method using adaptive bilateral filter for noisy image segmentation. Multimed Tools Appl. 10.1007/s11042-019-08493-1

[45]

Zhang S, He F (2019) DRCDN: learning deep residual convolutional dehazing networks. Vis Comput. 10.1007/s00371-019-01774-8

[46]

Zhang J, He F, Chen Y (2020) A new haze removal approach for sky/river alike scenes based on external and internal clues. Multimed Tools Appl. 10.1007/s11042-019-08399-y

Cited By

George AMary XGeorge S(2022)Development of an intelligent model for musical key estimation using machine learning techniquesMultimedia Tools and Applications10.1007/s11042-022-12432-y81:14(19945-19964)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s11042-022-12432-y

Recommendations

Synchronization and fluctuation of rhythm in musical cooperative performance
HCII'11: Proceedings of the 14th international conference on Human-computer interaction: design and development approaches - Volume Part I

A live musical performance gives us better impression than recorded music heard from a portable music player. From player's point of view, live performance also gives better impression than playing music with metronome or recorded music. Thus, the ...
Functional modelling of musical harmony: an experience report
ICFP '11

Music theory has been essential in composing and performing music for centuries. Within Western tonal music, from the early Baroque on to modern-day jazz and pop music, the function of chords within a chord sequence can be explained by harmony theory. ...
Functional modelling of musical harmony: an experience report
ICFP '11: Proceedings of the 16th ACM SIGPLAN international conference on Functional programming

Music theory has been essential in composing and performing music for centuries. Within Western tonal music, from the early Baroque on to modern-day jazz and pop music, the function of chords within a chord sequence can be explained by harmony theory. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Multimedia Tools and Applications

Multimedia Tools and Applications Volume 79, Issue 29-30

Aug 2020

1555 pages

ISSN:1380-7501

Issue’s Table of Contents

© Springer Science+Business Media, LLC, part of Springer Nature 2020.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 August 2020

Accepted: 13 April 2020

Revision received: 02 March 2020

Received: 23 June 2019

Author Tags

Qualifiers

Research-article

Funding Sources

the Natural Science Foundation of Zhejiang Province of China
Key Project of Natural Science Foundation of Zhejiang Province of China
National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

George AMary XGeorge S(2022)Development of an intelligent model for musical key estimation using machine learning techniquesMultimedia Tools and Applications10.1007/s11042-022-12432-y81:14(19945-19964)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s11042-022-12432-y

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents