The Sound Sketchpad: Expressively Combining Large and Diverse Audio Collections
Pages 297 - 301
Abstract
Software tools for media production have largely been adapted from physical media paradigms, offering blank canvases upon which to import, combine, and process content. In music production, this increasingly involves meticulous manual assembly of audio clips often carefully curated from diverse sources. As collections of audio content scale upwards in sample size, diversity, and number, creative projects require exponentially more time, effort, and attention to effectively shape them. New tools must find new ways to contend with this abundance of content. We propose the Sound Sketchpad, an algorithm-in-the-loop audio-graphical system and interface for combining sounds from a database into new music. It allows a user to sketch broad musical ideas by making sound, and then interactively modify and refine the resulting composition by drawing visual paths. We discuss the design, implementation, and advantages of this approach.
References
[1]
Alexander Travis Adams, Berto Gonzalez, and Celine Latulipe. 2014. SonicExplorer: Fluid Exploration of Audio Parameters. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 237–246. https://doi.org/10.1145/2556288.2557206
[2]
Irina Aldoshina and Ekaterina Davidenkova. 2016. The History of Electro-Musical Instruments in Russia in the First Half of the Twentieth Century. (01 2016).
[3]
M. A. Bartsch and G. H. Wakefield. 2001. To Catch a Chorus: Using Chroma-Based Representations for Audio Thumbnailing. In Proceedings of the 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics (Cat. No.01TH8575). 15–18.
[4]
David Bawiec. 2018. What Is Mix Automation? Everything You’ve Been Too Afraid to Ask. [Online]. Available from: https://www.izotope.com/en/learn/what-is-mix-automation.html.
[5]
Gilberto Bernardes, Carlos Guedes, and Bruce Pennycook. 2013. EarGram: An Application for Interactive Exploration of Concatenative Sound Synthesis in Pure Data. In From Sounds to Music and Emotions, Mitsuko Aramaki, Mathieu Barthet, Richard Kronland-Martinet, and Sølvi Ystad (Eds.). 110–129.
[6]
Margaret Boden. 01 Jan. 2009. Chapter Thirteen. Creativity: How Does It Work?Brill, Leiden, The Netherlands, 235 – 250. https://doi.org/10.1163/ej.9789004174443.i-348.74
[7]
Dmitry Bogdanov, Nicolas Wack, Emilia Gómez, Sankalp Gulati, Perfecto Herrera, Oscar Mayor, Gerard Roma, Justin Salamon, José R. Zapata, and Xavier Serra. 2013. Essentia: An Audio Analysis Library for Music Information Retrieval. In ISMIR.
[8]
Grégoire Carpentier, Gérard Assayag, and Emmanuel Saint-James. 2010. Solving the Musical Orchestration Problem Using Multiobjective Constrained Optimization with a Genetic Local Search Approach. Journal of Heuristics 16, 5 (Oct. 2010), 681–714. https://doi.org/10.1007/s10732-009-9113-7
[9]
Morwaread M. Farbood, Egon Pasztor, and Kevin Jennings. 2004. Hyperscore: A Graphical Sketchpad for Novice Composers. IEEE Comput. Graph. Appl. 24, 1 (Jan. 2004), 50–54. https://doi.org/10.1109/MCG.2004.1255809
[10]
Frederic Font, Gerard Roma, and Xavier Serra. 2013. Freesound Technical Demo. In Proceedings of the 21st ACM International Conference on Multimedia (Barcelona, Spain) (MM ’13). Association for Computing Machinery, New York, NY, USA, 411–412. https://doi.org/10.1145/2502081.2502245
[11]
Gabriela Goldschmidt. 1991. The Dialectics of Sketching. Creativity Research Journal 4, 2 (1991), 123–143. https://doi.org/10.1080/10400419109534381
[12]
Benjamin Hackbarth, Norbert Schnell, and Diemo Schwarz. 2010. AudioGuide: A Framework for Creative Exploration of Concatenative Sound Synthesis. (2010).
[13]
Paul Harkins. 2016. Microsampling: From Akufen’s Microhouse to Todd Edwards and the Sound of UK Garage. In Musical Rhythm in the Age of Digital Reproduction. 177–194.
[14]
Scott Kirkpatrick, C Daniel Gelatt, and Mario P Vecchi. 1983. Optimization by Simulated Annealing. science 220, 4598 (1983), 671–680.
[15]
Malte Kobel. 2019. The Drum Machine’s Ear: XLN Audio’s Drum Sequencer XO and Algorithmic Listening. Sound Studies 5, 2 (2019), 201–204. https://doi.org/10.1080/20551940.2019.1661163
[16]
Peter Manning. 2012. The Oramics Machine: From Vision to Reality. Organised Sound 17, 2 (2012), 137–147. https://doi.org/10.1017/S1355771812000064
[17]
Kyle McDonald, Manny Tan, and Yotam Mann. 2017. The Infinite Drum Machine. [Online]. Available from: https://experiments.withgoogle.com/drum-machine.
[18]
Kembrew McLeod. 2002. How Copyright Law Changed Hip Hop: An Interview with Public Enemy’s Chuck D and Hank Shocklee. Stay Free Magazine 20(2002).
[19]
Dalibor Mitrović, Matthias Zeppelzauer, and Christian Breiteneder. 2010. Features for Content-Based Audio Retrieval. In Advances in computers. Vol. 78. Elsevier, 71–150.
[20]
Dan R. Olsen. 2007. Evaluating User Interface Systems Research. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (Newport, Rhode Island, USA) (UIST ’07). Association for Computing Machinery, New York, NY, USA, 251–258. https://doi.org/10.1145/1294211.1294256
[21]
John Oswald. 1985. Plunderphonics, or Audio Piracy as a Compositional Prerogative. In Wired Society Electro-Acoustic Conference, Toronto.
[22]
Geoffroy Peeters. 2004. A Large Set of Audio Features for Sound Description (Similarity and Classification) in the CUIDADO Project. CUIDADO IST Project Report 54, 0 (2004), 1–25.
[23]
Mitchel Resnick and Brian Silverman. 2005. Some Reflections on Designing Construction Kits for Kids. In Proceedings of the 2005 conference on Interaction Design and Children. 117–122.
[24]
Emily Robertson. 2010. ”It Looks Like Sound!”: Drawing a History of ”Animated Music” in the Early Twentieth Century. Master’s thesis. University of Maryland, College Park.
[25]
Davide Rocchesso, Guillaume Lemaitre, Patrick Susini, Sten Ternström, and Patrick Boussard. 2015. Sketching Sound with Voice and Gesture. Interactions 22, 1 (Jan. 2015), 38–41. https://doi.org/10.1145/2685501
[26]
D. Schwarz. 2007. Corpus-Based Concatenative Synthesis. IEEE Signal Processing Magazine 24, 2 (2007), 92–104.
[27]
Diemo Schwarz, Grégory Beller, Bruno Verbrugghe, and Sam Britton. 2006. Real-Time Corpus-Based Concatenative Synthesis with CataRT. In 9th International Conference on Digital Audio Effects (DAFx). 279–282. https://hal.archives-ouvertes.fr/hal-01161358
[28]
Akito van Troyer. 2013. Constellation: A Tool for Creative Dialog Between Audience and Composer. In Proceedings of the International Symposium on Computer Music Multidisciplinary Research. 534–541.
[29]
Edgard Varèse and Chou Wen-chung. 1966. The Liberation of Sound. Perspectives of New Music 5, 1 (1966), 11–19. http://www.jstor.org/stable/832385
Index Terms
- The Sound Sketchpad: Expressively Combining Large and Diverse Audio Collections
Index terms have been assigned to the content through auto-classification.
Recommendations
Generative Audio and Real-Time Soundtrack Synthesis in Gaming Environments: An exploration of how dynamically rendered soundtracks can introduce new artistic sound design opportunities and enhance the immersion of interactive audio spaces.
OzCHI '20: Proceedings of the 32nd Australian Conference on Human-Computer InteractionAn important yet oft-overlooked front in the scope of interactive media, audio technologies have remained relatively stagnant compared to groundbreaking advancements made in fields such as visual fidelity and virtual reality. This paper explores the use ...
RhythmSynthesis: visual music instrument
C&C '11: Proceedings of the 8th ACM conference on Creativity and cognitionOriginating as an investigation into the relationships between rhythm and technology, RhythmSynthesis applies color, shape, and sound to demonstrate how our understanding of visual music, computation, and tangible, audio-visual interactions can be ...
Melody Transcription From Music Audio: Approaches and Evaluation
Although the process of analyzing an audio recording of a music performance is complex and difficult even for a human listener, there are limited forms of information that may be tractably extracted and yet still enable interesting applications. We ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2021
618 pages
ISBN:9781450380171
DOI:10.1145/3397481
- General Chairs:
- Tracy Hammond,
- Katrien Verbert,
- Dennis Parra,
- Program Chairs:
- Bart Knijnenburg,
- John O'Donovan,
- Paul Teale
Copyright © 2021 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 April 2021
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
IUI '21: 26th International Conference on Intelligent User Interfaces
April 14 - 17, 2021
TX, College Station, USA
Acceptance Rates
Overall Acceptance Rate 746 of 2,811 submissions, 27%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 717Total Downloads
- Downloads (Last 12 months)174
- Downloads (Last 6 weeks)22
Reflects downloads up to 20 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in