research-article

PM4Music: A Scriptable Parametric Modeling Interface for Music Visualizer Design Using PM4VR

Author:

Wanwan LiAuthors Info & Claims

ICDSP '24: Proceedings of the 2024 8th International Conference on Digital Signal Processing

Pages 33 - 38

https://doi.org/10.1145/3653876.3653906

Published: 01 August 2024 Publication History

Abstract

The intersection of music and visual arts has long been a captivating field, offering immersive experiences that engage both auditory and visual senses. Music visualizers, in particular, serve as a dynamic medium for translating sound into captivating visual displays. This research paper introduces PM4Music, a cutting-edge scriptable parametric modeling interface tailored for music visualizer design. Leveraging the power of PM4VR, a versatile virtual reality environment, PM4Music empowers artists to create intricate and synchronized visualizations that enhance the auditory experience.

Supplemental Material

MP4 File

Supplementary Video

Download
8.42 MB

References

[1]

Vladislav Angelov, Emiliyan Petkov, Georgi Shipkovenski, and Teodor Kalushkov. 2020. Modern virtual reality headsets. In 2020 International congress on human-computer interaction, optimization and robotic applications (HORA). IEEE, 1–5.

[2]

YAGV Boas. 2013. Overview of virtual reality technologies. In Interactive Multimedia Conference, Vol. 2013. sn.

[3]

Gabriel Dias Cantareira, Luis Gustavo Nonato, and Fernando V Paulovich. 2016. Moshviz: A detail+ overview approach to visualize music elements. IEEE Transactions on Multimedia 18, 11 (2016), 2238–2246.

Digital Library

[4]

Yongli Dai. 2021. Digital art into the design of cultural and creative products. In Journal of Physics: Conference Series, Vol. 1852. IOP Publishing, 032042.

[5]

Joyce Horn Fonteles, Maria Andréia Formico Rodrigues, and Victor Emanuel Dias Basso. 2013. Creating and evaluating a particle system for music visualization. Journal of Visual Languages & Computing 24, 6 (2013), 472–482.

Digital Library

[6]

Ben Gold, Nelson Morgan, and Dan Ellis. 2011. Speech and audio signal processing: processing and perception of speech and music. John Wiley & Sons.

Digital Library

[7]

Dong Jun Kim and Wanwan Li. 2023. A View Direction-Driven Approach for Automatic Room Mapping in Mixed Reality. In Proceedings of the 2023 5th International Conference on Image Processing and Machine Vision. 29–33.

Digital Library

[8]

Kelian Li and Wanwan Li. 2021. MusicTXT: A Text-based Interface for Music Notation. In Proceedings of the 11th Workshop on Ubiquitous Music (UbiMus 2021). g-ubimus, 62–71.

[9]

Wanwan Li. 2021. Make uber faster: Automatic optimization of uber schedule using openstreetmap data. In Proceedings of the 2021 EURASIAGRAPHICS. 19–26.

[10]

Wanwan Li. 2021. Pen2VR: A Smart Pen Tool Interface for Wire Art Design in VR. (2021).

[11]

Wanwan Li. 2021. Procedural Modeling of the Great Barrier Reef. In Advances in Visual Computing: 16th International Symposium, ISVC 2021, Virtual Event, October 4-6, 2021, Proceedings, Part I. Springer, 381–391.

[12]

Wanwan Li. 2022. Creative Molecular Model Design for Chemistry Edutainment. In Proceedings of the 14th International Conference on Education Technology and Computers. 226–232.

Digital Library

[13]

Wanwan Li. 2022. Musical Instrument Performance in Augmented Virtuality. In Proceedings of the 6th International Conference on Digital Signal Processing. 91–97.

Digital Library

[14]

Wanwan Li. 2022. PM4VR: A Scriptable Parametric Modeling Interface for Conceptual Architecture Design in VR. In Proceedings of the 18th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. 1–8.

Digital Library

[15]

Wanwan Li. 2022. Procedural Marine Landscape Synthesis for Swimming Exergame in Virtual Reality. In 2022 IEEE Games, Entertainment, Media Conference (GEM). IEEE, 1–8.

[16]

Wanwan Li. 2022. Simulating Ice Skating Experience in Virtual Reality. In 2022 7th International Conference on Image, Vision and Computing (ICIVC). IEEE, 706–712.

[17]

Wanwan Li. 2022. Simulating Virtual Construction Scenes on OpenStreetMap. In Proceedings of the 6th International Conference on Virtual and Augmented Reality Simulations. 14–20.

Digital Library

[18]

Wanwan Li. 2023. Animating Parametric Kinetic Spinner in Virtual Reality. In 2023 7th International Conference on Advances in Image Processing (ICAIP 2023). 1–5.

[19]

Wanwan Li. 2023. Compare The Size: Automatic Synthesis of Size Comparison Animation in Virtual Reality. In 2023 5th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA). IEEE, 1–4.

[20]

Wanwan Li. 2023. Elliptical4VR: An Interactive Exergame Authoring Tool for Personalized Elliptical Workout Experience in VR. In Proceedings of the 2023 5th International Conference on Image, Video and Signal Processing. 111–116.

Digital Library

[21]

Wanwan Li. 2023. InsectVR: Simulating Crawling Insects in Virtual Reality for Biology Edutainment. In Proceedings of the 7th International Conference on Education and Multimedia Technology. 8–14.

Digital Library

[22]

Wanwan Li. 2023. Patch-based Monte Carlo Terrain Upsampling via Gaussian Laplacian Pyramids. In Proceedings of the 2023 5th International Conference on Image, Video and Signal Processing. 172–177.

Digital Library

[23]

Wanwan Li. 2023. PlanetTXT: A Text-based Planetary System Simulation Interface for Astronomy Edutainment. In Proceedings of the 2023 14th International Conference on E-Education, E-Business, E-Management and E-Learning. 47–53.

Digital Library

[24]

Wanwan Li. 2023. SurfChessVR: Deploying Chess Game on Parametric Surface in Virtual Reality. In 2023 9th International Conference on Virtual Reality (ICVR). IEEE, 171–178.

[25]

Wanwan Li. 2023. Synthesizing 3D VR Sketch Using Generative Adversarial Neural Network. In Proceedings of the 2023 7th International Conference on Big Data and Internet of Things. 122–128.

Digital Library

[26]

Wanwan Li. 2023. Synthesizing Realistic Cracked Terrain for Virtual Arid Environment Generation. In 2023 8th International Conference on Communication, Image and Signal Processing (CCISP). IEEE, 361–366.

[27]

Wanwan Li. 2023. Synthesizing virtual night scene on openstreetmap. In 2023 International Conference on Communications, Computing and Artificial Intelligence (CCCAI). IEEE, 177–181.

[28]

Wanwan Li. 2023. Synthesizing Virtual World Palace Scenes on OpenStreetMap. In 2023 7th International Conference on Computer, Software and Modeling (ICCSM). IEEE, 62–66.

[29]

Wanwan Li. 2023. Terrain synthesis for treadmill exergaming in virtual reality. In 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 263–269.

[30]

Wanwan Li. 2024. PM4Car: A Scriptable Parametric Modeling Interface for Concept Car Design Using PM4VR. In 2024 International Conference on Computer Graphics and Image Processing (CGIP). 1–6.

[31]

Wanwan Li. 2024. PM4Pottery: A Scriptable Parametric Modeling Interface for Conceptual Pottery Design Using PM4VR. In 2024 International Conference on Automation, Robotics, and Applications (ICARA). 1–5.

[32]

Wanwan Li. 2024. PM4WireArt: A Scriptable Parametric Modeling Interface for Conceptual Wire Art Design Using PM4VR. In 2024 International Conference on International Conference on Information and Computer Technologies (ICICT). 1–6.

[33]

Wanwan Li, Behzad Esmaeili, and Lap-Fai Yu. 2022. Simulating Wind Tower Construction Process for Virtual Construction Safety Training and Active Learning. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 369–372.

[34]

Wanwan Li, Haikun Huang, Tomay Solomon, Behzad Esmaeili, and Lap-Fai Yu. 2022. Synthesizing personalized construction safety training scenarios for VR training. IEEE Transactions on Visualization and Computer Graphics 28, 5 (2022), 1993–2002.

[35]

Wanwan Li, Javier Talavera, Amilcar Gomez Samayoa, Jyh-Ming Lien, and Lap-Fai Yu. 2020. Automatic synthesis of virtual wheelchair training scenarios. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 539–547.

[36]

Wanwan Li, Biao Xie, Yongqi Zhang, Walter Meiss, Haikun Huang, and Lap-Fai Yu. 2020. Exertion-aware path generation.ACM Trans. Graph. 39, 4 (2020), 115.

Digital Library

[37]

Hugo B Lima, Carlos GR Dos Santos, and Bianchi S Meiguins. 2021. A survey of music visualization techniques. ACM Computing Surveys (CSUR) 54, 7 (2021), 1–29.

Digital Library

[38]

Delfina Malandrino, Donato Pirozzi, and Rocco Zaccagnino. 2018. Visualization and music harmony: Design, implementation, and evaluation. In 2018 22nd International Conference Information Visualisation (IV). IEEE, 498–503.

[39]

Meinard Mueller, Bryan A Pardo, Gautham J Mysore, and Vesa Valimaki. 2018. Recent advances in music signal processing [from the guest editors]. IEEE Signal Processing Magazine 36, 1 (2018), 17–19.

[40]

Meinard Muller, Daniel PW Ellis, Anssi Klapuri, and Gaël Richard. 2011. Signal processing for music analysis. IEEE Journal of selected topics in signal processing 5, 6 (2011), 1088–1110.

[41]

Devin Perry, Thomas Bivins, Bianca Dehaan, and Wanwan Li. 2023. Procedural Rhythm Game Generation in Virtual Reality. In Proceedings of the 2023 6th International Conference on Image and Graphics Processing. 218–222.

Digital Library

[42]

Archana Rani. 2018. Digital Technology: It’s Role in Art Creativity. Journal of Commerce and Trade 13, 2 (2018), 61–65.

[43]

William R Sherman and Alan B Craig. 2018. Understanding virtual reality: Interface, application, and design. Morgan Kaufmann.

[44]

Yujia Wang, Wei Liang, Wanwan Li, Dingzeyu Li, and Lap-Fai Yu. 2020. Scene-aware background music synthesis. In Proceedings of the 28th ACM International Conference on Multimedia. 1162–1170.

Digital Library

[45]

Isabell Wohlgenannt, Alexander Simons, and Stefan Stieglitz. 2020. Virtual reality. Business & Information Systems Engineering 62 (2020), 455–461.

[46]

Biao Xie, Huimin Liu, Rawan Alghofaili, Yongqi Zhang, Yeling Jiang, Flavio Destri Lobo, Changyang Li, Wanwan Li, Haikun Huang, Mesut Akdere, 2021. A review on virtual reality skill training applications. Frontiers in Virtual Reality 2 (2021), 645153.

[47]

Wei Zhu. 2020. Study of creative thinking in digital media art design education. Creative Education 11, 2 (2020), 77–85.

Cited By

Li W(2024)PM4Bag: A Scriptable Parametric Modeling Interface for Conceptual Bag Design Using PM4VR2024 8th International Conference on Imaging, Signal Processing and Communications (ICISPC)10.1109/ICISPC63824.2024.00038(178-181)Online publication date: 19-Jul-2024
https://doi.org/10.1109/ICISPC63824.2024.00038
Li W(2024)PM4WallArt: A Scriptable Parametric Modeling Interface for Conceptual Wall Art Design Using PM4VR2024 9th International Conference on Computer and Communication Systems (ICCCS)10.1109/ICCCS61882.2024.10602918(1380-1383)Online publication date: 19-Apr-2024
https://doi.org/10.1109/ICCCS61882.2024.10602918

Index Terms

PM4Music: A Scriptable Parametric Modeling Interface for Music Visualizer Design Using PM4VR
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
  2. Modeling and simulation
    1. Simulation types and techniques
      1. Molecular simulation
2. Human-centered computing
  1. Interaction design
    1. Systems and tools for interaction design

Recommendations

Automatic Guitar Music Transcription
ACSAT '12: Proceedings of the 2012 International Conference on Advanced Computer Science Applications and Technologies

This paper presents a system that helps in automatically generating guitar tablatures and musical scores based on musical audio data. Information gathered from the audio consists of pitch, onsets and durations, chords, and beat and tempo. Major issues ...
PM4VR: A Scriptable Parametric Modeling Interface for Conceptual Architecture Design in VR
VRCAI '22: Proceedings of the 18th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry

In this paper, we propose PM4VR, a novel scriptable parametric modeling interface for the Unity3D game engine which can be applied to VR-driven parametric modeling designs. By simplifying prevailing advanced programming languages such as C# and Java, we ...
Resynchronize Japanese "Geisha" Dance Video Using Music of Different Styles
CULTURECOMPUTING '13: Proceedings of the 2013 International Conference on Culture and Computing

Music and dancing are two different arts yet inseparable and which can be both powerful expression channels for a society or an artist. In most cases, the rhythm, tempo and performance of a dance depend on those of the music. This paper presents a new ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICDSP '24: Proceedings of the 2024 8th International Conference on Digital Signal Processing

February 2024

233 pages

ISBN:9798400709029

DOI:10.1145/3653876

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICDSP 2024

ICDSP 2024: 2024 8th International Conference on Digital Signal Processing

February 23 - 25, 2024

Hangzhou, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
4
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)4

Reflects downloads up to 23 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li W(2024)PM4Bag: A Scriptable Parametric Modeling Interface for Conceptual Bag Design Using PM4VR2024 8th International Conference on Imaging, Signal Processing and Communications (ICISPC)10.1109/ICISPC63824.2024.00038(178-181)Online publication date: 19-Jul-2024
https://doi.org/10.1109/ICISPC63824.2024.00038
Li W(2024)PM4WallArt: A Scriptable Parametric Modeling Interface for Conceptual Wall Art Design Using PM4VR2024 9th International Conference on Computer and Communication Systems (ICCCS)10.1109/ICCCS61882.2024.10602918(1380-1383)Online publication date: 19-Apr-2024
https://doi.org/10.1109/ICCCS61882.2024.10602918

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents