abstract

JamGesture: an improvisation support system based on physical gesture observed with smartphone

Authors:

Souta Mizuno,

Tetsuro Kitahara,

Shun Shiramatsu,

Shugo IchinoseAuthors Info & Claims

VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology

Article No.: 101, Pages 1 - 2

https://doi.org/10.1145/3281505.3283380

Published: 28 November 2018 Publication History

Get Access

Abstract

The physical gestures promote musical comprehension because they can provide visual information of musical performance for others. Melodic outlines especially have a high affinity with intuitive physical gestures. We propose methods for recognizing physical gestures using motion sensor cameras and smartphone sensors, and we have developed an improvisation support system, JamGesture, by integrating a method for recognizing physical gestures using smartphones and JamSketch, a system for melody generation based on melodic outlines. JamGesture enables users to improvise music by using the input from their intuitive physical gestures with the melody-generation function of JamSketch.

Supplementary Material

MP4 File (a101-mizuno.mp4)

Download
4.62 MB

References

[1]

Souta Mizuno, Shugo Ichinose, Shun Shiramatsu, Tetsuro Kitahara, "Support System of Improvisational Ensemble Based on User's Motion Using Smartphone Sensors" in Proceedings of The Twelfth 2017 International Conference on Knowledge, Information and Creativity Support Systems, (2017).

Google Scholar

[2]

Tetsuro Kitahara, Sergio Giraldo, Rafael Ramirez, "JamSketch: Improvisation Support System with GA-based Melody Creation from User's Drawing," Proceedings of the 13th International Symposium on ComputerMusic Multidisciplinary Research, pp. 352--363, (2017).

Google Scholar

[3]

Serge de Laubier, "The Meta-Instrument," Computer Music Journal, 22 (1), pp. 25--29, (1998).

Crossref

Google Scholar

[4]

M. Mandanici, S. Canazza, "The "Hand Composer": Gesture-driven Music Composition Machines," in Proceedings of Post-Conference Workshop NRF-IAS-2014, (2014).

Google Scholar

[5]

P. Rothenberg, "Kagura shows what the future of music could be like", Tech in Asia, (2016) https://www.techinasia.com/kaglloura-kickstarterfuture-music, (accessed on 2017-06-2).

Google Scholar

Index Terms

JamGesture: an improvisation support system based on physical gesture observed with smartphone
1. Applied computing
  1. Arts and humanities
    1. Sound and music computing
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. Usability testing

Recommendations

ImproViz: visual explorations of jazz improvisations
CHI EA '05: CHI '05 Extended Abstracts on Human Factors in Computing Systems

ImproViz is a visualization technique for diagramming music that brings to light the signature patterns of a jazz musician's improvisational style. ImproViz consists of two parts: (1) melodic landscapes show the general contours of musical phrasing; and ...
Shimon: an interactive improvisational robotic marimba player
CHI EA '10: CHI '10 Extended Abstracts on Human Factors in Computing Systems

Shimon is an autonomous marimba-playing robot designed to create interactions with human players that lead to novel musical outcomes. The robot combines music perception, interaction, and improvisation with the capacity to produce melodic and harmonic ...
Beat my bass, pluck my drum
DIS Companion '14: Proceedings of the 2014 companion publication on Designing interactive systems

Beating a bass, plucking a drum -- new systems of instruments make it possible. In this paper we describe recent research into networked musical instruments for group improvisation; instruments that reciprocally influence each other's behaviour, making, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology

November 2018

570 pages

ISBN:9781450360869

DOI:10.1145/3281505

Editor:
Stephen N. Spencer
University of Washington
,
General Chair:
Shigeo Morishima
Waseda University
,
Program Chairs:
Yuichi Itoh
Osaka University
,
Takaaki Shiratori
Facebook Reality Labs
,
Yonghao Yue
The University of Tokyo
,
Rob Lindeman
University of Canterbury

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 November 2018

Check for updates

Author Tags

Qualifiers

Abstract

Conference

VRST '18

Sponsor:

VRST '18: 24th ACM Symposium on Virtual Reality Software and Technology

November 28 - December 1, 2018

Tokyo, Japan

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
62
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)1

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Index Terms

Recommendations

ImproViz: visual explorations of jazz improvisations

Shimon: an interactive improvisational robotic marimba player

Beat my bass, pluck my drum