short-paper

Real-Time Optical Motion Capture Balance Sonification System

Authors:

Mitchell Tillman,

Christopher B. Knowlton,

Antonia ZaferiouAuthors Info & Claims

MOCO '20: Proceedings of the 7th International Conference on Movement and Computing

Article No.: 25, Pages 1 - 4

https://doi.org/10.1145/3401956.3404244

Published: 15 July 2020 Publication History

Abstract

In this study, we explored the effects of a motion capture-based real-time sonified biofeedback system on balance. We present the initial efforts towards developing a task-independent optical motion capture based real-time balance sonification system. Five healthy young adults (two female; 24 ± 2.65 years) stood on one foot before and during listening to sonified biofeedback that expressed information in real-time about the state of their balance. In two of five participants, interacting with our sonified biofeedback system resulted in increased "Margin of Stability", a metric indicative of how well the body center of mass is supported by a person's stance. This result indicates our system's initial promise towards training balance strategies. Qualitatively, the participants who increased the Margin of Stability during sonification reported enjoying the experience more and were more aware of changes in their behavior, compared to those who did not increase their Margin of Stability. We also learned that our sonification system has design elements that are incompatible with the stationary tasks in the present study, which will inform our next iteration of sonification design. Future work will examine sonifying balance in dynamic balance tasks, with the goal of aiding clinical balance training.

References

[1]

Stevens, J. A., Mack, K. A., Paulozzi, L. J., & Ballesteros, M. F. (2008). Self-Reported Falls and Fall-Related Injuries Among Persons Aged ≥ 65 Years-United States, 2006. Journal of Safety Research.

[2]

Gorgas, A. M., Schön, L., Dlapka, R., Doppler, J., Iber, M., Gradl, C., ... Horsak, B. (2017). Short-Term Effects of Real-Time Auditory Display (Sonification) on Gait Parameters in People with Parkinsons' Disease---A Pilot Study. In Biosystems and Biorobotics.

[3]

Hasegawa, S., Ishijima, S., Kato, F., Mitake, H., & Sato, M. (2012). Realtime sonification of the center of gravity for skiing. ACM International Conference Proceeding Series.

Digital Library

[4]

Costantini, G., Casali, D., Todisco, M., Saggio, G., Giansanti, D., & Maccioni, G. (2015). Towards the improvement of postural stability through audio biofeedback. WISP 2015 - IEEE International Symposium on Intelligent Signal Processing, Proceedings, (June).

[5]

Fleszar, Z., Mellone, S., Giese, M., Tacconi, C., Becker, C., Schöls, L., ... Ilg, W. (2019). Real-time use of audio-biofeedback can improve postural sway in patients with degenerative ataxia. Annals of Clinical and Translational Neurology, 6(2), 285--294.

[6]

Scholz, D. S., Rohde, S., Nikmaram, N., Brückner, H. P., Grobbach, M., Rollnik, J. D., & Altenmüller, E. O. (2016). Sonification of arm movements in stroke rehabilitation - a novel approach in neurologic music therapy. Frontiers in Neurology, 7(JUN), 1--7.

[7]

Mirelman, A., Herman, T., Nicolai, S., Zijlstra, A., Zijlstra, W., Becker, C., ... Hausdorff, J. M. (2011). Audio-biofeedback training for posture and balance in patients with Parkinson's disease. Journal of NeuroEngineering and Rehabilitation.

[8]

Nicolai, S., Mirelman, A., Herman, T., Zijlstra, A., Mancini, M., Becker, C., ... Maetzler, W. (2010). Improvement of balance after audio-biofeedback. A 6-week intervention study in patients with progressive supranuclear palsy. Zeitschrift Fur Gerontologie Und Geriatrie.

[9]

Dubus, G. (2012). Evaluation of four models for the sonification of elite rowing. Journal on Multimodal User Interfaces.

[10]

Chow, D. H. K., & Cheng, C. T. K. (2000). Quantitative analysis of the effects of audio biofeedback on weight-bearing characteristics of persons with transtibial amputation during early prosthetic ambulation. Journal of Rehabilitation Research and Development, 37(3), 255--260.

[11]

Schaffert, N., Janzen, T. B., Mattes, K., & Thaut, M. H. (2019). A review on the relationship between sound and movement in sports and rehabilitation. Frontiers in Psychology.

[12]

Dahl, L., Knowlton, C.B., Zaferiou, A. (2019). Developing real-time sonification with optical motion capture to convey balance-related metrics to dancers. ACM International Conference Proceeding Series, 1

Digital Library

[13]

Hof, A. L., Gazendam, M. G. J., & Sinke, W. E. (2005). The condition for dynamic stability. Journal of Biomechanics.

[14]

Cliff, N. (2014). Ordinal Methods for Behavioral Data Analysis. In Ordinal Methods for Behavioral Data Analysis.

[15]

De Leva, P. (1996). Adjustments to Zatsiorsky-Seluyanov's segment inertia paremeters. Journal of Biomechanics, 29(9), 1223--1230.

[16]

Bevilacqua, F., Peyre, I., Segalen, M., Pradat-Diehl, P., Marchand-Pauvert, V., & Roby-Brami, A. (2018). Exploring different movement sonification strategies for rehabilitation in clinical settings. ACM International Conference Proceeding Series.

Digital Library

[17]

Effenberg, A. O. (2005). Movement sonification: Effects on perception and action. IEEE Multimedia.

[18]

Cook, P. R. (2011). Sound synthesis and auditory display. In The Sonification Handbook.

[19]

Neuhoff, J. G. (2011). Perception, Cognition and Action in Auditory Displays. The Sonification Handbook.

[20]

Walker, B., & Nees, M. A. (2011). Theory of Sonification. In The Sonification Handbook.

[21]

Dyer, J. F., Stapleton, P., & Rodger, M. W. M. (2017). Advantages of melodic over rhythmic movement sonification in bimanual motor skill learning. Experimental Brain Research, 235(10), 3129--3140.

[22]

Dyer, J. F., Stapleton, P., & W. M. Rodger, M. (2016). Sonification as Concurrent Augmented Feedback for Motor Skill Learning and the Importance of Mapping Design. The Open Psychology Journal, 8(1), 192--202.

Cited By

Dotov DMotsenyat ATrainor L(2024)Concurrent Supra-Postural Auditory–Hand Coordination Task Affects Postural Control: Using Sonification to Explore Environmental Unpredictability in Factors Affecting Fall RiskSensors10.3390/s2406199424:6(1994)Online publication date: 21-Mar-2024
https://doi.org/10.3390/s24061994
Zaferiou A(2023)Dance-Themed National Biomechanics Day Community Engagement to Inspire our Future STEAM LeadersJournal of Biomechanics10.1016/j.jbiomech.2023.111511150(111511)Online publication date: Mar-2023
https://doi.org/10.1016/j.jbiomech.2023.111511

Index Terms

Real-Time Optical Motion Capture Balance Sonification System
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction devices
      1. Sound-based input / output

Recommendations

Developing real-time sonification with optical motion capture to convey balance-related metrics to dancers
MOCO '19: Proceedings of the 6th International Conference on Movement and Computing

This paper presents initial efforts in developing and evaluating a system for real-time sonification of balance during dance and other dynamic real-world movements. The system uses marker-based motion capture to calculate multiple balance metrics, which ...
Aesthetic strategies in sonification

Sound can be listened to in various ways and with different intentions. Multiple factors influence how and what we perceive when listening to sound. Sonification, the acoustic representation of data, is in essence just sound. It functions as ...
Evaluation of psychoacoustic sound parameters for sonification
ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal Interaction

Sonification designers have little theory or experimental evidence to guide the design of data-to-sound mappings. Many mappings use acoustic representations of data values which do not correspond with the listener's perception of how that data value ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

MOCO '20: Proceedings of the 7th International Conference on Movement and Computing

July 2020

205 pages

ISBN:9781450375054

DOI:10.1145/3401956

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Rutgers University: Rutgers University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

MOCO '20

MOCO '20: 7th International Conference on Movement and Computing

July 15 - 17, 2020

NJ, Jersey City/Virtual, USA

Acceptance Rates

Overall Acceptance Rate 85 of 185 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
140
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)1

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dotov DMotsenyat ATrainor L(2024)Concurrent Supra-Postural Auditory–Hand Coordination Task Affects Postural Control: Using Sonification to Explore Environmental Unpredictability in Factors Affecting Fall RiskSensors10.3390/s2406199424:6(1994)Online publication date: 21-Mar-2024
https://doi.org/10.3390/s24061994
Zaferiou A(2023)Dance-Themed National Biomechanics Day Community Engagement to Inspire our Future STEAM LeadersJournal of Biomechanics10.1016/j.jbiomech.2023.111511150(111511)Online publication date: Mar-2023
https://doi.org/10.1016/j.jbiomech.2023.111511

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents