research-article

An Approach to Ballet Dance Training through MS Kinect and Visualization in a CAVE Virtual Reality Environment

Authors:

Paisarn Muneesawang,

Ling GuanAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 6, Issue 2

Article No.: 23, Pages 1 - 37

https://doi.org/10.1145/2735951

Published: 31 March 2015 Publication History

Abstract

This article proposes a novel framework for the real-time capture, assessment, and visualization of ballet dance movements as performed by a student in an instructional, virtual reality (VR) setting. The acquisition of human movement data is facilitated by skeletal joint tracking captured using the popular Microsoft (MS) Kinect camera system, while instruction and performance evaluation are provided in the form of 3D visualizations and feedback through a CAVE virtual environment, in which the student is fully immersed. The proposed framework is based on the unsupervised parsing of ballet dance movement into a structured posture space using the spherical self-organizing map (SSOM). A unique feature descriptor is proposed to more appropriately reflect the subtleties of ballet dance movements, which are represented as gesture trajectories through posture space on the SSOM. This recognition subsystem is used to identify the category of movement the student is attempting when prompted (by a virtual instructor) to perform a particular dance sequence. The dance sequence is then segmented and cross-referenced against a library of gestural components performed by the teacher. This facilitates alignment and score-based assessment of individual movements within the context of the dance sequence. An immersive interface enables the student to review his or her performance from a number of vantage points, each providing a unique perspective and spatial context suggestive of how the student might make improvements in training. An evaluation of the recognition and virtual feedback systems is presented.

References

[1]

D. Alexiadis, P. Daras, P. Kelly, N. E. O’Connor, T. Boubekeur, and M. B. Moussa. 2011. Evaluating a dancer's performance using Kinect--based skeleton tracking. In ACM Multimedia. 659--662.

Digital Library

[2]

O. Arikan and E. Forsyth. 2002. Interactive motion generation from examples. ACM Transactions on Graphics 21, 3 (2002), 483--490.

Digital Library

[3]

C. W. Armstrong and S. J. Hoffman. 1979. Effects of teaching experience, knowledge of performer competence, and knowledge of performance outcome on performance error identification. Research Quarterly 50, 3 (1979), 318--327.

[4]

J. Barbic, A. Safonova, J. Y. Pan, C. Faloutsos, J. K. Hodgins, and N. S. Pollard. 2004. Segmenting motion capture data into distinct behaviors. In Proceedings of Graphics Interface. 185--194.

Digital Library

[5]

M. A. Barnes, D. Krasnow, S. J. Tupling, and M. Thomas. 2000. Knee rotation in classical dancers during the grand plié. Medical Problems of Performing Artists 15, 4 (2000), 140--147.

[6]

D. A. Becker and A. Pentland. 1996. Using a virtual environment to teach cancer patients T’ai Chi, relaxation, and self-imagery. In Proceedings of the International Conference on Automatic Face and Gesture Recognition.

[7]

M. Bertucco and P. Cesari. 2010. Does movement planning follow Fitts’ law&quest; Scaling anticipatory postural adjustments with movement speed and accuracy. Neuroscience 171, 1 (2010), 205--213.

[8]

A. F. Bobick and J. W. Davis. 2001. The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 3 (2001), 257--267.

Digital Library

[9]

D. Brennan and M. M. Van Hulle. 2007. Comparison of flat SOM with spherical SOM: A case study. In The Self-Organizing Maps and the Development -- From Medicine and Biology to the Sociological Field. 31--41. Springer, Tokyo, Japan, (2007).

[10]

S. Bronner and S. Ojofeitimi. 2006. Gender and limb differences in healthy elite dancers: Passé kinematics. Journal of Motor Behavior 38, 1 (2006), 71--79.

[11]

S. Bronner and S. Ojofeitimi. 2011. Pelvis and hip three-dimensional kinematics in grand battement movements. Journal of Dance Medicine and Science 15, 1 (2011), 23--30.

[12]

J. C. Chan, H. Leung, J. K. Tang, and T. Komura. 2011. A virtual reality dance training system using motion capture technology. IEEE Transactions on Learning Technologies 4, 2 (2011), 187--195.

Digital Library

[13]

P. T. Chua, R. Crivella, B. Daly, N. Hu, R. Schaaf, T. Ventura, J. Camill, J. Hodgins, and R. Pausch. 2003. Tai chi: Training for physical tasks in virtual environments. In Proceedings of IEEE Virtual Reality. 87--94.

Digital Library

[14]

A. Clay, N. Couture, and L. Nigay. 2009. Towards an architecture model for emotion recognition in interactive systems: Application to a ballet dance show. In Proceedings of the World Conference on Innovative Virtual Reality. 19--24.

[15]

A. Couillandres, P. Lewton-Brain, and P. Portero. 2008. Exploring the effects of kinesiological awareness and mental imagery on movement intention in the performance of demi-plie. Journal of Dance Medicine and Science 12, 3 (2008), 91--98.

[16]

J. L. Deckert, S. M. Barry, and T. M. Welsh. 2007. Analysis of pelvic alignment in university ballet majors. Journal of Dance Medicine and Science 11, 4 (2007), 110--117.

[17]

L. Deng, H. Leung, N. Gu, and Y. Yang. 2011. Real time mocap dance recognition for an interactive dancing game. Computer Animation and Virtual Worlds 22 (2011), 229--237.

Digital Library

[18]

K. A. Ericsson, R. T. Krampe, and C. Tesch-Roemer. 1993. The role of deliberate practice in the acquisition of expert performance. Psychological Review 100, 3 (1993), 363--406.

[19]

N. Gamboian, S. J. Chatfield, and M. H. Woollacott. 2000. Further effects of somatic training on pelvic tilt and lumbar lordosis alignment during quiet stance and dynamic dance movement. Journal of Dance Medicine and Science 4, 3 (2000), 90--98.

[20]

E. Golomer, R. M. Gravenhorst, and Y. Toussaint. 2009. Influence of vision and motor imagery styles on equilibrium control during whole-body rotations. Somatosensory and Motor Research 26, 4 (2009), 105--110.

[21]

A. O. Gonsales and M. Kyan. 2012. Trajectory analysis on spherical self-organizing maps with application to gesture recognition. In Proceedings of the 9th International Workshop (WSOM’12). 125--134.

[22]

K. Hachimura, H. Kato, and H. Tamura. 2004. A prototype dance training support system with motion capture and mixed reality technologies. In Proceedings of the IEEE International Workshop on Robot and Human Interactive Communication. 217--222.

[23]

E. Ho, J. Chan, T. Komura, and H. Leung. 2013. Interactive partner control in close interactions for real-time applications. ACM Transactions on Multimedia Computing, Communications, and Applications 9, 3 (2013), 21.

Digital Library

[24]

K. M. Holt, T. M. Welsh, and J. Speights. 2011. A within-subject analysis of the effects of remote cueing on pelvic alignment in dancers. Journal of Dance Medicine and Science 15, 1 (2011), 15--22.

[25]

A. Imura, Y. Iino, and T. Kojima. 2008. Biomechanics of the continuity and speed change during one revolution of the fouette turn. Human Movement Science 27, 6 (2008), 903--913.

[26]

A. Imura, Y. Iino, and T. Kojima. 2010. Kinematic and kinetic analysis of the fouette turn in classical ballet. Journal of Applied Biomechanics 26, 4 (2010), 484--492.

[27]

G. Kassing and M. J. Danielle. 1998. Teaching Beginning Ballet Technique. Human Kinetics.

[28]

E. Kavakli, S. Bakogianni, A. Damianakis, M. Loumou, and D. Tsatsos. 2004. Traditional dance and e-learning: The WEBDANCE learning environment. In Proceedings of the International Conference on Theory and Applications of Mathematics and Informatics. 272--281.

[29]

M. Kawashima, A. Shimada, and R.-I. Taniguchi. 2009. Early recognition of gesture patterns using sparse code of self-organising map. In Advances in Self-Organizing Maps. Springer, Berlin, 116--123.

Digital Library

[30]

E. Keogh. 2002. Exact indexing of dynamic time warping. In Proceedings of the International Conference on Very Large Data Bases. 406--417.

Digital Library

[31]

E. Keogh, T. Palpanas, V. B. Zordan, E. Gunopulos, and M. Cardle. 2004. Indexing large human-motion databases. In Proceedings of the International Conference on Very Large Data Bases. 780--791.

Digital Library

[32]

T. Komura, B. Lam, R. W. H. Lau, and H. Leung. 2006. e-Learning martial arts. Lecture Notes in Computer Science 4181 (2006), 239--248.

Digital Library

[33]

D. H. Krasnow and S. J. Chatfield. 1997. Imagery and conditioning practices for dancers. Dance Research Journal 29, 1 (1997), 43--64.

[34]

K. Kulig, A. L. Fietzer, and J. M. Popovich. 2011. Ground reaction forces and knee mechanics in the weight acceptance phase of a dance leap take-off and landing. Journal of Sports Sciences 29, 2 (2011), 125--131.

[35]

C. F. Lin, F. C. Su, and H. W. Wu. 2005. Ankle biomechanics of ballet dancers in relevé en pointé dance. Research in Sports Medicine 13, 1 (2005), 23--35.

[36]

F. Lv, R. Nevatia, and M. W. Lee. 2005. 3D human action recognition using spatio-temporal motion templates. In Computer Vision in Human-Computer Interaction. Springer, Berlin, 120--130.

Digital Library

[37]

N. Masso, A. Germain, F. Rey, L. Costa, D. Romero, and S. Guitart. 2004. Study of muscle activity during releve in first position and sixth positions. Journal of Dance Medicine and Science 8, 4 (2004), 101--107.

[38]

L. Mayers, S. Bronner, S. Agraharasamakulam, and S. Ojofeitimi. 2010. Lower extremity kinetics in tap dance. Journal of Dance Medicine and Science 14, 1 (2010), 3--10.

[39]

J. D. McAuley and M. R. Jones. 2003. Modeling effects of rhythmic context on perceived duration: A comparison of interval and entrainment approaches to short-interval timing. Journal of Experimental Psychology: Human Perception and Performance 29 (2003), 1102--1125.

[40]

M. Naemura and M. Suzuki. 2006. A method for estimating dance action based on motion analysis. In Computer Vision and Graphics. Springer Netherlands, 695--702.

[41]

J. G. Noverre. 1760. Letters on Dancing and Ballet, translated by Cyril W. Beaumont. London 1830, Reproduced, New York Dance Horizons.

[42]

M. Raptis, D. Kirovski, and H. Hoppe. 2011. Real-time classification of dance gestures from skeleton animation. In Proceedings of ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 147--156.

Digital Library

[43]

A. P. Sangole and A. Leontitis. 2006. Spherical self-organizing feature map: An introductory review. International Journal of Bifurcation and Chaos 16, 11 (2006), 3195--3206.

[44]

C. Schuldt, I. Laptev, and B. Caputo. 2004. Recognizing human actions: A local SVM approach. In Proceedings of the IEEE International Conference on Pattern Recognition. 32--36.

Digital Library

[45]

G. Shan. 2005. Comparison of repetitive movements between ballet dancers and martial artists: Risk assessment of muscle overuse injuries and prevention strategies. Research in Sports Medicine 13, 1 (2005), 63--76.

[46]

J. M. Shippen and B. May. 2010. Calculation of muscle loading and joint contact forces during the rock step in Irish dance. Journal of Dance Medicine and Science 14, 1 (2010), 11--18.

[47]

R. W. Simmons. 2005. Sensory organization determinants of postural stability in trained ballet dancers. International Journal of Neuroscience 115, 1 (2005), 87--97.

[48]

J. K. T. Tang, H. Leung, T. Komura, and H. P. H. Shum. 2008. Emulating human perception of motion similarity. Computer Animation and Virtual Worlds 19, 3--4 (2008), 211--221.

Digital Library

[49]

M. Uejou, H.-H. Huang, J.-H. Lee, and K. Kawagoe. 2011. Toward a conversational virtual instructor of ballroom dance. In Intelligent Virtual Agents. Springer, Berlin, 477--478.

Digital Library

[50]

R. E. Ward. 2012. Biomechanical Perspectives on Classical Ballet Technique and Implications for Teaching Practice. PhD Thesis, University of New South Wales.

[51]

M. Wilson, J.-H. Ryu, and Y.-H. Kwon. 2007. Contribution of the pelvis to gesture leg range of motion in a complex ballet movement grand rond de jambe en l'air en dehors. Journal of Dance Medicine and Science 11, 4 (2007), 118--123.

[52]

U. Yang and G. J. Kim. 2002. Implementation and evaluation of “just follow me”: An immersive, VR-based, motion-training system. Presence: Teleoperators and Virtual Environments 11, 3 (2002), 304--323.

Digital Library

[53]

Y. Yang, H. Leung, L. Yue, and L. Deng. 2012. Automatic dance lesson generation. IEEE Transactions on Learning Technologies 3, 5 (2012), 191--198.

Digital Library

[54]

Y. Yang, H. Leung, L. Yue, and L. Deng. 2013. Generating a two-phase lesson for guiding beginners to learn basic dance movements. Computers and Education 61, 1 (2013), 1--20.

Digital Library

Cited By

Cai WLiu W(2024)Innovative Strategies of Virtual Reality Technology in Ethnic Dance InheritanceApplied Mathematics and Nonlinear Sciences10.2478/amns-2024-08449:1Online publication date: 1-Apr-2024
https://doi.org/10.2478/amns-2024-0844
Mizukami HNiijima APark COgawa T(2024)Exploring the Impact of Size and Position on Visual Feedback Efficacy for Ballet DancersCompanion Proceedings of the 2024 Conference on Interactive Surfaces and Spaces10.1145/3696762.3698052(54-57)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3696762.3698052
Lee SHwang SOakley ILee K(2024)Expanding the Design Space of Vision-based Interactive Systems for Group Dance PracticeProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661568(2768-2787)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661568
Show More Cited By

Index Terms

An Approach to Ballet Dance Training through MS Kinect and Visualization in a CAVE Virtual Reality Environment
1. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

An Advanced Computational Intelligence System for Training of Ballet Dance in a Cave Virtual Reality Environment
ISM '14: Proceedings of the 2014 IEEE International Symposium on Multimedia

This paper presents a computer-based system for assessment and training of ballet dance in a CAVE virtual reality environment. The system utilizes Kinect sensor to capture student's dance and extracts features from skeleton joints. This system depends ...
CAVE and Fishtank Virtual-Reality Displays: A Qualitative and Quantitative Comparison

We present the results from a qualitative and quantitative user study comparing fishtank virtual-reality (VR) and CAVE displays. The results of the qualitative study show that users preferred the fishtank VR display to the CAVE system for our scientific ...
Thinning trainer based on forest-growth model, virtual reality and computer-aided virtual environment

Immersive virtual reality is applied in many human activities. This virtual reality can be used as a training tool for thinning operations in forests. The aim of this study is to describe the complex solution of Thinning Trainer, that we developed. This ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 6, Issue 2

Special Section on Visual Understanding with RGB-D Sensors

May 2015

381 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/2753829

Editor:
Huan Liu
Arizona State University

Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 March 2015

Accepted: 01 January 2014

Revised: 01 November 2013

Received: 01 July 2013

Published in TIST Volume 6, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

92
Total Citations
View Citations
1,531
Total Downloads

Downloads (Last 12 months)161
Downloads (Last 6 weeks)18

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cai WLiu W(2024)Innovative Strategies of Virtual Reality Technology in Ethnic Dance InheritanceApplied Mathematics and Nonlinear Sciences10.2478/amns-2024-08449:1Online publication date: 1-Apr-2024
https://doi.org/10.2478/amns-2024-0844
Mizukami HNiijima APark COgawa T(2024)Exploring the Impact of Size and Position on Visual Feedback Efficacy for Ballet DancersCompanion Proceedings of the 2024 Conference on Interactive Surfaces and Spaces10.1145/3696762.3698052(54-57)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3696762.3698052
Lee SHwang SOakley ILee K(2024)Expanding the Design Space of Vision-based Interactive Systems for Group Dance PracticeProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661568(2768-2787)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661568
Hassan SDe Lacerda Pataca CNourian LTigwell GDavis BSilver Wagman W(2024)Designing and Evaluating an Advanced Dance Video Comprehension Tool with In-situ Move Identification CapabilitiesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642710(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642710
Laattala MPiitulainen RAdy NTamariz MHämäläinen P(2024)WAVE: Anticipatory Movement Visualization for VR DancingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642145(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642145
Yu XLee BSedlmair M(2024)Design Space of Visual Feedforward And Corrective Feedback in XR-Based Motion Guidance SystemsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642143(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642143
Cao CLu Y(2024)OnStage: Design and Development of a Performance and Practice Simulation System2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00246(899-900)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00246
Tokida SIshiguro Y(2024)Enhancing Seamless Body Movement Learning with Frame-by-Frame VR Presentation2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00117(604-607)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00117
Wang LGai WJiang NChen GBian YLuan HHuang LYang C(2024)Effective Motion Self-Learning Genre Using 360° Virtual Reality Content on Mobile Device: A Study Based on Taichi Training PlatformIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2024.337147028:6(3695-3708)Online publication date: Jun-2024
https://doi.org/10.1109/JBHI.2024.3371470
Limmanee APairoch SPhothisonothai M(2024)CS-TE Abstraction Framework: Ballet Teaching, Training, and Testing Station with Kinect Sensors and EEG Measurement2024 Tenth International Conference on Communications and Electronics (ICCE)10.1109/ICCE62051.2024.10634700(582-587)Online publication date: 31-Jul-2024
https://doi.org/10.1109/ICCE62051.2024.10634700
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents