research-article

HappyFeet: Challenges in Building an Automated Dance Recognition and Assessment Tool

Authors:

Abu Zaher Md Faridee,

Sreenivasan Ramasamy Ramamurthy,

Nirmalya RoyAuthors Info & Claims

GetMobile: Mobile Computing and Communications, Volume 22, Issue 3

Pages 10 - 16

https://doi.org/10.1145/3308755.3308759

Published: 17 January 2019 Publication History

Abstract

In this paper, we discuss our experience in building an automated dance assessment tool with IMU and IoT devices and highlight the major challenges of such an endeavor. In a typical dance classroom scenario, where the students frequently outnumber their instructors, such a system can add an immense value to both parties by providing systematic breakdown of the dance moves, comparing the dance moves between the students and the instructors, and pinpointing the places for improvement in an autonomous way. Along that direction, our prototypical work, HappyFeet [1], showcases our initial attempts of developing such an intelligent Dance Activity Recognition (DAR) system. Our CNN based Body Sensor Network proves more effective (by ≈7% margin at 94.20%) at accurately recognizing the micro-steps of the dance activities than traditional feature engineering approaches. These metrics are derived by purposely evaluating the setup on a dance form known for its gentle, smooth and subtle limb movements. In this paper, we articulate how our proposed DAR framework will be generalizable for diverse dance styles involving very pronounced movements, human body kinematics and energy profiles.

References

[1]

A. Z. M. Faridee, S. Ramasamy Ramamurthy, H. M. S. Hossain, and N. Roy, "HappyFeet: Recognizing and assessing dance on the floor," in Proceedings of the 19th International Workshop on Mobile Computing Systems & Applications - HotMobile '18, 2018, pp. 49--54.

Digital Library

[2]

S. Ramasamy Ramamurthy and N. Roy, "Recent trends in machine learning for human activity recognition -- a survey," Wiley Interdiscip. Rev. Data Min. Knowl. Discov., vol. 8, no. 4, p. e1254, Jul. 2018.

[3]

M. A. U. Alam, N. Roy, A. Misra, and J. Taylor, "CACE: Exploiting behavioral interactions for improved activity recognition in multi-inhabitant smart homes," in 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS), 2016, pp. 539--548.

[4]

N. Roy, A. Misra, and D. Cook, "Ambient and smartphone sensor assisted ADL recognition in multi-inhabitant smart environments," J. Ambient Intell. Humaniz. Comput., vol. 7, no. 1, pp. 1--19, Feb. 2016.

[5]

H. M. S. Hossain, M. A. A. H. Khan, and N. Roy, "Active learning enabled activity recognition," Pervasive Mob. Comput., vol. 38, pp. 312--330, Jul. 2017.

[6]

N. Roy, A. Misra, and D. Cook, "Infrastructureassisted smartphone-based ADL recognition in multi-inhabitant smart environments," in 2013 IEEE International Conference on Pervasive Computing and Communications (PerCom), 2013, pp. 38--46.

[7]

M. A. U. Alam, N. Roy, S. Holmes, A. Gangopadhyay, and E. Galik, "Automated functional and behavioral health assessment of older adults with dementia," in 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), 2016, pp. 140--149.

[8]

M.A.H. Khan, N. Roy, and A. Misra, "Scaling human activity recognition via deep learningbased domain adaptation," 2018 IEEE Int. Conf. Pervasive Comput. Commun. Athens, Greece, March 19--23, Mar. 2018.

[9]

R. Massey, India's dances : their history, technique, and repertoire. Abhinav Publications, 2004.

[10]

A. Stisen et al., "Smart devices are different: Assessing and mitigating mobile sensing heterogeneities for activity recognition," Proc. 13th ACM Conf. Embed. Networked Sens. Syst. -- SenSys '15, 2015.

Digital Library

[11]

H. Lausberg and H. Sloetjes, "Coding gestural behavior with the NEUROGES-ELAN system," Behav. Res. Methods, vol. 41, no. 3, pp. 841--849, Aug. 2009.

[12]

V. Kuleshov, S. Z. Enam, and S. Ermon, "Audio super resolution using neural networks," Aug. 2017.

[13]

G. E. Hinton, "A practical guide to training restricted Boltzmann machines," Springer, Berlin, Heidelberg, 2012, pp. 599--619.

[14]

I. Goodfellow et al., "Generative adversarial nets." pp. 2672--2680, 2014.

Digital Library

[15]

Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, "Realtime multi-person 2D pose estimation using part affinity fields *."

[16]

R. A. Güler, N. Neverova, and I. Kokkinos, "DensePose: Dense human pose estimation in the wild."

[17]

W. Zhu et al., "Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks," Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. AAAI Press, pp. 3697--3703, 2016.

Digital Library

[18]

C. Chen, R. Jafari, and N. Kehtarnavaz, "A survey of depth and inertial sensor fusion for human action recognition," Multimed. Tools Appl., vol. 76, no. 3, pp. 4405--4425, Feb. 2017.

Digital Library

[19]

D. Picard, "Testing and estimating changepoints in time series," Adv. Appl. Probab., vol. 17, no. 04, pp. 841--867, Dec. 1985.

[20]

H. M. S. Hossain, M. A. Al Haiz Khan, and N. Roy, "DeActive: Scaling activity recognition with active deep learning," Proc. ACM Interactive, Mobile, Wearable Ubiquitous Technol., vol. 2, no. 2, pp. 1--23, Jul. 2018.

Digital Library

Cited By

Song ZWu YZhang LTao WLi LDing G(2024)Modeling of Drama Performance Intelligent Evaluation Driven by Multimodal DataGenetic and Evolutionary Computing10.1007/978-981-97-0068-4_22(220-232)Online publication date: 2-Feb-2024
https://doi.org/10.1007/978-981-97-0068-4_22
Li C(2023)Research on the identification and integration of folk dance creation elements based on big data technologyApplied Mathematics and Nonlinear Sciences10.2478/amns.2023.2.00412Online publication date: 30-Sep-2023
https://doi.org/10.2478/amns.2023.2.00412
El Raheb KBuccoli MZanoni MKatifori AKasomoulis ASarti AIoannidis Y(2023)Towards a general framework for the annotation of dance motion sequencesMultimedia Tools and Applications10.1007/s11042-022-12602-y82:3(3363-3395)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s11042-022-12602-y
Show More Cited By

Recommendations

HappyFeet: Recognizing and Assessing Dance on the Floor
HotMobile '18: Proceedings of the 19th International Workshop on Mobile Computing Systems & Applications

The widespread availability of Internet-of-Thing (IoT) devices, wearable sensors and smart watches have been promoting innovative activity recognition applications in our everyday lives. Recognizing dance steps with fine granularity using wearables is ...
HappyFeet: Embodiments for Joint Remote Dancing
GI '16: Proceedings of the 42nd Graphics Interface Conference

We present HappyFeet, a dancing system designed for supporting the dancing experience of remotely located dance partners. HappyFeet uses 3D representations of dancers' feet in a shared virtual dance space to emphasize timing and placement of feet. It ...
Voices of The Cello: Speak, Sing, Play; An Aesthetic Examination of Style Periods in The Cello Repertoire and How They Relate to The Viability of Transcription

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image GetMobile: Mobile Computing and Communications

GetMobile: Mobile Computing and Communications Volume 22, Issue 3

September 2018

34 pages

ISSN:2375-0529

EISSN:2375-0537

DOI:10.1145/3308755

Editor:
Eyal de Lara
University of Toronto

Issue’s Table of Contents

Copyright © 2019 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 January 2019

Published in SIGMOBILE-GETMOBILE Volume 22, Issue 3

Check for updates

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
282
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)2

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Song ZWu YZhang LTao WLi LDing G(2024)Modeling of Drama Performance Intelligent Evaluation Driven by Multimodal DataGenetic and Evolutionary Computing10.1007/978-981-97-0068-4_22(220-232)Online publication date: 2-Feb-2024
https://doi.org/10.1007/978-981-97-0068-4_22
Li C(2023)Research on the identification and integration of folk dance creation elements based on big data technologyApplied Mathematics and Nonlinear Sciences10.2478/amns.2023.2.00412Online publication date: 30-Sep-2023
https://doi.org/10.2478/amns.2023.2.00412
El Raheb KBuccoli MZanoni MKatifori AKasomoulis ASarti AIoannidis Y(2023)Towards a general framework for the annotation of dance motion sequencesMultimedia Tools and Applications10.1007/s11042-022-12602-y82:3(3363-3395)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s11042-022-12602-y
Jiang D(2022)Application of Rotationally Symmetrical Triangulation Stereo Vision Sensor in National Dance Movement Detection and RecognitionWireless Communications & Mobile Computing10.1155/2022/90324002022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/9032400
Groß-Vogt KSvoronos-Kanavas IWeger MAmon C(2022)The Augmented Floor - Assessing Auditory AugmentationProceedings of the 17th International Audio Mostly Conference10.1145/3561212.3561219(7-14)Online publication date: 6-Sep-2022
https://dl.acm.org/doi/10.1145/3561212.3561219
Faridee AChakma AHasan ZRoy NMisra A(2022)CoDEm: Conditional Domain Embeddings for Scalable Human Activity Recognition2022 IEEE International Conference on Smart Computing (SMARTCOMP)10.1109/SMARTCOMP55677.2022.00017(9-18)Online publication date: Jun-2022
https://doi.org/10.1109/SMARTCOMP55677.2022.00017
Li Y(2021)Dance Motion Capture Based on Data Fusion Algorithm and Wearable Sensor NetworkComplexity10.1155/2021/26562752021Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1155/2021/2656275
Ghosh IRamamurthy SRoy N(2020)StanceScorer: A Data Driven Approach to Score Badminton Player2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)10.1109/PerComWorkshops48775.2020.9156220(1-6)Online publication date: Mar-2020
https://doi.org/10.1109/PerComWorkshops48775.2020.9156220
Ali SKhan AZia SMukhtar M(2020)Human Activity Recognition System using Smart Phone based Accelerometer and Machine Learning2020 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT)10.1109/IAICT50021.2020.9172037(69-74)Online publication date: Jul-2020
https://doi.org/10.1109/IAICT50021.2020.9172037
Fomichev MMaass MHollick M(2019)Zero-Interaction Security-Towards Sound Experimental ValidationGetMobile: Mobile Computing and Communications10.1145/3372300.337230423:2(16-21)Online publication date: 14-Nov-2019
https://dl.acm.org/doi/10.1145/3372300.3372304
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents