Abstract
A combined approach, involving 3D spatial datasets, noise removal prepossessing and deep learning regression approaches for the estimation of rough skeleton data, is presented in this paper. The application scenario involved data sequences from Greek traditional dances. In particular, a visualization application interface was developed allowing the user to load the C3D sequences, edit the data and remove possible noise. The interface was developed using the OpenGL language and is able to parse aby C3D format file. The interface is supported by several functionalities such as a pre-processing of the 3D point data and noise removal of 3D points that fall apart from the human skeleton. The main research innovation of this paper is the use of a deep machine learning framework through which human skeleton can be extracted. The points are selected on the use of a Convolutional Neural Network (CNN) model. Experimental results on real-life dances being captured by the Vicon motion capturing system are presented to show the great performance of the proposed scheme.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The developed user interface can be found in https://github.com/JohnCrabs/Crabs3Dv122.
References
Barbara, S.-Y., Shay, A.: The Oxford Handbook of Dance and Ethnicity. Oxford University Press, Oxford (2016)
Dimitropoulos, K., Manitsaris, S., Tsalakanidou, F., Denby, B., Buchman, L., Dupont, S., Nikolopoulos, S., Kompatsiaris, Y., Charisis, V., Hadjileontiadis, L., Pozzi, F., Cotescu, M., Ciftci, S., Katos, A., Manitsaris, A., Grammalidis, N.: A multimodal approach for the safeguarding and transmission of intangible cultural heritage: the case of i-treasures. IEEE Intell. Syst. 1–1. https://doi.org/10.1109/MIS.2018.111144858 (2018)
Doulamis, A.D., Voulodimos, A., Doulamis, N.D., Soile, S., Lampropoulos, A.: Transforming intangible folkloric performing arts into tangible choreographic digital objects: the terpsichore approach. In: International Conference on Computer Vision, Theory and Applications (VISIGRAPP), Porto, Portugal, pp. 451–460 (2017)
Zhang, Z.: Microsoft kinect sensor and its effect. IEEE Multimed. 19, 4–10 (2012)
Windolf, M., Gtzen, N., Morlock, M.: Systematic accuracy and precision analysis of video motion capturing systems-exemplified on the Vicon-460 system. J. Biomech. 41, 2776–2780 (2008)
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images, pp. 1297–1304 (2011)
Kitsikidis, A., Dimitropoulos, K., Douka, S., Grammalidis, N.: Dance analysis using multiple kinect sensors. In: VISAPP 2014—Proceedings of the 9th International Conference on Computer Vision Theory and Applications, vol. 2, pp. 789–795 (2014)
Kim, D., Kim, D.-H., Kwak, K.-C.: Classification of k-pop dance movements based on skeleton information obtained by a kinect sensor. Sensors 17, 1261 (2017)
Hisatomi, K., Katayama, M., Tomiyama, K., Iwadate, Y.: 3D archive system for traditional performing arts: application of 3D reconstruction method using graph-cuts. Int. J. Comput. Vis. 94, 78–88 (2011)
Stavrakis, E., Aristidou, A., Savva, M., Himona, S., Chrysanthou, Y.: Digitization of cypriot folk dances. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7616, pp. 404–413 (2012)
Rallis, I., Georgoulas, I., Doulamis, N., Voulodimos, A., Terzopoulos, P.: Extraction of key postures from 3D human motion data for choreography summarization. In: Proceedings of the IEEE 9th International Conference on Virtual Worlds and Games for Serious Applications, (VS-Games), pp. 94–101 (2017)
Rallis, I., Doulamis, N., Doulamis, A., Voulodimos, A., Vescoukis, V.: Spatio-temporal summarization of dance choreographies. Comput. Graph. 73, 88–101 (2018)
Elhamifar, E., Sapiro, G., Vidal, R.: See all by looking at a few: sparse modeling for finding representative objects. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1600–1607 (2012)
Wang, H.: A survey on deep neural networks for human action recognition based on skeleton information. In: Advances in Intelligent Systems and Computing, vol. 541, pp. 329–336 (2017)
Protopapadakis, E., Voulodimos, A., Doulamis, A., Camarinopoulos, S., Doulamis, N., Miaoulis, G.: Dance pose identification from motion capture data: a comparison of classifiers. Technologies 6(1), 31 (2018)
Dewan, S., Agarwal, S., Singh, N.: A deep learning pipeline for Indian dance style classification, vol. 10696 (2018)
Dimitropoulos, K., Barmpoutis, P., Kitsikidis, A., Grammalidis, N.: Classification of multidimensional time-evolving data using histograms of Grassmannian points. IEEE Trans. Circuits Syst. Video Technol. 28, 892–905 (2018)
Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E.: Deep learning for computer vision: a brief review. Comput. Intell. Neurosci. 2018, 13 pages (2018)
McReynolds, T., Blythe, D.: Advanced Graphics Programming Using OpenGL. Morgan Kaufmann Publishers Inc., San Francisco (2005)
Motion Lab systems Inc.: The C3D File Format User Guide. United States of America, 1997–2008
Alfalah, S., Chan, W., Khan, S., Falah, J., Alfalah, T., Harrison, D., Charissis, V.: Gait analysis data visualisation in virtual environment (GADV/VE). In: Proceedings of 2014 Science and Information Conference, SAI 2014, pp. 742–751 (2014)
Barre, A., Armand, S.: Biomechanical toolkit: open-source framework to visualize and process biomechanical data. Comput. Methods Programs Biomed. 114(1), 80–87 (2014)
Nguyen, T.-H., Huynh, V.-N.: A k-means-like algorithm for clustering categorical data using an information theoretic-based dissimilarity measure. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9616, pp. 115–130 (2016)
Makantasis, K., Doulamis, A., Doulamis, N., Psychas, K.: Deep learning based human behavior recognition in industrial workflows. In: Proceedings—International Conference on Image Processing, ICIP, August 2016, pp. 1609–1613 (2016)
Laptev, I., Oquab, M., Bottou, L., Sivic, J.: Is object localization for free? - weakly-supervised learning with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, pp. 685–694, June 2011
Jia, Y., Szegedy, C., Liu, W.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), Boston, Mass, USA. p. 19, June 2011
Wikipedia. Convolutional neural network. https://en.wikipedia.org/wiki/Convolutional_neural_network
Sutskever, I., Krizhevsky, A., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of the 26th Annual Conference on Neural Information Processing Systems (NIPS 2012), Lake Tahoe, Nev, USA, pp. 1097–1105, December 2012
Darrell, T., Girshick, R., Donahue, J., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 27th IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587, June 2015
Doulamis, N., Doulamis, A.: Semi-supervised deep learning for object tracking and classification, pp. 848–852 (2014)
Doulamis, N., Doulamis, A.: Fast and adaptive deep fusion learning for detecting visual objects. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7585, no. PART 3, pp. 345–354 (2012)
Acknowledgments
This work was supported by the EU H2020 TERPSICHORE project “Transforming Intangible Folkloric Performing Arts into Tangible Choreographic Digital Objects” under the grant agreement 691218.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Kavouras, I., Protopapadakis, E., Doulamis, A., Doulamis, N. (2019). Skeleton Extraction of Dance Sequences from 3D Points Using Convolutional Neural Networks Based on a New Developed C3D Visualization Interface. In: Auer, M., Tsiatsos, T. (eds) The Challenges of the Digital Transformation in Education. ICL 2018. Advances in Intelligent Systems and Computing, vol 917. Springer, Cham. https://doi.org/10.1007/978-3-030-11935-5_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-11935-5_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-11934-8
Online ISBN: 978-3-030-11935-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)