Abstract
Yoga is a centuries-old style of exercise followed by sports personnel, patients, and physiotherapist as their regime. A correct posture and technique are the key points in yoga to reap the maximum benefits. Hence, developing a model to classify yoga postures correctly is a recently emerging research topic. The paper presents a novel architecture that aims to classify various yoga poses. The proposed model estimates and classifies yoga poses into five broad categories with low latency. In the proposed architecture, the images are skeletonized before inputting into the model. The skeletonization process is done using the MediaPipe library for body keypoint detection. The paper compares the performance of various deep learning models with and without skeletonization. Different learning models showed the optimum result with the training of skeletonized images to the network. The comparison is drawn to establish the positive impact of skeletonization on the results obtained by various models. VGG16 achieves the highest validation accuracy on non-skeletonized images (95.6%), followed by InceptionV3, NASNetMobile, YogaConvo2d (proposed model) (89.9%), and lastly, InceptionResNetV2. In contrast, the proposed model YogaConvo2d using skeletonized images reports a validation accuracy of 99.62%, followed by VGG16, InceptionResNetV2, NASNetMobile, and InceptionV3.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Agrawal Y, Shah Y, Sharma A (2020) Implementation of machine learning technique for identification of yoga poses. In: 2020 IEEE 9th international conference on communication systems and network technologies (CSNT), pp 40–43. https://doi.org/10.1109/CSNT48778.2020.9115758
Anilkumar A, Athulya KT, Sajan S, Sreeja KA (2021) Pose estimated yoga monitoring system. Available at SSRN 3882498. https://doi.org/10.2139/ssrn.3882498
Beddiar DR, Oussalah M, Nini B (2022) Fall detection using body geometry and human pose estimation in video sequences. J vis Commun Image Represent 82:103407. https://doi.org/10.1016/j.jvcir.2021.103407
Byeon YH, Lee JY, Kim DH, Kwak KC (2020) Posture recognition using ensemble deep models under various home environments. Appl Sci 10(4):1287. https://doi.org/10.3390/app10041287
Cao Z, Hidalgo G, Simon T, Wei SE, Sheikh Y (2021) OpenPose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans Pattern Anal Mach Intell 43(1):172–186. https://doi.org/10.1109/TPAMI.2019.2929257
Chen HT, He YZ, Hsu CC, Chou CL, Lee SY, Lin BSP (2014) Yoga posture recognition for self-training. In: Gurrin C, Hopfgartner F, Hurst W, Johansen H, Lee H, O’Connor N (eds) MultiMedia modeling. MMM 2014. Lecture notes in computer science. Springer, Cham, p 8325. https://doi.org/10.1007/978-3-319-04114-8_42
Chen HT, He YZ, Hsu CC (2018) Computer-assisted yoga training system. Multimed Tools Appl 77(18):23969–23991. https://doi.org/10.1007/s11042-018-5721-2
Jain S, Rustagi A, Saurav S, Saini R, Singh S (2021) Three-dimensional CNN-inspired deep learning architecture for yoga pose recognition in the real-world environment. Neural Comput Appl 33(12):6427–6441. https://doi.org/10.1007/s00521-020-05405-5
Jose J, Shailesh S (2021) Yoga asana identification: a deep learning approach. In IOP Conf Ser: Mater Sci Eng 1110(1):012002. https://doi.org/10.1088/1757-899x/1110/1/012002
Kale G, Patil V, Munot M (2021) A novel and intelligent vision-based tutor for yogāsana: e-YogaGuru. Mach vis Appl 32(1):1–17. https://doi.org/10.1007/s00138-020-01141-x
Khanal H, Khanal U (2021) Benefits, barriers and determinants of practicing yoga: a cross sectional study from Kathmandu, Nepal. J Ayurveda Integr Med 12(1):102–106. https://doi.org/10.1016/j.jaim.2021.01.007
Kim Y, Kim D (2020) A CNN-based 3D human pose estimation based on projection of depth and ridge data. Pattern Recogn 106:107462. https://doi.org/10.1016/j.patcog.2020.107462
Kothari S (2020) Yoga pose classification using deep learning. Master's project, San Jose State University. https://scholarworks.sjsu.edu/etd_projects/932
Kovačič T, Kovačič M (2011) Impact of relaxation training according to yoga in daily life® system on perceived stress after breast cancer surgery. Integr Cancer Ther 10(1):16–26. https://doi.org/10.1177/1534735410387418
Kutálek J, Kutálek K (2021) Detection of Yoga Poses in Image and Video. Brno Fcaulty University of Information and Technology
Lamas A, Tabik S, Montes AC, Pérez-Hernández F, García J, Olmos R, Herrera F (2022) Human pose estimation for mitigating false negatives in weapon detection in video-surveillance. Neurocomputing. https://doi.org/10.1016/j.neucom.2021.12.059
Längkvist M, Karlsson L, Loutfi A (2014) Inception-v4, inception-ResNet and the impact of residual connections on learning. Pattern Recogn Lett 42:11–24
Liaqat S, Dashtipour K, Arshad K, Assaleh K, Ramzan N (2021) A hybrid posture detection framework: integrating machine learning and deep neural networks. IEEE Sens J 21(7):9515–9522. https://doi.org/10.1109/JSEN.2021.3055898
Long C, Jo E, Nam Y (2022) Development of a yoga posture coaching system using an interactive display based on transfer learning. J Supercomput 78(4):5269–5284. https://doi.org/10.1007/s11227-021-04076-w
Luo Z et al (2011) Left arm up! Interactive yoga training in virtual environment. In: 2011 IEEE virtual reality conference, pp, 261–262. IEEE. https://doi.org/10.1109/VR.2011.5759498
Malek S, Rossi S (2021) Head pose estimation using facial-landmarks classification for children rehabilitation games. Pattern Recogn Lett 152:406–412. https://doi.org/10.1016/j.patrec.2021.11.002
Narayanan SS, Misra DK, Arora K, Rai H (2021) Yoga pose detection using deep learning techniques. SSRN Electron J. https://doi.org/10.2139/ssrn.3842656
Patil S, Pawar A, Peshave A, Ansari AN, Navada A (2011) Yoga tutor visualization and analysis using SURF algorithm. In: 2011 IEEE control and system graduate research colloquium, pp 43–46 IEEE. https://doi.org/10.1109/ICSGRC.2011.5991827
Patki A (2021) Review of artificial intelligence system for correcting exercise movements and health monitoring. Int J Res Appl Sci Eng Technol 9(9):1615–1620. https://doi.org/10.22214/ijraset.2021.38228
Pismenskova M, Balabaeva O, Voronin V, Fedosov V (2017) Classification of a two-dimensional pose using a human skeleton. MATEC Web Conf EDP Sci 132:05016. https://doi.org/10.1051/matecconf/201713205016
Rector K, Bennett CL, Kientz JA (2013) Eyes-free yoga: an exergame using depth cameras for blind & low vision exercise. In: Proceedings of the 15th international acm sigaccess conference on computers and accessibility, pp 1–8. https://doi.org/10.1145/2513383.2513392
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Sukarsa IN (2018) Asana yoga meditation as a spiritual development ananda marga ashram denpasar (perspectives theology hindu). Vidyottama Sanatana: Int J Hindu Sci Relig Stud 2(2):301–306. https://doi.org/10.25078/ijhsrs.v2i2.632
Verma M, Kumawat S, Nakashima Y, Raman S (2020) Yoga-82: a new dataset for fine-grained classification of human poses. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 1038–1039
Wang H (2021) Neural network-oriented big data model for yoga movement recognition. Comput Intell Neurosci 2021:4334024. https://doi.org/10.1155/2021/4334024
Wu W, Yin W, Guo F (2010) Learning and self-instruction expert system for yoga. In: 2010 2nd international workshop on intelligent systems and applications, pp 1–4. https://doi.org/10.1109/IWISA.2010.5473592
Zeng N, Wu P, Wang Z, Li H, Liu W, Liu X (2022) A small-sized object detection oriented multi-scale feature fusion approach with application to defect detection. IEEE Trans Instrum Meas 71:1–14. https://doi.org/10.1109/TIM.2022.3153997
Zhang Y, Mi S, Wu J, Geng X (2020) Simultaneous 3D hand detection and pose estimation using single depth images. Pattern Recogn Lett 140:43–48. https://doi.org/10.1016/j.patrec.2020.09.026
Zoph B, Vasudevan V, Shlens J, Le QV (2018) Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8697–8710. https://doi.org/10.1109/CVPR.2018.00907z
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflicts of interest in this research work.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Garg, S., Saxena, A. & Gupta, R. Yoga pose classification: a CNN and MediaPipe inspired deep learning approach for real-world application. J Ambient Intell Human Comput 14, 16551–16562 (2023). https://doi.org/10.1007/s12652-022-03910-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-022-03910-0