Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Laban descriptors for gesture recognition and emotional analysis

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, we introduce a new set of 3D gesture descriptors based on the laban movement analysis model. The proposed descriptors are used in a machine learning framework (with SVM and different random forest techniques) for both gesture recognition and emotional analysis purposes. In a first experiment, we test our expressivity model for action recognition purposes on the Microsoft Research Cambridge-12 dataset and obtain very high recognition rates (more than 97 %). In a second experiment, we test our descriptors’ ability to qualify the emotional content, upon a database of pre-segmented orchestra conductors’ gestures recorded in rehearsals. The results obtained show the relevance of our model which outperforms results reported in similar works on emotion recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. McNeill, D.: Language and gesture. Cambridge University Press, Cambridge (2000)

    Book  Google Scholar 

  2. Wang, Y., Huang, K., Tan, T.: Human activity recognition based on r transform. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR’07, IEEE, pp. 1–8 (2007)

  3. Huang, F., Xu, G.: Viewpoint insensitive action recognition using envelop shape. In: Computer Vision-ACCV 2007. Springer, Berlin Heidelberg, pp. 477–486 (2007)

  4. Song, Y., Demirdjian, D., Davis, R.: Tracking body and hands for gesture recognition: Natops aircraft handling signals database. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011). IEEE, pp. 500–506 (2011)

  5. Balomenos, T., et al.: Emotion analysis in man-machine interaction systems. Machine learning for multimodal interaction, pp. 318–328 (2005)

  6. Camurri, A., Mazzarino, B., Ricchetti, M., Timmers, R., Volpe, G.: Multimodal analysis of expressive gesture in music and dance performances. In: Gesture-based communication in human-computer interaction, pp. 20–39 (2004)

  7. Gunes, H., Piccardi, M.: Bi-modal emotion recognition from expressive face and body gestures. J. Netw. Comput. Appl. 30(4), 1334–1345 (2007)

    Article  Google Scholar 

  8. Nicolaou, M.A., Gunes, H., Pantic, M.: Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. IEEE Trans. Affect. Comput. 2(2), 92–105 (2011)

    Article  Google Scholar 

  9. Swaminathan, D., et al.: A dynamic bayesian approach to computational laban shape quality analysis. Adv. Hum. Comput. Interact. pp. 1–17 (2009)

  10. Zhao, L., Badler, N.I.: Acquiring and validating motion qualities from live limb gestures. Graph. Models 67(1), 1–16 (2005)

    Article  Google Scholar 

  11. Samadani, AA., Burton, S., Gorbet, R., Kulic, D.: Laban effort and shape analysis of affective hand and arm movements. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), IEEE, pp. 343–348 (2013)

  12. Braem, P.B., Bräm, T.: A pilot study of the expressive gestures used by classical orchestra conductors. J. Conduct. Guild 22(1–2), 14–29 (2001)

    Google Scholar 

  13. Glowinski, Donald, et al.: Toward a minimal representation of affective gestures. IEEE Trans. Affect. Comput. 2(2), 106–118 (2011)

    Article  MathSciNet  Google Scholar 

  14. Boutet, D.: Une morphologie de la gestualité: structuration articulaire. Cahiers de linguistique analogique, vol. 5, pp. 81–115 (2008)

  15. Luo, P., Neff, M.: A perceptual study of the relationship between posture and gesture for virtual characters. In: Motion in Games. Springer, Berlin, pp. 254–265 (2012)

  16. Laban, R.: La Maîtrise du Mouvement. Actes Sud, Arles (1994)

    Google Scholar 

  17. Junejo, I.N., Junejo, K.N., Al Aghbari, Z.: Silhouette-based human action recognition using SAX-Shapes. Vis. Comput. 30(3), 259–269 (2014)

    Article  Google Scholar 

  18. Chen, H.S., Chen, H.T., Chen, Y.W., Lee, S.Y.: Human action recognition using star skeleton. In: Proceedings of the 4th ACM international workshop on Video surveillance and sensor networks. ACM, pp. 171–178 (2006)

  19. Cimen, G., Ilhan, H., Capin, T., Gurcay, H.: Classification of human motion based on affective state descriptors. Comput. Animat. Virtual Worlds 24(3–4), 355–363 (2013)

    Article  Google Scholar 

  20. Etemad, S.A., Arya, A.: Correlation-optimized time warping for motion. Vis. Comput. (2014)

  21. Singh, V.K., Nevatia, R.: Simultaneous tracking and action recognition for single actor human actions. Vis. Comput. 27(12), 1115–1123 (2011)

    Article  Google Scholar 

  22. Kellokumpu, V., Zhao, G., Pietikäinen, M.: Human activity recognition using a dynamic texture based method. BMVC, pp. 1–10 (2008)

  23. Blank, M., Gorelick, L., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. In: Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005, IEEE, vol. 2, pp. 1395–1402 (2005)

  24. Jiang, X., Zhong, F., Peng, Q., Qin, X.: Online robust action recognition based on a hierarchical model. Vis. Comput. 30, 1021–1033 (2014)

    Article  Google Scholar 

  25. Oikonomopoulos, A., Patras, I., Pantic, M.: Spatiotemporal salient points for visual recognition of human actions. IEEE Trans. Syst. Man Cybern. Part B Cybern. 36(3), 710–719 (2005)

    Article  Google Scholar 

  26. Rapantzikos, K., Avrithis, Y., Kollias, S.: Dense saliency-based spatiotemporal feature points for action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2009, IEEE, pp. 1454–1461 (2009)

  27. Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: a local SVM approach. In: Proceedings of the 17th International Conference on Pattern Recognition. ICPR 2004, IEEE, vol. 3, pp. 32–36 (2004)

  28. Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2008, IEEE, pp. 1–8 (2008)

  29. Kaâniche, M., Brémont, F.: Recognizing gestures by learning local motion signatures of HOG descriptors. IEEE Trans. Pattern Mach. Intell. (2012)

  30. Li, Y., Ye, J., Wang, T., Huang, S.: Augmenting bag-of-words: a robust contextual representation of spatiotemporal interest points for action recognition. Vis. Comput. (2014)

  31. Wu, J., Hu, D., Chen, F.: Action recognition by hidden temporal models. Vis. Comput. 30, 1395–1404 (2013)

    Article  Google Scholar 

  32. Alon, J., Athitsos, V., Yuan, Q., Sclaroff, S.: A unified framework for gesture recognition and spatiotemporal gesture segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 31(9), 1685–1699 (2009)

    Article  Google Scholar 

  33. Pedersoli, F., Benini, S., Adami, N., Leonardi, R.: XKin: an open source framework for hand pose and gesture recognition using kinect. Vis. Comput. 30, 1107–1122 (2014)

    Article  Google Scholar 

  34. Bouchard, D., Badler, N.: Semantic segmentation of motion capture using laban movement analysis. In: Intelligent virtual agents. Springer, Berlin Heidelberg, pp. 37–44 (2007)

  35. Camurri, A., Lagerlöf, I., Volpe, G.: Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques. Int. J. Hum. Comput. Stud. 59(1), 213–225 (2003)

    Article  Google Scholar 

  36. Gunes, H., Schuller, B.: Categorical and dimensional affect analysis in continuous input: current trends and future directions. Image Vision Comput. (2012)

  37. Bernhardt, D., Robinson, P.: Detecting affect from non-stylised body motions. In: Affective Computing and Intelligent Interaction. Springer, Berlin Heidelberg, pp. 59–70 (2007)

  38. Nakata, T., Mori, T., Sato, T.: Analysis of impression of robot bodily expression. J. Robot. Mechatron. 14(1), 27–36 (2002)

    Google Scholar 

  39. Hachimura, K., Takashina, K., Yoshimura, M.: Analysis and evaluation of dancing movement based on LMA. In: IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2005, IEEE, pp. 294–299 (2005)

  40. Kapadia, M., Chiang, I.K., Thomas, T., Badler, NI., Kider J.T. Jr: Efficient motion retrieval in large motion databases. In: Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, pp. 19–28 (2013)

  41. Laban, R.: Espace Dynamique. Contredanse (2003)

  42. Clauser, C.E., McConville, J.T., Young J.W.: Weight, volume and center of mass of segments of the human body. Wright-Patterson Air Force Base, Ohio (AMRL-TR-69-70): ANTIOCH COLL YELLOW SPRINGS OH (1969)

  43. Fothergill, S., Mentis, H., Kohli, P., Nowozin, S.: Instructing people for training gestural interactive systems. In: Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems. ACM, pp. 1737–1746 (2012)

  44. Song, Y., Morency, L.P., Davis, R.: Distribution-Sensitive Learning for Imbalanced Datasets. In: 2013 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2013). IEEE (2013)

  45. Suykens, J.A.K., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)

    Article  MathSciNet  Google Scholar 

  46. Milgram, J., Cheriet, M., Sabourin, R.: “One Against One” or “One Against All”: Which One is Better for Handwriting Recognition with SVMs? In: Tenth International Workshop on Frontiers in Handwriting Recognition (2006)

  47. Breiman, L.: Random forests. 45(1), 5–32 (2001)

  48. Geurts, Pierre, Ernst, Damien, Wehenkel, Louis: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)

    Article  MATH  Google Scholar 

  49. Scikit-learn. (bibOnline). http://scikit-learn.org/stable/

  50. Hripcsak, G., Rothschild, A.S.: Agreement, the f-measure, and reliability in information retrieval. J. Am. Med. Inf. Assoc. 12(3), 296–298 (2005)

    Article  Google Scholar 

  51. Adrien, J.M: Une approche polyvalente: Direction Musicale Dansée, Captation Gestuelle Causale. In: Actes des Journées d’Informatique Musicale. Rennes (2010)

  52. Jorland, G., Thirioux, B.: Note sur l’origine de l’empathie. Revue de métaphysique et de morale. février, no. 58, pp. 269–280 (2008)

  53. Hautbois, X.: Les Unités Sémiotiques Temporelles : de la sémiotique musicale vers une sémiotique générale du temps dans les arts. In: ICMS 8. Gestes, formes et processus signifiants en musique et sémiotique interarts, Paris, Huitième Congrès International sur la Signification Musicale (2004)

  54. Shove, P., Repp, BH.: Musical motion and performance: theoretical and empirical perspectives. In: Rink, J. (ed.) The practice of performance. Cambridge University Press, pp. 55–83 (1995)

  55. Grewe, O., Kopiez, R., Altenmüller, E.: L’évaluation des sentiments musicaux: une comparaison entre le modèle circomplexe et les inventaires d’émotions à choix forcé”, Musique, langage, émotion. Approche neuro-cognitive, pp. 49–73 (2010)

  56. Chen, Y., Zhou, X.S., Huang, T.S.: One-class SVM for learning in image retrieval. In: 2001 International Conference on Image Processing Proceedings IEEE, vol. 1, pp. 34–37 (2001)

  57. Kanluan, I., Grimm, M., Kroschel, K.: Audio-visual emotion recognition using an emotion space concept. In: 16th European Signal Processing Conference. Lausanne (2008)

Download references

Author information

Authors and Affiliations

  1. ARTEMIS Department, Institut Mines-Telecom, Telecom SudParis, CNRS UMR 8145-MAP5, 9 rue Charles Fourier, 91 011, Évry Cedex, France

    Arthur Truong, Hugo Boujut & Titus Zaharia

Authors
  1. Arthur Truong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hugo Boujut
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Titus Zaharia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arthur Truong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Truong, A., Boujut, H. & Zaharia, T. Laban descriptors for gesture recognition and emotional analysis. Vis Comput 32, 83–98 (2016). https://doi.org/10.1007/s00371-014-1057-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-014-1057-8

Keywords

Search

Navigation