article

Continuous body emotion recognition system during theater performances

Authors:

Andreas Aristidou,

Nadia Magnenat-ThalmannAuthors Info & Claims

Computer Animation and Virtual Worlds, Volume 27, Issue 3-4

Pages 311 - 320

https://doi.org/10.1002/cav.1714

Published: 01 May 2016 Publication History

Abstract

Understanding emotional human behavior in its multimodal and continuous aspect is necessary for studying human machine interaction and creating constituent social agents. As a first step, we propose a system for continuous emotional behavior recognition expressed by people during communication based on their gesture and their whole body dynamical motion. The features used to classify the motion are inspired by the Laban Movement Analysis entities and are mapped onto the well-known Russell Circumplex Model . We choose a specific case study that corresponds to an ideal case of multimodal behavior that emphasizes the body motion expression: theater performance. Using a trained neural network and annotated data, our system is able to describe the motion behavior as trajectories on the Russell Circumplex Model diagram during theater performances over time. This work contributes to the understanding of human behavior and expression and is a first step through a complete continuous emotion recognition system whose next step will be adding facial expressions. Copyright © 2016 John Wiley & Sons, Ltd.

References

[1]

Ekman P, Friesen WV. Facial Action Coding System. Consulting Psychologists Press, Stanford University: Palo Alto, 1977.

[2]

EMOVU. Available from: "www.emovu.com" {Accessed on February 2016}.

[3]

NVISO. Available from: "www.nviso.ch" {Accessed on February 2016}.

[4]

Russell JA. A circumplex model of affect. Journal of Personality and Social Psychology. 1980; Volume 39: pp.1161-1178.

[5]

Mehrabian A. Pleasure-arousal. Dominance: a general framework for describing and measuring individual differences in temperament. Current Psychology. 1996; Volume 14 Issue 4: pp.261-292.

[6]

Ortony A, Clore GL, Collins A. The Cognitive Structure of Emotions. Cambridge University Press: Cambridge, New York, 1994. Autre tirage: 1999.

[7]

Russell JA. Core affect and the psychological construction of emotion. Psychological Review. 2003; Volume 110 Issue 1: pp.145-72.

[8]

Stanislavsky K. Building a Character, <bookSeriesTitle>Theatre Arts Books</bookSeriesTitle>. Theatre Arts Books, 1948.

[9]

Stanislavski K. An Actor's Work: A Student's Diary. Taylor & Francis, 1938.

[10]

Kipp M, Martin J-c. Gesture and emotion: can basic gestural form features discriminate emotions? In Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interactions: De Rode Hoed Amsterdam, Netherlands, 2009.

[11]

<familyNamePrefix>von</familyNamePrefix>Laban R, Ullmann L. The Mastery of Movement. Northcote House, 1988.

[12]

Bartenieff I, Lewis D. Body Movement: Coping with the Environment. Gordon and Breach Science Publishers, 1980.

[13]

Chi D, Costa M, Zhao L, Badler N. The EMOTE model for effort and shape. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques-SIGGRAPH '00: Ernest N. Morial Convention Center New Orleans. Louisiana USA, 2000; pp.173-182.

Digital Library

[14]

Zhao L, Badler NI. Acquiring and validating motion qualities from live limb gestures. Graphical Models. 2005; Volume 67 Issue 1: pp.1-16.

Digital Library

[15]

Torresani L, Hackney P, Bregler C. Learning motion style synthesis from perceptual observations. Neural Information Processing Systems. 2006; Volume 19: pp.1393-1400.

Digital Library

[16]

Luo P, Neff M. In Proceedings of Motion in Games MIG'12: Rennes, France, 2012; pp.254-265.

[17]

Wakayama Y, Okajima S, Takano S, Okada Y. In Knowledge-Based and Intelligent Information and Engineering Systems: 14th International Conference, KES 2010, Cardiff, UK, 8-10 September 2010, Proceedings, Part IV, Setchi R, Jordanov I, Howlett RJ, Jain LC ¿eds. Springer Berlin Heidelberg: Berlin, Heidelberg, 2010; pp.251-260.

Digital Library

[18]

Santos L, Dias J. In Emerging Trends in Technological Innovation: First IFIP WG 5.5/SOCOLNET Doctoral Conference on Computing, Electrical and Industrial Systems, doCEIS 2010, Costa de Caparica, Portugal, 22-24 February 2010. Proceedings, Camarinha-Matos LM, Pereira P, Ribeiro L ¿eds. Springer Berlin Heidelberg: Berlin, Heidelberg, 2010; pp.187-194.

[19]

Nakata T, Mori T, Sato T. Quantitative analysis of impression of robot bodily expression based on Laban Movement Theory. Journal of the Robotics Society of Japan. 2001; Volume 19 Issue 2: pp.252-259.

[20]

Masuda M, Kato S, Itoh H. Emotion detection from body motion of human. In The 12th International Conference Principles of Practice in Multi-Agent Systems: Nagoya, Japan, 2009; pp.322-334.

Digital Library

[21]

Masuda M, Kato S. Motion rendering system for emotion expression of human form robots based on Laban Movement Analysis. In Proceedings-IEEE International Workshop on Robot and Human Interactive Communication: Congress Center Il Principino Viareggio, Italy, 2010; pp.324-329.

[22]

Fagerberg P, Stå¿hl A, Höök K. Designing gestures for affective input: an analysis of shape, effort and valence. In MUM 2003: Proceedings of the 2nd International Conference on Mobile and Ubiquitous Multimedia, 10-12 December, 2003, Norrköping, Sweden. ACM: Norrköping, Sweden, 2003; pp.57-65.

[23]

Hartmann B, Mancini M, Pelachaud C. 2006. Implementing expressive gesture synthesis for embodied conversational agents. In Lecture Notes in Computer Science including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, vol.¿3881 LNAI Springer; pp.188-199.

Digital Library

[24]

Truong A, Boujut H, Zaharia T. Laban descriptors for gesture recognition and emotional analysis. The Visual Computer. 2016; Volume 32 Issue 1: pp.83-98.

Digital Library

[25]

Zacharatos H, Gatzoulis C, Chrysanthou Y, Aristidou A. Emotion recognition for exergames using Laban Movement Analysis. In Proceedings of the Motion on Games, <bookSeriesTitle>MIG '13</bookSeriesTitle>: New York, NY, USA, 2013; pp.39-44.ACM.

Digital Library

[26]

Morita J, Nagai Y, Moritsu T. Relations between body motion and emotion: analysis based on Laban Movement Analysis. In The Annual Meeting of the Cognitive Science Society: Berlin, Germany, 2013; pp.1026-1031.

[27]

Shiratori T, Nakazawa A, Ikeuchi K. Dancing-to-music character animation. Computer Graphics Forum. 2006; Volume 25 Issue 3: pp.449-458.

[28]

Fourati N, Pelachaud C. Relevant body cues for the classification of emotional body expression in daily actions. In Proceedings of the Sixth International Conference on Affective Computing and Intelligent Interaction ACII2015: Xi'an, China, 2015; pp.267-273.

Digital Library

[29]

Kim WH, Park JW, Lee WH, Chung MJ, Lee HS. LMA based emotional motion representation using RGB-D camera. In HRI '13: Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction: Tokyo, Japan, 2013; pp.163-164.

Digital Library

[30]

Aristidou A, Chrysanthou Y. Motion indexing of different emotional states using LMA components. In SIGGRAPH Asia 2013 Technical Briefs on-SA '13: Hong Kong, 2013; pp.1-4.

Digital Library

[31]

Valstar M, Schuller B, Smith K, et al. AVEC 2013: the continuous audio/visual emotion and depression recognition challenge. In Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge AVEC '13: Barcelona, Catalunya, Spain, 2013; pp.3-10.

Digital Library

[32]

Aristidou A, Charalambous P, Chrysanthou Y. Emotion analysis and classification: understanding the performers' emotions using the LMA entities. Computer Graphics Forum. 2015; Volume 34 Issue 6: pp.262-276.

Digital Library

[33]

Levine DS. Neural network modeling of emotion. Physics of Life Reviews. 2007; Volume 4 Issue 1: pp.37-63.

[34]

Lotfi E, Akbarzadeh-T MR. Practical emotional neural networks. Neural Networks. 2014; Volume 59: pp.61-72.

Digital Library

[35]

Thenius R, Zahadat P, Schmickl T. EMANN-a model of emotions in an artificial neural network. Advances in Artificial Life, ECAL 2013. 2013; Volume 12 Issue SEPTEMBER: pp.830-837.

[36]

Bouchard D. Segmenting motion capture data using a qualitative analysis. In MIG '15: Proceedings of the 8th ACM SIGGRAPH Conference on Motion in Games: Paris, France, 2015; pp.23-30.

Digital Library

[37]

Kitsikidis A, Dimitropoulos K, Douka S, Grammalidis N. Dance analysis using multiple Kinect sensors. In VISAPP2014: Lisbon, Portugal, 2014; pp.789-795.

Cited By

Hakim AMarsland SGuesgen H(2022)Computational representation and analysis of emotion dynamicsMultimedia Tools and Applications10.1007/s11042-022-12490-281:15(21111-21133)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s11042-022-12490-2
Novais RCardoso PRodrigues J(2022)Facial Emotions Classification Supported in an Ensemble StrategyUniversal Access in Human-Computer Interaction. Novel Design Approaches and Technologies10.1007/978-3-031-05028-2_32(477-488)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-05028-2_32
Zhou Y(2021)Role of Human Body Posture Recognition Method Based on Wireless Network Kinect in Line Dance Aerobics and Gymnastics TrainingWireless Communications & Mobile Computing10.1155/2021/92088912021Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1155/2021/9208891
Show More Cited By

Recommendations

Human motions and emotions recognition inspired by LMA qualities
Abstract
The purpose of this paper is to describe human motions and emotions that appear on real video images with compact and informative representations. We aimed to recognize expressive motions and analyze the relationship between human body features ...
EmotiW 2016: video and group-level emotion recognition challenges
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

This paper discusses the baseline for the Emotion Recognition in the Wild (EmotiW) 2016 challenge. Continuing on the theme of automatic affect recognition `in the wild', the EmotiW challenge 2016 consists of two sub-challenges: an audio-video based ...
Emotion Recognition from Body Expressions with a Neural Network Architecture
HAI '17: Proceedings of the 5th International Conference on Human Agent Interaction

The recognition of emotions plays an important role in our daily life and is essential for social communication. Although multiple studies have shown that body expressions can strongly convey emotional states, emotion recognition from body motion ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computer Animation and Virtual Worlds

Computer Animation and Virtual Worlds Volume 27, Issue 3-4

May 2016

270 pages

ISSN:1546-4261

Issue’s Table of Contents

Publisher

John Wiley and Sons Ltd.

United Kingdom

Publication History

Published: 01 May 2016

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hakim AMarsland SGuesgen H(2022)Computational representation and analysis of emotion dynamicsMultimedia Tools and Applications10.1007/s11042-022-12490-281:15(21111-21133)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s11042-022-12490-2
Novais RCardoso PRodrigues J(2022)Facial Emotions Classification Supported in an Ensemble StrategyUniversal Access in Human-Computer Interaction. Novel Design Approaches and Technologies10.1007/978-3-031-05028-2_32(477-488)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-05028-2_32
Zhou Y(2021)Role of Human Body Posture Recognition Method Based on Wireless Network Kinect in Line Dance Aerobics and Gymnastics TrainingWireless Communications & Mobile Computing10.1155/2021/92088912021Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1155/2021/9208891
Noroozi FCorneanu CKamińska DSapiński TEscalera SAnbarjafari G(2021)Survey on Emotional Body Gesture RecognitionIEEE Transactions on Affective Computing10.1109/TAFFC.2018.287498612:2(505-523)Online publication date: 1-Apr-2021
https://dl.acm.org/doi/10.1109/TAFFC.2018.2874986
Stoeva DGelautz MBelpaeme TYoung JGunes HRiek L(2020)Body Language in Affective Human-Robot InteractionCompanion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3371382.3377432(606-608)Online publication date: 23-Mar-2020
https://dl.acm.org/doi/10.1145/3371382.3377432
Bakalos NRallis IDoulamis NDoulamis AVoulodimos AVescoukis V(2020)Motion Primitives Classification Using Deep Learning Models for Serious Game PlatformsIEEE Computer Graphics and Applications10.1109/MCG.2020.298503540:4(26-38)Online publication date: 1-Jul-2020
https://dl.acm.org/doi/10.1109/MCG.2020.2985035
Senecal SNijdam NAristidou AMagnenat-Thalmann N(2020)Salsa dance learning evaluation and motion analysis in gamified virtual reality environmentMultimedia Tools and Applications10.1007/s11042-020-09192-y79:33-34(24621-24643)Online publication date: 1-Sep-2020
https://dl.acm.org/doi/10.1007/s11042-020-09192-y
Aristidou AShamir AChrysanthou Y(2019)Digital Dance EthnographyJournal on Computing and Cultural Heritage 10.1145/334438312:4(1-27)Online publication date: 17-Nov-2019
https://dl.acm.org/doi/10.1145/3344383
Ajili IMallem MDidier J(2019)Human motions and emotions recognition inspired by LMA qualitiesThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-018-01619-w35:10(1411-1426)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1007/s00371-018-01619-w
Furuichi SAbe KNakamura S(2019)The Possibility of Personality Extraction Using Skeletal Information in Hip-Hop Dance by Human or MachineHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29390-1_28(511-519)Online publication date: 2-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-29390-1_28
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents