Authors:
Pedro V. V. Paiva
1
;
2
;
Josué Ramos
3
;
Marina Gavrilova
2
and
Marco A. G. Carvalho
1
Affiliations:
1
School of Technology, University of Campinas, Limeira, Brazil
;
2
Departement of Computer Science, University of Calgary, Calgary, Canada
;
3
Cyber-Physical Systems Division, Renato Archer IT Center, Campinas, Brazil
Keyword(s):
Body Emotion Recognition, Affective Computing, Video, Image Processing, Gait Analysis, Attention-Based Design.
Abstract:
Capturing humans’ emotional states from images in real-world scenarios is a key problem in affective computing, which has various real-life applications. Emotion recognition methods can enhance video games to increase engagement, help students to keep motivated during e-learning sections, or make interaction more natural in social robotics. Body movements, a crucial component of non-verbal communication, remain less explored in the domain of emotion recognition, while face expression-based methods are widely investigated. Transformer networks have been successfully applied across several domains, bringing significant breakthroughs. Transformers’ self-attention mechanism captures relationships between different features across different spatial locations, allowing contextual information extraction. In this work, we introduce Emotion Transformer, a self-attention architecture leveraging spatial configurations of body joints for Body Emotion Recognition. Our approach is based on the vis
ual transformer linear projection function, allowing the conversion of 2D joint coordinates to a regular matrix representation. The matrix projection then feeds a regular transformer multi-head attention architecture. The developed method allows a more robust correlation between joint movements with time to recognize emotions using contextual information learning. We present an evaluation benchmark for acted emotional sequences extracted from movie scenes using the BoLD dataset. The proposed methodology outperforms several state-of-the-art architectures, proving the effectiveness of the method.
(More)