Abstract
Recently, emotion recognition through gait, which is more difficult to imitate than other biological characteristics, has aroused extensive attention. Although some deep-learning studies have been conducted in this field, there are still two challenges. First, it is hard to extract the representational features of the gait from video effectively. Second, the input of body joints sequences has noise introduced during dataset collection and feature production. In this work, we propose a global link, which extends the existing skeleton graph (the natural link) to capture the overall state of gait based on spatial-temporal convolution. In addition, we use soft thresholding to reduce noise. The thresholds are learned automatically by a block called shrinkage block. Combined with the global link and shrinkage block, we further propose the global graph convolution shrinkage network (G-GCSN) to capture the emotion-related features. We validate the effectiveness of the proposed method on a public dataset (i.e., Emotion-Gait dataset). The proposed G-GCSN achieves improvements compared with state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhang, Y.D., et al.: Facial emotion recognition based on biorthogonal wavelet entropy, fuzzy support vector machine, and stratified cross validation. IEEE Access 4, 8375–8385 (2016)
Mohammadpour, M., Khaliliardali, H., Hashemi, S.M.R., AlyanNezhadi, M.M.: Facial emotion recognition using deep convolutional networks. In: IEEE 4th International Conference on Knowledge-Based Engineering and Innovation (KBEI), pp. 0017–0021. IEEE (2017)
Zhao, J., Mao, X., Chen, L.: Speech emotion recognition using deep 1D & 2D CNN LSTM networks. Biomed. Sig. Process. Control 47, 312–323 (2019)
Nicholson, J., Takahashi, K., Nakatsu, R.: Emotion recognition in speech using neural networks. Neural Comput. Appl. 9, 290–296 (2000)
Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2067–2083 (2008)
Kim, K.H., Bang, S.W., Kim, S.R.: Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput. 42, 419–427 (2004)
Lin, Y.P., et al.: EEG-based emotion recognition in music listening. IEEE Trans. Biomed. Eng. 57, 1798–1806 (2010)
Montepare, J.M., Goldstein, S.B., Clausen, A.: The identification of emotions from gait information. J. Nonverbal Behav. 11, 33–42 (1987)
Halovic, S., Kroos, C.: Not all is noticed: kinematic cues of emotion-specific gait. Hum. Mov. Sci. 57, 478–488 (2018)
Roether, C.L., Omlor, L., Christensen, A., Giese, M.A.: Critical features for the perception of emotion from gait. J. Vis. 9, 15–15 (2009)
Bhattacharya, U., Mittal, T., Chandra, R., Randhavane, T., Bera, A., Manocha, D.: Step: spatial temporal graph convolutional networks for emotion perception from gaits. AAA I, 1342–1350 (2020)
Chao, H., He, Y., Zhang, J., Feng, J.: Gaitset: regarding gait as a set for cross-view gait recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, 33, pp. 8126–8133 (2019)
Kale, A., et al.: Identification of humans using gait. IEEE Trans. Image Process. 13, 1163–1173 (2004)
Gage, J.R.: Gait analysis. an essential tool in the treatment of cerebral palsy. Clin. Orthopaedics Related Res. 126–134 (1993)
Jellinger, K., Armstrong, D., Zoghbi, H., Percy, A.: Neuropathology of rett syndrome. Acta Neuropathol. 76, 142–158 (1988)
Jankovic, J.: Parkinson’s disease: clinical features and diagnosis. J. Neurol. Neurosurg. Psychiatry 79, 368–376 (2008)
Li, M., Chen, S., Chen, X., Zhang, Y., Wang, Y., Tian, Q.: Actional-structural graph convolutional networks for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3595–3603 (2019)
Shi, L., Zhang, Y., Cheng, J., Lu, H.: Skeleton-based action recognition with directed graph neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7912–7921 (2019)
Li, B., Zhu, C., Li, S., Zhu, T.: Identifying emotions from non-contact gaits information based on microsoft kinects. IEEE Trans. Affect. Comput. 9, 585–591 (2016)
Quiroz, J.C., Geangu, E., Yong, M.H.: Emotion recognition using smart watch sensor data: mixed-design study. JMIR Mental Health 5, e10153 (2018)
Ahmed, F., Sieu, B., Gavrilova, M.L.: Score and rank-level fusion for emotion recognition using genetic algorithm. In: 2018 IEEE 17th International Conference on Cognitive Informatics & Cognitive Computing (ICCI* CC), pp. 46–53. IEEE (2018)
Zhang, Z., Song, Y., Cui, L., Liu, X., Zhu, T.: Emotion recognition based on customized smart bracelet with built-in accelerometer. PeerJ 4, e2258 (2016)
Venture, G., Kadone, H., Zhang, T., Grèzes, J., Berthoz, A., Hicheur, H.: Recognizing emotions conveyed by human gait. Int. J. Social Robot. 6, 621–632 (2014)
Karg, M., Kühnlenz, K., Buss, M.: Recognition of affect based on gait patterns. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 40, 1050–1061 (2010)
Daoudi, M., Berretti, S., Pala, P., Delevoye, Y., Del Bimbo, A.: Emotion recognition by body movement representation on the manifold of symmetric positive definite matrices. In: Battiato, S., Gallo, G., Schettini, R., Stanco, F. (eds.) ICIAP 2017. LNCS, vol. 10484, pp. 550–560. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68560-1_49
Wang, W., Enescu, V., Sahli, H.: Adaptive real-time emotion recognition from body movements. ACM Trans. Interactive Intell. Syst. (TiiS) 5, 1–21 (2015)
Crenn, A., Khan, R.A., Meyer, A., Bouakaz, S.: Body expression recognition from animated 3D skeleton. In: 2016 International Conference on 3D Imaging (IC3D), pp. 1–7. IEEE (2016)
Randhavane, T., Bhattacharya, U., Kapsaskis, K., Gray, K., Bera, A., Manocha, D.: Identifying emotions from walking using affective and deep features. arXiv preprint arXiv:1906.11884 (2019)
Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. arXiv preprint arXiv:1801.07455 (2018)
Shi, L., Zhang, Y., Cheng, J., Lu, H.: Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 12026–12035 (2019)
Liu, Z., Zhang, H., Chen, Z., Wang, Z., Ouyang, W.: Disentangling and unifying graph convolutions for skeleton-based action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 143–152 (2020)
Yu, B., Yin, H., Zhu, Z.: Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875 (2017)
Diao, Z., Wang, X., Zhang, D., Liu, Y., Xie, K., He, S.: Dynamic spatial-temporal graph convolutional neural networks for traffic forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 890–897 (2019)
Yang, J., Zheng, W.S., Yang, Q., Chen, Y.C., Tian, Q.: Spatial-temporal graph convolutional network for video-based person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3289–3299 (2020)
Zhao, M., Zhong, S., Fu, X., Tang, B., Pecht, M.: Deep residual shrinkage networks for fault diagnosis. IEEE Trans. Industr. Inf. 16, 4681–4690 (2019)
Habibie, I., Holden, D., Schwarz, J., Yearsley, J., Komura, T.: A recurrent variational autoencoder for human motion synthesis. In: 28th British Machine Vision Conference (2017)
Acknowledgements
This work was supported in part by Major Scientific Research Project of Zhejiang Lab under the Grant No. 2018DG0ZX01, and in part by the Grant in Aid for Scientific Research from the Japanese Ministry for Education, Science, Culture and Sports (MEXT) under the Grant No. 20K21821.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhuang, Y., Lin, L., Tong, R., Liu, J., Iwamoto, Y., Chen, YW. (2021). G-GCSN: Global Graph Convolution Shrinkage Network for Emotion Perception from Gait. In: Sato, I., Han, B. (eds) Computer Vision – ACCV 2020 Workshops. ACCV 2020. Lecture Notes in Computer Science(), vol 12628. Springer, Cham. https://doi.org/10.1007/978-3-030-69756-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-69756-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-69755-6
Online ISBN: 978-3-030-69756-3
eBook Packages: Computer ScienceComputer Science (R0)