Abstract
Cognitive neuroscience research has revealed that electroencephalography (EEG) has a strong correlation with human emotions. However, due to the individual differences in EEG signals, the traditional models have the shortcoming of poor generalization ability. Based on the discovery that responses to emotional stimuli in the cerebral hemispheres is asymmetric, in this paper, we propose a Bi-hemispheric Capsule Network (Bi-CapsNet) Model for cross-subject EEG emotion recognition. Specifically, we firstly use a long short term memory (LSTM) layer to learn the asymmetry of emotion expression between the left and right hemispheres of the human brain and the deep representations of all the EEG electrodes’signals in different frequency bands. In order to capture the relationship between EEG channels more detailedly, a special mechanism called routing-by-agreement mechanism has been implemented between LSTM and EmotionCaps. We also use a domain discriminator working corporately with the EmotionCaps to reduce the domain shift between the source domain and the target domain. In addition, we propose a method to reduce the uncertainty of predictions on the target domain data by minimizing the entropy of the prediction posterior. Finally, the cross-subject EEG emotion recognition experiments conducted on two public datasets, SEED and SEED-IV, were that when the length of the EEG data samples is 1 s, the proposed model can obtain better results than most methods on the SEED-IV dataset and also achieves state-of-the-art performance on the SEED dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Quan, X., Zeng, Z., Jiang, J., Zhang, Y., Lu, B., Wu, D.: Physiological signals based affective computing: a systematic review. IEEE/CAA J. Autom. Sinica 8 (2021)
Recio, G., Schacht, A., Sommer, W.: Recognizing dynamic facial expressions of emotion: specificity and intensity effects in event-related brain potentials. Biol. Psychol. 96, 111–125 (2014)
Gunes, H., Piccardi, M.: Bi-modal emotion recognition from expressive face and body gestures. J. Netw. Comput. Appl. 30(4), 1334–1345 (2007)
Han, K., Yu, D., Tashev, I.: Speech emotion recognition using deep neural network and extreme learning machine. In: Interspeech 2014 (2014)
Wang, X.W., Nie, D., Lu, B.L.: Emotional state classification from EEG data using machine learning approach. Neurocomputing 129, 94–106 (2014)
Alarcao, S.M., Fonseca, M.J.: Emotions recognition using EEG signals: a survey. IEEE Trans. Affect. Comput. 10(3), 374–393 (2017)
Li, H., Jin, Y.-M., Zheng, W.-L., Lu, B.-L.: Cross-subject emotion recognition using deep adaptation networks. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11305, pp. 403–413. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04221-9_36
Zheng, W.L., Lu, B.L.: Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans. Auton. Ment. Dev. 7(3), 162–175 (2015)
Zheng, W.L., Liu, W., Lu, Y., Lu, B.L., Cichocki, A.: Emotionmeter: a multimodal framework for recognizing human emotions. IEEE Trans. Cybern. 49(3), 1110–1122 (2018)
Li, Y., Zheng, W., Zong, Y., Cui, Z., Zhang, T., Zhou, X.: A bi-hemisphere domain adversarial neural network model for EEG emotion recognition. IEEE Trans. Affect. Comput. 12(2), 494–504 (2018)
Li, Y., et al.: A novel bi-hemispheric discrepancy model for EEG emotion recognition. IEEE Trans. Cogn. Develop. Syst. 13(2), 354–367 (2020)
Zhong, P., Wang, D., Miao, C.: EEG-based emotion recognition using regularized graph neural networks. IEEE Trans. Affect. Comput. 14, 1290– 1301 (2020)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Jain, A., Singh, A., Koppula, H.S., Soh, S., Saxena, A.: Recurrent neural networks for driver activity anticipation via sensory-fusion architecture. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3118–3125. IEEE (2016)
Liu, Y., et al.: Multi-channel EEG-based emotion recognition via a multi-level features guided capsule network. Comput. Biol. Med. 123, 103927 (2020)
Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Advances in Neural Information Processing Systems 30 (2017)
Fleuret, F., et al.: Uncertainty reduction for model adaptation in semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9613–9623 (2021)
Zheng, W.L., Lu, B.L.: Personalizing EEG-based affective models with transfer learning. In: Proceedings of the Twenty-fifth International Joint Conference on Artificial Intelligence, pp. 2732–2738 (2016)
Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)
Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Networks 22(2), 199–210 (2010)
Song, T., Zheng, W., Lu, C., Zong, Y., Zhang, X., Cui, Z.: MPED: a multi-modal physiological emotion database for discrete emotion recognition. IEEE Access 7, 12177–12191 (2019)
Song, T., Zheng, W., Song, P., Cui, Z.: EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans. Affect. Comput. 11(3), 532–541 (2018)
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No.61876126 and 61503278).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Luan, X., Zhang, G., Yang, K. (2023). A Bi-hemisphere Capsule Network Model for Cross-Subject EEG Emotion Recognition. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Communications in Computer and Information Science, vol 1793. Springer, Singapore. https://doi.org/10.1007/978-981-99-1645-0_27
Download citation
DOI: https://doi.org/10.1007/978-981-99-1645-0_27
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1644-3
Online ISBN: 978-981-99-1645-0
eBook Packages: Computer ScienceComputer Science (R0)