Abstract
EEG signals for real-time emotion identification are crucial for affective computing and human-computer interaction. The current emotion recognition models, which rely on a small number of emotion classes and stimuli like music and images in controlled lab conditions, have poor ecological validity. Furthermore, identifying relevant EEG signal features is crucial for efficient emotion identification. According to the complexity, non-stationarity, and variation nature of EEG signals, which make it challenging to identify relevant features to categorize and identify emotions, a novel approach for feature extraction and classification concerning EEG signals is suggested based on invariant wavelet scattering transform (WST) and support vector machine algorithm (SVM). The WST is a new time-frequency domain equivalent to a deep convolutional network. It produces scattering feature matrix representations that are stable against time-warping deformations, noise-resistant, and time-shift invariant existing in EEG signals. So, small, difficult-to-measure variations in the amplitude and duration of EEG signals can be captured. As a result, it addresses the limitations of the previous feature extraction approaches, which are unstable and sensitive to time-shift variations. In this paper, the zero, first, and second order features from DEAP datasets are obtained by performing the WST with two deep layers. Then, the PCA method is used for dimensionality reduction. Finally, the extracted features are fed as inputs for different classifiers. In the classification step, the SVM classifier is utilized with different classification algorithms such as k-nearest neighbours (KNN), random forest (RF), and AdaBoost classifier. This research employs a principal component analysis (PCA) approach to reduce the high dimensionality of scattering characteristics and increase the computational efficiency of our classifiers. The proposed method is performed across four different emotional classification models based on valence, arousal, dominance, and liking dimensions on the DEAP dataset. It achieves over 98% for two emotional classes and over 97% for three, four, and eight emotional classes. The results unequivocally demonstrate the efficacy of the proposed WST, PCA, and SVM-based emotion recognition approach for EEG signal emotion recognition.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The data utilized in this study are derived from the DEAP dataset, a publicly available collection of EEG recordings and associated response metrics for emotion analysis. Researchers can request and access the data through the website (https://www.eecs.qmul.ac.uk/mmv/datasets/deap/index.html). Authors had no special access privileges to this data.
References
Ackermann P, Kohlschein C, Bitsch JA et al (2016) EEG-based automatic emotion recognition: Feature extraction, selection and classification methods. In: 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom). IEEE
Almanza-Conejo O, Almanza-Ojeda DL, Contreras-Hernandez JL, Ibarra-Manzano MA (2022) Emotion recognition in EEG signals using the continuous wavelet transform and CNNs. Neural Comput Appl 35:1409–1422. https://doi.org/10.1007/s00521-022-07843-9
Andén J, Mallat S (2011) Multiscale Scattering for Audio Classification. In: ISMIR. pp 657–662
Anden J, Mallat S (2014) Deep scattering spectrum. IEEE Trans Signal Process 62:4114–4128. https://doi.org/10.1109/tsp.2014.2326991
Bhosale S, Chakraborty R, Kopparapu SK (2022) Calibration free meta learning based approach for subject Independent EEG emotion recognition. Biomed Signal Process Control 72:103289. https://doi.org/10.1016/j.bspc.2021.103289
Breiman L (2001) Random forests. Mach Learn 45:5–32
Bruna J, Mallat S (2011) Classification with invariant scattering representations. 2011 IEEE 10th IVMSP workshop. Perception and Visual Signal Analysis. IEEE
Bruna J, Mallat S (2013) Invariant scattering Convolution Networks. IEEE Trans Pattern Anal Mach Intell 35:1872–1886. https://doi.org/10.1109/tpami.2012.230
Chao H, Liu Y (2020) Emotion Recognition from Multi-channel EEG signals by exploiting the deep belief-conditional Random Field Framework. IEEE Access 8:33002–33012. https://doi.org/10.1109/access.2020.2974009
Cui H, Liu A, Zhang X et al (2020) EEG-based emotion recognition using an end-to-end regional-asymmetric convolutional neural network. Knowl Based Syst 205:106243. https://doi.org/10.1016/j.knosys.2020.106243
Dogan A, Akay M, Barua PD et al (2021) PrimePatNet87: Prime pattern and tunable q-factor wavelet transform techniques for automated accurate EEG emotion recognition. Comput Biol Med 138:104867. https://doi.org/10.1016/j.compbiomed.2021.104867
Islam MR, Islam MM, Rahman MM et al (2021) EEG Channel correlation based model for emotion recognition. Comput Biol Med 136:104757. https://doi.org/10.1016/j.compbiomed.2021.104757
Islam MR, Moni MA, Islam MM et al (2021) Emotion Recognition from EEG Signal Focusing on deep learning and shallow learning techniques. IEEE Access 9:94601–94624. https://doi.org/10.1109/access.2021.3091487
Koelstra S, Muhl C, Soleymani M et al (2012) DEAP: a database for emotion analysis;using physiological signals. IEEE Trans Affect Comput 3:18–31. https://doi.org/10.1109/t-affc.2011.15
Li J, Zhang Z, He H (2017) Hierarchical convolutional neural networks for EEG-Based emotion recognition. Cognit Comput 10:368–380. https://doi.org/10.1007/s12559-017-9533-x
Li H, Jin Y-M, Zheng W-L, Lu B-L (2018) Cross-subject emotion Recognition using Deep Adaptation Networks. Neural information Processing. Springer International Publishing, pp 403–413
Li D, Xie L, Chai B et al (2022) Spatial-frequency convolutional self-attention network for EEG emotion recognition. Appl Soft Comput 122:108740. https://doi.org/10.1016/j.asoc.2022.108740
Li X, Zhang Y, Tiwari P et al (2022) EEG based emotion recognition: a Tutorial and Review. ACM Comput Surv 55:1–57. https://doi.org/10.1145/3524499
Liu Y, Sourina O (2014a) EEG-based subject-dependent emotion recognition algorithm using fractal dimension. In: 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE
Liu Y, Sourina O (2014) Real-time subject-dependent EEG-Based emotion Recognition Algorithm. Transactions on Computational Science XXIII. Springer, Berlin, pp 199–223
Liu L, Wu J, Li D et al (2019) Fractional wavelet scattering network and applications. IEEE Trans Biomed Eng 66:553–563. https://doi.org/10.1109/tbme.2018.2850356
Liu J, Wu G, Luo Y et al (2020) EEG-Based emotion classification using a deep neural network and sparse autoencoder. Front Syst Neurosci 14:43. https://doi.org/10.3389/fnsys.2020.00043
Lostanlen V, Mallat S (2016) Wavelet scattering on the pitch spiral. arXiv preprint arXiv:160100287
Luo Y, Zhu L-Z, Wan Z-Y, Lu B-L (2020) Data augmentation for enhancing EEG-based emotion recognition with deep generative models. J Neural Eng 17:056021. https://doi.org/10.1088/1741-2552/abb580
Mallat S (2012) Group Invariant Scattering. Commun Pure Appl Math 65:1331–1398. https://doi.org/10.1002/cpa.21413
Martis RJ, Acharya UR, Mandana KM et al (2012) Application of principal component analysis to ECG signals for automated diagnosis of cardiac health. Expert Syst Appl 39:11792–11800. https://doi.org/10.1016/j.eswa.2012.04.072
McEvoy LK, Smith ME, Gevins A (2000) Test–retest reliability of cognitive EEG. Clin Neurophysiol 111:457–463. https://doi.org/10.1016/s1388-2457(99)00258-8
Mohammadi Z, Frounchi J, Amiri M (2016) Wavelet-based emotion recognition system using EEG signal. Neural Comput Appl 28:1985–1990. https://doi.org/10.1007/s00521-015-2149-8
Qing C, Qiao R, Xu X, Cheng Y (2019) Interpretable emotion Recognition using EEG signals. IEEE Access 7:94160–94170. https://doi.org/10.1109/access.2019.2928691
Raheel A, Anwar SM, Majid M (2018) Emotion recognition in response to traditional and tactile enhanced multimedia using electroencephalography. Multimed Tools Appl 78:13971–13985. https://doi.org/10.1007/s11042-018-6907-3
Siddharth JT-P, Sejnowski TJ (2022) Utilizing deep learning towards multi-modal Bio-sensing and Vision-Based Affective Computing. IEEE Trans Affect Comput 13:96–107. https://doi.org/10.1109/taffc.2019.2916015
Sifre L, Mallat S (2013) Rotation, scaling and deformation invariant scattering for texture discrimination. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition. IEEE
van den Broek EL (2011) Ubiquitous emotion-aware computing. Pers Ubiquitous Comput 17:53–67. https://doi.org/10.1007/s00779-011-0479-9
Yao Q, Gu H, Wang S, Li X (2022) A feature-fused convolutional neural network for emotion recognition from multichannel EEG signals. IEEE Sens J 22:11954–11964. https://doi.org/10.1109/jsen.2022.3172133
Yin Y, Zheng X, Hu B et al (2021) EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM. Appl Soft Comput 100:106954. https://doi.org/10.1016/j.asoc.2020.106954
Zheng W-L, Zhu J-Y, Lu B-L (2019) Identifying stable patterns over time for emotion recognition from EEG. IEEE Trans Affect Comput 10:417–429. https://doi.org/10.1109/taffc.2017.2712143
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Elrefaiy, A., Tawfik, N., Zayed, N. et al. EEG emotion recognition framework based on invariant wavelet scattering convolution network. J Ambient Intell Human Comput 15, 2181–2199 (2024). https://doi.org/10.1007/s12652-023-04746-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-023-04746-y