Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

EEG emotion recognition based on PLV-rich-club dynamic brain function network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

During emotional changes, the brain generates many highly connected and highly concentrated hub regions. Thus, barely studying the whole-brain network architecture while ignoring the connection information between small-scale structures, will not be able to accurately differentiate the information interaction patterns of the brain under different emotions. The connections between high-degree nodes in the brain are stronger than those between low-degree nodes. The rich-club structure composed by those high-degree nodes is a hot topic in the current research of emotion recognition. Therefore, this paper investigates the dynamics of the rich-club structure generated by high-degree nodes in different time periods during different emotional changes. Firstly, the dynamic PLV brain network is constructed using non-overlapping time windows. Afterwards, the rich-club coefficients in the brain network are calculated and the nodes with higher degrees are selected as rich-club nodes within the coefficient range. Furthermore, the ReliefF algorithm is used to filter the frequency domain features of rich-club nodes and PLV-rich-club graph-theoretical features to derive the most emotionally relevant features for emotion recognition. The experimental results show that the composition of the rich-club is roughly the same but there are subtle differences over time. The smaller the valence, the larger the K-degree range, in which the structure of the rich-club is more stable, and the information interaction is more complex. The larger the valence, in which the information contained by the rich-club structure is more modular, and its ability to transmit information is stronger. It is revealed that the key to distinguishing different emotions is the brain information represented by the connections between small-scale structures in complex brain networks. The LOSO validation method was used for verification on the DEAP and SEED datasets, and the features based on the rich-club structure achieved 86.11% and 87.92% accuracy in the valence dimension, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Alarcao SM, Fonseca MJ (2017) Emotions recognition using eeg signals: a survey. IEEE Trans Affect Comput 10(3):374–393

    Article  Google Scholar 

  2. Zheng W-L, Liu W, Lu Y, Lu B-L, Andrzej C (2018) Emotionmeter: a multimodal framework for recognizing human emotions. IEEE Trans Cybern 49(3):1110–1122

    Article  Google Scholar 

  3. Val-Calvo M, Benamara NK (2021) Real-time facial expression recognition using smoothed deep neural network ensemble. Integr Comput-Aided Eng 28:97–111

    Google Scholar 

  4. Lu BL, Wang XW, Nie D (2014) Emotional state classification from eeg data using machine learning approach. Neurocomputing 129:94–106

    Article  Google Scholar 

  5. Mühl C, Allison B, Nijholt A, Chanel G (2014) A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges. Brain-Comput Interfaces 1(2):66–84

    Article  Google Scholar 

  6. Ezzyat Y, Olson IR, Plotzker A (2007) The enigmatic temporal pole: a review of findings on social and emotional processing. Brain 130(7):1718–1731

    Article  Google Scholar 

  7. Putkinen V, Nazari-Farsani S, Seppälä K, Karjalainen T, Sun L, Karlsson HK, Hudson M, Heikkilä TT, Hirvonen J, Nummenmaa L (2021) Decoding music-evoked emotions in the auditory and motor cortex. Cereb Cortex 31(5):2549–2560

    Article  Google Scholar 

  8. Russell LL, Greaves CV, Convery RS, Nicholas J, Warren JD, Kaski D, Rohrer JD (2021) Novel instructionless eye tracking tasks identify emotion recognition deficits in frontotemporal dementia. Alzheimer’s Res Therapy 13(1):1–11

    Google Scholar 

  9. Gao Z, Dang W, Wang X, Hong X, Hou L, Ma K, Perc M (2021) Complex networks and deep learning for eeg signal analysis. Cogn Neurodyn 15(3):369–388

    Article  Google Scholar 

  10. Gu X, Cai W, Gao M, Jiang Y, Ning X, Qian P (2022) Multi-source domain transfer discriminative dictionary learning modeling for electroencephalogram-based emotion recognition. IEEE Trans Computat Social Syst

  11. Halim Z, Rahman AU (2022) Identifying dominant emotional state using handwriting and drawing samples by fusing features. Appl Intell:1–17

  12. Rehan M, Halim Z (2020) On identification of driving-induced stress using electroencephalogram signals: a framework based on wearable safety-critical scheme and machine learning. Inf Fusion 53:66–79

    Article  Google Scholar 

  13. Abbass HA, Wang M, Hu J (2020) Brainprint: Eeg biometric identification based on analyzing brain connectivity graphs. Pattern Recognit 105:107381

    Article  Google Scholar 

  14. Li C, Li P, Jiang L, Zhu X, Si Y, Zeng Y, Yao D, Xu P (2019) Emotion recognition with the feature extracted from brain networks. In: 2019 IEEE international conference on computational intelligence and virtual environments for measurement systems and applications (CIVEMSA). IEEE, pp 1–4

  15. Li Y, Zheng W, Wang L, Zong Y, Cui Z (2019) From regional to global brain: a novel hierarchical spatial-temporal neural network model for eeg emotion recognition. IEEE Trans Affect Comput

  16. Tao W, Li C, Song R, Cheng J, Liu Y, Wan F, Chen X (2020) Eeg-based emotion recognition via channel-wise attention and self attention. IEEE Trans Affect Comput

  17. Wu X, Zheng W-L, Li Z, Lu B-L (2022) Investigating eeg-based functional connectivity patterns for multimodal emotion recognition. J Neural Eng 19(1):016012

    Article  Google Scholar 

  18. Fries P (2005) A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cognit Sci 9(10):474–480

    Article  Google Scholar 

  19. Pan J, Yang F, Qiu L, Huang H (2022) Fusion of eeg-based activation, spatial, and connection patterns for fear emotion recognition. Computat Intell Neurosci, vol 2022

  20. McIntosh AR (2000) Towards a network theory of cognition. Neural Netw 13(8-9):861–870

    Article  Google Scholar 

  21. Jiao Z, Ma K, Wang H, Zou L, Zhang Y (2018) Research on node properties of resting-state brain functional networks by using node activity and alff. Multimed Tools Appl 77(17):22689–22704

    Article  Google Scholar 

  22. Gao Z, Li R, Ma C, Rui L, Sun X (2021) Core-brain-network-based multilayer convolutional neural network for emotion recognition. IEEE Trans Instrum Meas 70:1–9

    Google Scholar 

  23. Li L, Jie X, Cao R (2014) Emotion recognition based on the sample entropy of eeg[j]. Bio-medical materials and engineering. Bio-Med Materials Eng 24(1):1185–1192

    Article  Google Scholar 

  24. Sporns O, Van Den Heuvel MP (2013) Network hubs in the human brain. Trends Cognit Sci 17(12):683–696

    Article  Google Scholar 

  25. Collin G, Kahn RS, Reus MAD, Cahn W, Heuvel MPVD (2014) Impaired rich club connectivity in unaffected siblings of schizophrenia patients. Schizophrenia Bullet 40(2):438–448

    Article  Google Scholar 

  26. Daianu M, Jahanshad N, Nir TM, Jack CJR, Weiner MW, Bernstein MA, Thompson PM (2015) Alzheimer’s disease neuroimaging initiative rich club analysis in the alzheimer’s disease connectome reveals a relatively undisturbed structural core network. Human Brain Map 36(8):3087–3103

    Article  Google Scholar 

  27. Min BK, Kim DJ (2020) Rich-club in the brain’s macrostructure: insights from graph theoretical analysis. Computat Struct Biotechnol J 18:1761–1773

    Article  Google Scholar 

  28. Caetano TS, McAuley JJ, Da Fontoura Costa L (2007) Rich-club phenomenon across complex network hierarchies. Appl Phys Lett 91(8)

  29. Sporns O, Rubinov M (2010) Complex network measures of brain connectivity: uses and interpretations. Neuroimage 52(3):1059–1069

    Article  Google Scholar 

  30. Lou C, Cross AM, Peters L, Ansari D, Joanisse MF (2021) Rich-club structure contributes to individual variance of reading skills via feeder connections in children with reading disabilities. Develop Cognit Neurosci 49:100957

    Article  Google Scholar 

  31. Wang S, Yang M, Zhang Y, Li J, Zou L, Lu S, Liu B, Yang J, Zhang Y (2016) Detection of left-sided and right-sided hearing loss via fractional fourier transform. Entropy 18(5):194

    Article  Google Scholar 

  32. Piqueira JRC (2011) Network of phase-locking oscillators and a possible model for neural synchronization. Commun Nonlinear Sci Numer Simul 16(9):3844–3854

    Article  MathSciNet  MATH  Google Scholar 

  33. Kayhan E, Matthes D, Haresign IM, Bánki A, Michel C, Langeloh M, Wass S, Hoehl S (2022) Deep: a dual eeg pipeline for developmental hyperscanning studies. Develop Cognit Neurosci 54:101104

    Article  Google Scholar 

  34. Yildirim S, Dasdemir Y, Yildirim E (2017) Analysis of functional brain connections for positive–negative emotions using phase locking value. Cognit Neurodynamics 11(6):487–500

    Article  Google Scholar 

  35. Deng S, Yan J, Chen S (2019) A eeg-based emotion recognition model with rhythm and time characteristics. Brain Inf 6(1):1–8

    Google Scholar 

  36. Damaraju E, Tagliazucchi E, Laufs H, Calhoun VD (2020) Connectivity dynamics from wakefulness to sleep. Neuroimage 220:117047

    Article  Google Scholar 

  37. Zhou R, Wang Z (2020) Emotion-related rich-club organization in dynamic brain network. In: 2020 International conference on networking and network applications (naNA), vol 2020. IEEE, pp 298-303

  38. Koelstra S, Muhl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2011) Deap: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3(1):18–31

    Article  Google Scholar 

  39. Lu BL, Zheng WL (2015) Investigating critical frequency bands and channels for eeg-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7(3):162–175

    Article  Google Scholar 

  40. Lu BL, Li M (2009) Lu BL, Li M (2009) Emotion classification based on gamma-band eeg. In: 2009 Annual international conference of the IEEE engineering in medicine and biology society, vol 2009. IEEE, pp 1223-1226

  41. Wang Z-M, Zhang J-W, He Y, Zhang J (2022) Eeg emotion recognition using multichannel weighted multiscale permutation entropy. Appl Intell:1–13

  42. Zheng R, Wang Z, He Y, Zhang J (2022) Eeg-based brain functional connectivity representation using amplitude locking value for fatigue-driving recognition. Cogn Neurodyn 16(2):325–336

    Article  Google Scholar 

  43. Nasab SA, Panahi S, Ghassemi F, Jafari S, Rajagopal K, Ghosh D, Perc M (2022) Functional neuronal networks reveal emotional processing differences in children with adhd. Cogn Neurodyn 16 (1):91–100

    Article  Google Scholar 

  44. Gao X, Cao H, Ming D, Qi H, Wang X, Wang X, Chen R, Zhou P (2014) Analysis of eeg activity in response to binaural beats with different frequencies. Int J Psychophysiol 94(3):399–406

    Article  Google Scholar 

  45. Yi W, Qiu S, Wang K, Qi H, Zhang L, Zhou P, He F, Ming D (2014) Evaluation of eeg oscillatory patterns and cognitive process during simple and compound limb motor imagery. Plos One 9(12):e114853

    Article  Google Scholar 

  46. Wang Z-M, Zhou R, He Y, Guo X-M (2020) Functional integration and separation of brain network based on phase locking value during emotion processing. IEEE Trans Cognit Develop Syst

  47. Zhao S, Wang G, Yan T, Xiang J, Yu X, Li H, Wang B (2021) Sex differences in anatomical rich-club and structural–functional coupling in the human brain network. Cereb Cortex 31(4):1987–1997

    Article  Google Scholar 

  48. Sporns O, Rubinov M (2010) Complex network measures of brain connectivity: uses and interpretations. Neuroimage 52(3):1059–1069

    Article  Google Scholar 

  49. Strogatz SH, Watts DJ (1998) Collective dynamics of ‘small-world’networks. Neuroimage 393 (6684):440–442

    MATH  Google Scholar 

  50. Marchiori M, Latora V (2001) Efficient behavior of small-world networks. Neuroimage 87 (19):198701

    Google Scholar 

  51. Newman MEJ (2003) The structure and function of complex networks. SIAM Rev 45(2):167–256

    Article  MathSciNet  MATH  Google Scholar 

  52. Gao Q, Yi Y, Kang Q, Tian Z, Yu S (2022) Eeg-based emotion recognition with feature fusion networks. Int J Mach Learn Cybern 13(2):421–429

    Article  Google Scholar 

  53. Ali M, Mosa AH, Machot FA, Kyamakya K (2016) Eeg-based emotion recognition approach for e-healthcare applications. In: 2016 Eighth international conference on ubiquitous and future networks (ICUFN). IEEE, pp 946–950

  54. Sporns O, Bullmore E (2012) The economy of brain network organization. Nature Rev Neurosci 13(5):336–349

    Article  Google Scholar 

  55. Tuncer T, Dogan S, Baygin M, Rajendra Acharya U (2022) Tetromino pattern based accurate eeg emotion classification model. Artif Intell Med 123:102210

    Article  Google Scholar 

  56. Subasi A, Tuncer T, Dogan S (2021) A new fractal pattern feature generation function based emotion recognition method using eeg. Chaos, Solitons Fractals 144:110671

    Article  MathSciNet  Google Scholar 

  57. Dogan A, Akay M, Barua PD, Baygin M, Dogan S, Tuncer T, Dogru AH, Acharya UR (2021) Primepatnet87: Prime pattern and tunable q-factor wavelet transform techniques for automated accurate eeg emotion recognition. Comput Biol Med 138: 104867

    Article  Google Scholar 

  58. Heng X, Wang Z, Tong Y (2019) Phase-locking value based graph convolutional neural networks for emotion recognition. IEEE Access 7:93711–93722

    Article  Google Scholar 

  59. Fu G, Liu Y (2021) Emotion recognition by deeply learned multi-channel textual and eeg features. Futur Gener Comput Syst 119:1–6

    Article  Google Scholar 

  60. Xing X, Li Z, Xu T, Shu L, Hu B, Xu X (2019) Sae+ lstm: a new framework for emotion recognition from multi-channel eeg. Frontiers Neurorobotics 13:37

    Article  Google Scholar 

  61. Asghar MA, Khan MJ, Shahid H, Shorfuzzaman M, Xiong NN, Mehmood RM (2021) Semi-skipping layered gated unit and efficient network: hybrid deep feature selection method for edge computing in eeg-based emotion classification. IEEE Access 9:13378–13389

    Article  Google Scholar 

  62. Yin Y, Zheng X, Hu B, Zhang Y, Cui X (2021) Eeg emotion recognition using fusion model of graph convolutional neural networks and lstm. Appl Soft Comput 100:106954

    Article  Google Scholar 

  63. Liu Y, Chao H (2020) Emotion recognition from multi-channel eeg signals by exploiting the deep belief-conditional random field framework. IEEE Access 8:33002–33012

    Article  Google Scholar 

  64. Dai Y, Wang X, Zhang P, Zhang W, Chen J (2018) Sparsity constrained differential evolution enabled feature-channel-sample hybrid selection for daily-life eeg emotion recognition. Multimed Tools Appl 77 (17):21967–21994

    Article  Google Scholar 

  65. Xu H, Wang X, Li W, Wang H, Qing Bi (2019) Research on eeg channel selection method for emotion recognition. In: 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, pp 2528–2535

  66. Li JW, Barma S, Mak PU, Chen F, Li C, Li MT, Vai MI, Pun SH (2022) Single-channel selection for eeg-based emotion recognition using brain rhythm sequencing. IEEE J Biomed Health Inf 26(6):2493–2503

    Article  Google Scholar 

  67. Md AR, Md FH, Hossain M, Ahmmed R (2020) Employing pca and t-statistical approach for feature extraction and classification of emotion from multichannel eeg signal. Egyptian Inf J 21(1):23–35

    Article  Google Scholar 

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 62002287,in part by the National Natural Science Foundation of China under Grant 61373116,in the part by the Shanxi Provincial Key Research and Development Project under Grant 2022SF-037,in the part by the Xi’an University of Posts and Telecommunications under Grant CXJJZL2021017.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Xi’an University of Posts and Telecommunications, Xi’an, 710121, China

    Zhong-Min Wang, Zhe-Yu Chen & Jie Zhang

  2. Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi’an University of Posts and Telecommunications, Xi’an, 710121, China

    Zhong-Min Wang & Jie Zhang

  3. Xi’an Key Laboratory of Big Data and Intelligent Computing, Xi’an, 710121, Shaanxi, China

    Zhong-Min Wang & Jie Zhang

Authors
  1. Zhong-Min Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhe-Yu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhong-Min Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: A

Appendix: A

$$ \begin{array}{@{}rcl@{}} &&DE(x) \\ &=& - {\int}_{- \infty }^{+ \infty } \frac{1}{{\sqrt {2\pi {\sigma^{2}}} }}{e^{- \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}}}}\log \left( {\frac{1}{{\sqrt {2\pi {\sigma^{2}}} }}{e^{- \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}}}}} \right)dx\\ &=& - {\int}_{- \infty }^{+ \infty } \frac{1}{{\sqrt {2\pi {\sigma^{2}}} }}{e^{- \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}}}}\left( {\log \frac{1}{{\sqrt {2\pi {\sigma^{2}}} }} - \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}}} \right)dx\\ &=& - {\int}_{- \infty }^{+ \infty } \left( \log \frac{1}{{\sqrt {2\pi {\sigma^{2}}} }} \cdot \frac{1}{{\sqrt {2\pi {\sigma^{2}}} }}{e^{- \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}}}}\right)dx \\ &&- \left( \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}} \cdot \frac{1}{{\sqrt {2\pi {\sigma^{2}}} }}{e^{- \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}}}}\right)dx\\ &=& - {\int}_{- \infty }^{+ \infty } \log \frac{1}{{\sqrt {2\pi {\sigma^{2}}} }} \cdot \frac{1}{{\sqrt {2\pi {\sigma^{2}}} }}{e^{- \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}}}}dx \\ &&+ {\int}_{- \infty }^{+ \infty } \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}} \cdot \frac{1}{{\sqrt {2\pi {\sigma^{2}}} }}{e^{- \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}}}}dx\\ &=& -\log \frac{1}{\sqrt{2\pi \sigma^{2}}}{\int}_{- \infty }^{+ \infty } \frac{1}{{\sqrt {2\pi {\sigma^{2}}} }}{e^{- \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}}}}dx \\ && + \frac{1}{{2{\sigma^{2}}}}{\int}_{- \infty }^{+ \infty } {{(x - \mu )}^{2}}\frac{1}{{\sqrt {2\pi {\sigma^{2}}} }}{e^{- \frac{{{{(x - \mu )}^{2}}}}{{2{\sigma^{2}}}}}}dx\\ &=& \frac{1}{2}\log 2\pi {\sigma^{2}} + \frac{1}{{2{\sigma^{2}}}}\times {\sigma^{2}}\\ &=& \frac{1}{2}\log 2\pi {\sigma^{2}} + \frac{1}{2}\\ &=& \frac{1}{2}\log 2\pi {\sigma^{2}} + \frac{1}{2} \log e\\ &=& \frac{1}{2}\log 2\pi e{\sigma^{2}} \end{array} $$

Where \({\int \limits }_{- \infty }^{+ \infty } \frac {1}{{\sqrt {2\pi {\sigma ^{2}}} }}{e^{- \frac {{{{(x - \mu )}^{2}}}}{{2{\sigma ^{2}}}}}}dx\) obeys the Gaussian distribution, \({\int \limits }_{- \infty }^{+ \infty } \frac {1}{{\sqrt {2\pi {\sigma ^{2}}} }}{e^{- \frac {{{{(x - \mu )}^{2}}}}{{2{\sigma ^{2}}}}}}dx = 1 \) \({\int \limits }_{- \infty }^{+ \infty } {{(x - \mu )}^{2}}\frac {1}{{\sqrt {2\pi {\sigma ^{2}}} }}{e^{- \frac {{{{(x - \mu )}^{2}}}}{{2{\sigma ^{2}}}}}}dx =\\ {\int \limits }_{- \infty }^{+ \infty } {{x}^{2}}\frac {1}{{\sqrt {2\pi {\sigma ^{2}}} }}{e^{- \frac {{{{x}^{2}}}}{{2{\sigma ^{2}}}}}}dx=\sigma ^{2}\)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, ZM., Chen, ZY. & Zhang, J. EEG emotion recognition based on PLV-rich-club dynamic brain function network. Appl Intell 53, 17327–17345 (2023). https://doi.org/10.1007/s10489-022-04366-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04366-7

Keywords