CN114699078A - Emotion recognition method and system based on small number of channel EEG signals - Google Patents

Abstract

The invention belongs to the field of brain-computer interfaces, and particularly relates to an emotion recognition method and system based on a small number of channel EEG (electroencephalogram), which comprises the steps of acquiring 4 channels of EEG signals of a user scalp and preprocessing the EEG signals; converting the preprocessed electroencephalogram signals into basic signals, and decomposing and reconstructing the basic signals to obtain a series of component signals; inputting the smoothed features into a trained classification model to detect the emotion of the user; the invention uses the EEG signals with fewer channels to detect the emotion of the user, the user only needs to carry out emotion mobilization according to the actual scene, the PC analyzes the collected EEG signals of the cerebral cortex of the user, and the final characteristics are input into the classifier for classification through preprocessing and characteristic extraction, so that the emotion category of the user at that time can be reflected.

Description

Emotion recognition method and system based on small number of channel EEG signals

Technical Field

The invention belongs to the field of brain-computer interfaces, and particularly relates to a method and a system for emotion recognition based on a small number of channel EEG.

Background

Emotions play an important role in daily life, and encompass psychological states such as positive emotions, negative emotions, alertness and fatigue, which affect a person's physiological state, and researchers in various fields analyze the psychological state of the person using data such as facial expressions, voice signals, short messages, electrocardiograms and electroencephalograms, and find that electroencephalogram signals are least affected by differences in personal characteristics. However, the channel of the electroencephalogram signal has noise, and the signal-to-noise ratio is low, so that the analysis and the processing of the electroencephalogram signal are limited.

With the rapid development of computers, wireless communication, and intelligent computing technologies, researchers are constantly developing intelligent wearable devices to improve people's lives. However, currently, few wearable devices for emotion recognition based on electroencephalogram (EEG) are in the market, which is caused by limitations of electroencephalogram signal acquisition technologies, noise of electroencephalograms acquired by a head-mounted electroencephalogram cap is large, and meanwhile, a data acquisition mode and an emotion recognition algorithm based on multiple channels in a laboratory are complex at present, and are difficult to apply to wearable devices.

In recent years, most of researches on emotion recognition based on electroencephalogram signals by many researchers are focused on improving classification accuracy under the condition of a large number of channels, and the algorithms are difficult to apply to wearable equipment. Meanwhile, electroencephalogram data of all channels of the cerebral cortex contain relevant information of all brain areas, but for specific tasks, the application of all channels to wearable equipment is not facilitated, and the application of algorithms to the mass market is facilitated. Therefore, the practicability of the algorithm on the wearable equipment can be greatly improved by adopting fewer channels to collect real-time data and finish classification tasks.

Disclosure of Invention

In order to solve the problems, the invention provides a method and a system for emotion recognition based on a small number of channel EEG.

In a first aspect, the present invention provides a method for emotion recognition based on a small number of channel EEG, comprising:

s1, acquiring electroencephalogram signals of 4 channels of a user scalp, and preprocessing the electroencephalogram signals;

s2, converting the preprocessed electroencephalogram signals into basic signals, and decomposing and reconstructing the basic signals to obtain a series of component signals;

s3, extracting differential entropy of the basic signal and the component signal as features, and smoothing the features;

and S4, inputting the smoothed features into a trained classification model to detect the emotion of the user.

Further, the 4 channels of the user's scalp are the FP1, FP2 channel located at the prefrontal lobe and the T7, T8 channel located at the temporal lobe.

Further, preprocessing is carried out on electroencephalogram signals of FP1, FP2, T7 and T8 channels, and basic signals are constructed by adopting potential differences among channel electrodes, namely, 6 basic signals are obtained by subtracting every two of the four preprocessed electroencephalogram signals and are respectively FP1-T7, FP1-T8, FP1-FP2, FP2-T7, FP2-T8 and T7-T8.

Further, the process of preprocessing the electroencephalogram signals is as follows:

performing down-sampling on the electroencephalogram signal to obtain an electroencephalogram signal of 200 Hz;

and filtering the 200Hz electroencephalogram signal by adopting a Butterworth filter to obtain the filtered electroencephalogram signal.

Further, the original time scale decomposition, discrete wavelet transformation, variational modal decomposition and phase space reconstruction are adopted to decompose and reconstruct the basic signal, thereby expanding the existing mode, wherein:

inherent time scale decomposition processes the basic signal in an affine linear transformation mode to obtain a PRC1 rotation component signal and a PRC2 rotation component signal;

the discrete wavelet transform extracts component signals of four physiological electroencephalogram wave bands including Gamma, Beta, Alpha and Theta from a basic signal by scaling and shifting a mother wavelet function;

the variational modal decomposition adopts a parallel method to extract an inherent modal function from the basic signal to obtain 4 component signals of VMD1, VMD2, VMD3 and VMD 4;

and reconstructing the basic signal by phase space reconstruction to obtain a 3D-PSR-ED component signal.

Further, filtering components irrelevant to the emotional state in the features by adopting a linear dynamic system to finish feature smoothing, wherein the linear dynamic system is expressed as:

x_t＝z_t+w_t；

z_t＝Az_t-1+v_t；

wherein x_tRepresenting an observed variable, z_tRepresenting hidden emotion variables, A represents a transition matrix, w_tRepresents a mean value of

Gaussian noise with variance Q, v_tRepresents a mean value of

Gaussian noise with variance R.

On the basis of the first aspect, the invention provides an emotion recognition system based on a small number of channel EEG signals, which comprises:

the electroencephalogram signal collection module is used for collecting electroencephalogram signals generated in FP1, FP2, T7 and T84 channels of the scalp during emotion detection of a target user;

the signal preprocessing module is used for processing the 4-channel electroencephalogram signals acquired by the electroencephalogram signal collecting module and converting the 4-channel electroencephalogram signals into basic signals;

the signal decomposition and reconstruction module is used for decomposing and reconstructing the basic signal output by the signal preprocessing module to obtain a plurality of component signals;

the characteristic extraction module is used for extracting the characteristics of the basic signal and the component signal and smoothing the characteristics;

and the classification module is used for inputting the features after the smoothing treatment into the trained classification model and detecting the emotion of the user.

The invention has the beneficial effects that:

the invention provides a sentiment recognition method and system based on a small number of channel EEG, which detects the sentiment of a user by collecting EEG signals of a small number of channels, the user only needs to do sentiment mobilization according to the actual scene, a PC analyzes the collected EEG signals of the cerebral cortex of the user, and the final characteristics are input into a classifier for classification through preprocessing and characteristic extraction, so that the sentiment category of the user at the moment can be reflected.

Drawings

Fig. 1 is a flow chart of a method of emotion recognition based on a few-channel EEG of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.

In order to improve emotion recognition precision and improve practicability of an algorithm on wearable equipment, the invention provides an emotion recognition method and system based on a small number of channel EEGs.

In one embodiment, a method for emotion recognition based on a small number of channel EEG, as shown in fig. 1, includes:

s1, acquiring electroencephalogram signals of 4 channels of a user scalp, and preprocessing the electroencephalogram signals;

s2, converting the preprocessed electroencephalogram signals into basic signals, and decomposing and reconstructing the basic signals to obtain a series of component signals;

s3, extracting differential entropy of the basic signal and the component signal as features, and smoothing the features;

and S4, inputting the smoothed features into a trained classification model to detect the emotion of the user.

Preferably, in order to accurately identify the emotion category of the user and simultaneously meet the requirement of convenience in designing the portable device, the FP1 and FP2 channels located in the forehead lobe and the FP 7 and T8 channels located in the temporal lobe are selected in the embodiment, and the 4 channels are located to provide convenience for designing wearable products such as head rings, earphones and glasses. For the electroencephalogram signals generated by the 4 channels of the mobile phone, in order to keep the spatial asymmetry of the electroencephalogram signals in the brain area as much as possible, potential differences among the channel electrodes are adopted to construct 6 original basic signals which are respectively as follows: each original basic signal is sliced according to a time period of 5 seconds, namely FP1-T7, FP1-T8, FP1-FP2, FP2-T7, FP2-T8 and T7-T8, and slices of 6 original basic signals in a time period are used as a group of basic signals for subsequent processing and feature extraction.

Preferably, in this embodiment, the process of preprocessing the electroencephalogram signal before converting it into the basic signal is as follows:

performing down-sampling on the electroencephalogram signal to obtain an electroencephalogram signal of 200 Hz;

filtering the 200Hz electroencephalogram signal by adopting a Butterworth filter, and intercepting the electroencephalogram signal after filtering in a frequency range of 0-49 Hz.

In one embodiment, the fundamental signal is decomposed and reconstructed using intrinsic time scale decomposition, discrete wavelet transform, variational modal decomposition, and phase space reconstruction.

Specifically, inherent time scale Decomposition (ITD), also called ITD method, can adaptively process nonlinear non-stationary signals, strictly limit the endpoint effect at the endpoint, and does not need to propagate the contaminated whole-segment data inwards, thereby overcoming the problem that the Empirical Mode Decomposition (EMD) method propagates the endpoint effect inwards to contaminate the whole-segment data. The ITD method processes a signal in the form of affine linear transformation, decomposes the signal into a plurality of non-interfering Proper Rotation Components (PRCs) and monotonic trend components in the frequency domain, and obtains better transient characteristics of the signal after decomposition. Since emotion recognition is mainly related to the high frequency signals of the human brain, the first two proper rotation components PRC1, PRC2 are mainly selected as the main proper rotation components in the present embodiment.

In particular, Discrete Wavelet Transform (DWT) is a discretization of the scale and translation of the basic Wavelet, which is decomposed into a series of functions called wavelets by scaling and shifting the mother Wavelet, while forming a decomposed set of Wavelet coefficients. In the embodiment, 5-level DWT with three-order Daubechies (db3) wavelet functions is used, five physiological electroencephalogram bands are extracted, namely Gamma (31-48Hz), Beta (14-30), Alpha (8-13Hz), Theta (4-7Hz) and Delta (0.5-3Hz), and then Delta wave is removed, because the appearance of the wave is often related to human sleep, and the wave plays a very small role in the emotion recognition task.

Specifically, the Variational Modal Decomposition (VMD) has a strict mathematical model and a solid theoretical basis, can overcome the disadvantages of frequency aliasing and the like in the EMD and Local Mean Decomposition (LMD), and has a better noise filtering effect. Instead of using a recursive method, the VMD extracts the natural mode functions from the signal using a parallel method. The number of decomposition components, K, in the VMD needs to be predefined. If K is too small, it is not beneficial to get more modalities, but if K is too large, so-called over-segmentation, increases the computational load. Fortunately, one good property of VMDs is that when the signal is over-partitioned, the redundant pattern is essentially a noise component. In consideration of performance and noise reduction requirements, K is set to 4 in the present embodiment, that is, the first 4 natural mode components VMD1, VMD2, VMD3 and VMD4 are selected as main modes.

In particular, since the brain electrical signal has a strong continuity in time, it is sometimes necessary to search for patterns in time series and high-dimensional transformation of the time series. Although the phase space reconstruction method uses the values of a variable at different moments to form a phase space, the change of the variable of the power system is naturally related to the interaction of the variable and other variables of the system, namely the change of the variable along with time implies the dynamics law of the whole system. Therefore, the trajectory of the reconstructed phase space also reflects the evolution law of the system state. The phase space reconstruction technique has two key parameters: the embedded dimension D and delay time τ, in this embodiment, set D to 3 and τ to 1, acquire the 3D-PSR-ED component signal.

In one embodiment, differential entropy of the base signal and the component signal is extracted as features and the features are smoothed, that is, the base signal, the phase-space reconstructed signal (3D-PSR-ED), the inherent rotation amount decomposed by the ITD (PRC1, PRC2), the rhythm decomposed by the DWT (Gamma, Beta, Alpha, Theta), the component decomposed by the VMD (VMD1, VMD2, VMD3, VMD4), the differential entropy is extracted as features, each base signal can extract 1x12 features after the above processing, and finally 6 base signals obtain 6x12 features. For the extracted features, a Linear Dynamic System (LDS) method is applied over a time series to filter components that are not related to emotional state, thereby completing feature smoothing.

Specifically, the linear dynamic system is represented as:

x_t＝z_t+w_t；

z_t＝Az_t-1+v_t；

wherein x_tRepresents an observed variable, z_tRepresenting hidden emotion variables, A represents a transition matrix, w_tRepresents a mean value of

Gaussian noise with variance Q, v_tRepresents a mean value of

Gaussian noise with variance R.

On the basis, the invention provides an emotion recognition system based on a small number of channel EEG signals, which comprises:

the electroencephalogram signal collection module is used for collecting electroencephalogram signals generated in FP1, FP2, T7 and T84 channels of the scalp during emotion detection of a target user;

the signal preprocessing module is used for processing the 4-channel electroencephalogram signals acquired by the electroencephalogram signal collecting module and converting the 4-channel electroencephalogram signals into basic signals;

the signal decomposition and reconstruction module is used for decomposing and reconstructing the basic signal output by the signal preprocessing module to obtain a plurality of component signals;

the characteristic extraction module is used for extracting the characteristics of the basic signal and the component signal and smoothing the characteristics;

and the classification module is used for inputting the features after the smoothing treatment into the trained classification model and detecting the emotion of the user.

Specifically, a simple CNN structure similar to LeNet-5 is adopted in the classification module. Compared with LeNet5, the CNN model adopted in the embodiment achieves the purpose of reducing model parameters by reducing the number of layers and the number of convolution kernels, so that the model is more suitable for a small sample classification task based on artificial features. The specific structure of the model is shown in table 1, the CNN model includes three convolutional layers, a Convolution, two Pooling layers Max power, and a Linear layer, the Convolution kernels of the convolutional layers and the Pooling layers are all 2, the input data is subjected to maximum Pooling after passing through the two convolutional layers in sequence, the data subjected to maximum Pooling is convolved and pooled again, the result of the second Pooling is input into the Linear layer, and emotion recognition is performed through Softmax.

Table 1 architecture of CNN model of the invention

In one embodiment, experiments were conducted using a public emotion EEG data set named SEED, which uses movie fragments as emotion-inducing materials, including three classes: positive, neutral and negative emotions, participants watched 15 videos per set of experiments, 5 for each emotion type. The SEED data set contained 62 channels (electrodes) of electroencephalographic affective signals recorded by 15 participants (7 males and 8 females). In this embodiment, an experimental paradigm similar to the SEED data set is adopted, and data of 10 participants are collected in a laboratory, and since the method of the present invention only uses 4 channels, only 14 channels are collected during data collection, which are: FP1, FP2, F7, F8, F3, F4, FC5, FC6, T7, T8, P7, P8, O1, O2. In the embodiment, the experiment is carried out by adopting two modes of Type1 and Type2, and the experiment result is recorded by using Accuracy1 and Accuracy 2.

Type 1: collecting electroencephalogram signals generated by cerebral cortex when each participant watches 15 videos, dividing the electroencephalogram signals by taking 5 seconds as one trail, extracting all the trails, randomly disordering all the trails, extracting 20% of data in various emotion samples every time to serve as a test set, and taking the rest of data as a training set to finish a 5-fold cross validation experiment.

And (2) selecting the electroencephalogram signals acquired when each participant watches 3 videos (positive, negative and neutral) each time, taking all trail extracted from the electroencephalogram signals as a test set to watch the electroencephalograms acquired when other videos are watched as a training set, and completing a 5-fold cross validation experiment.

In Type1 experiments, the method achieved average classification accuracy of 98.67% and 99.97% on the SEED data set and the data set of the present invention, respectively. While Type2 experiments ensure that the training set and the test set come from completely different time periods, which is much closer to a real usage scenario. An average accuracy of 90.77% was obtained on the SEED data and an average accuracy of 91.38% was obtained on the inventive data set.

Experimental results show that the optimal performance can be obtained in the Type1 mode. However, since the emotional state of a person has a certain duration in time, it does not change significantly in a very short time. The operation of disturbing the whole data in the Type1 experiment can cause the trails originally extracted in a period of continuous time to be simultaneously distributed in the training set and the test set, thereby causing data leakage. This is also a significant reason why the results of either the SEED data set or the data set of the present invention were significantly better than the Type2 in the Type1 experiment. In addition, the difference between participants is also an important reason for the difference of experimental results, which may be caused by subjective personal consciousness, growth environment and physiological state, and also can be caused by objective experimental environment.

In order to verify the effectiveness of the method, according to a classical electroencephalogram emotion recognition method, experiments are carried out under a 62-channel SEED data set and a 14-channel data set, and PSD (phase position detector) characteristics, DE (DE) characteristics and normalized first-order difference (1) are respectively extracted from each channel of five specific frequency bands (Gamma, Beta, Alpha, Theta and Delta) of the electroencephalogram_STOD) are input into the SVM, Type1 and Type2 experiments are respectively carried out, the method is compared with a classical method on the same characteristics and classifiers (the classical method usually only adopts discrete wavelet transform to carry out signal decomposition), PSD characteristics and DE characteristics are obtained, LDS characteristics smoothing is carried out on the DE characteristics, and then the results are applied to the SVM and the text CNN model. The results are shown in tables 2 and 3:

table 2 comparison of performance using 4 channels on SEED data set with conventional 62 channels

Table 3 comparison of performance of the inventive data set using 4 channels with the conventional 14 channels

As table 1, table 2 results show that although only 4 channels of electroencephalographic data were collected in the methods herein, the present invention achieved average accuracy better than the 62-channel SEED data set with the same experimental patterns, features and classifiers. On the data set of the invention, the method of the invention is superior to the traditional method when the characteristic is 1stOD, and the performance difference of the method of the invention is very small compared with the multi-channel of the traditional method when the characteristic is PSD or DE.

In addition, under the DE characteristic, the comparison of the results of using SVM and CNN after LDS characteristic smoothing shows that great performance improvement can be obtained through the improvement of the characteristic smoothing and classification method.

Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (7)

1. A method for emotion recognition based on a small number of channel EEG signals, comprising:

s1, acquiring electroencephalogram signals of 4 channels of a user scalp, and preprocessing the electroencephalogram signals;

s2, converting the preprocessed electroencephalogram signals into basic signals, and decomposing and reconstructing the basic signals to obtain a series of component signals;

s3, extracting differential entropy of the basic signal and the component signal as features, and smoothing the features;

and S4, inputting the smoothed features into a trained classification model to detect the emotion of the user.

2. The method of claim 1, wherein the 4 channels of the user's scalp are the FP1, FP2 channel located in the prefrontal lobe and the T7, T8 channel located in the temporal lobe.

3. The method as claimed in claim 2, wherein the electroencephalogram signals of the FP1, FP2, T7 and T8 channels are preprocessed, and the potential difference between the channel electrodes is used to construct a basic signal, i.e. the four preprocessed electroencephalogram signals are subtracted from each other to obtain 6 basic signals, which are FP1-T7, FP1-T8, FP1-FP2, FP2-T7, FP2-T8 and T7-T8, respectively.

4. The method for emotion recognition based on EEG signals of a small number of channels as claimed in claim 1 or 3, wherein the preprocessing of EEG signals is performed by:

performing down-sampling on the electroencephalogram signal to obtain an electroencephalogram signal of 200 Hz;

and filtering the 200Hz electroencephalogram signal by adopting a Butterworth filter to obtain the filtered electroencephalogram signal.

5. The method of claim 1, wherein the fundamental signal is decomposed and reconstructed using intrinsic time scale decomposition, discrete wavelet transform, variational modal decomposition, and phase space reconstruction, respectively, wherein:

inherent time scale decomposition processes the basic signal in an affine linear transformation mode to obtain a PRC1 rotation component signal and a PRC2 rotation component signal;

extracting component signals of four physiological electroencephalogram wave bands including Gamma, Beta, Alpha and Theta from the basic signals by adopting discrete wavelet transformation;

the variational modal decomposition adopts a parallel method to extract an inherent modal function from the basic signal to obtain 4 component signals of VMD1, VMD2, VMD3 and VMD 4;

and reconstructing the basic signal by phase space reconstruction to obtain a 3D-PSR-ED component signal.

6. The method of claim 1, wherein the linear dynamics system is used to filter components of the features that are not associated with emotional states, and perform feature smoothing, wherein the linear dynamics system is represented as:

x_t＝z_t+w_t；

z_t＝Az_t-1+v_t；

wherein x_tRepresenting an observed variable, z_tRepresenting hidden emotion variables, A represents a transition matrix, w_tRepresents a mean value of

Gaussian noise with variance Q, v_tRepresents a mean value of

Gaussian noise with variance R.

7. An emotion recognition system based on a small number of channel EEG signals, comprising:

the electroencephalogram signal collection module is used for collecting electroencephalogram signals generated in FP1, FP2, T7 and T84 channels of the scalp during emotion detection of a target user;

the signal preprocessing module is used for processing the 4-channel electroencephalogram signals acquired by the electroencephalogram signal collecting module and converting the 4-channel electroencephalogram signals into basic signals;

the signal decomposition and reconstruction module is used for decomposing and reconstructing the basic signal output by the signal preprocessing module to obtain a plurality of component signals;

the characteristic extraction module is used for extracting the characteristics of the basic signal and the component signal and smoothing the characteristics;

and the classification module is used for inputting the features after the smoothing treatment into the trained classification model and detecting the emotion of the user.

Images