CN115017960B - Electroencephalogram signal classification method based on space-time combined MLP network and application - Google Patents
Electroencephalogram signal classification method based on space-time combined MLP network and application Download PDFInfo
- Publication number
- CN115017960B CN115017960B CN202210849649.5A CN202210849649A CN115017960B CN 115017960 B CN115017960 B CN 115017960B CN 202210849649 A CN202210849649 A CN 202210849649A CN 115017960 B CN115017960 B CN 115017960B
- Authority
- CN
- China
- Prior art keywords
- electroencephalogram
- layer
- sample
- channel
- mlp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 210000004556 brain Anatomy 0.000 claims abstract description 33
- 238000012549 training Methods 0.000 claims abstract description 21
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 9
- 238000001914 filtration Methods 0.000 claims abstract description 4
- 238000013145 classification model Methods 0.000 claims abstract description 3
- 230000006870 function Effects 0.000 claims description 15
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000009467 reduction Effects 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 9
- 230000005611 electricity Effects 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 5
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 230000010354 integration Effects 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000001131 transforming effect Effects 0.000 claims description 2
- 238000007781 pre-processing Methods 0.000 abstract description 4
- 238000005070 sampling Methods 0.000 abstract description 2
- 238000013136 deep learning model Methods 0.000 abstract 1
- 239000010410 layer Substances 0.000 description 75
- 238000013135 deep learning Methods 0.000 description 6
- 230000003993 interaction Effects 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 239000002346 layers by function Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000000052 comparative effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000000537 electroencephalography Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 108010076504 Protein Sorting Signals Proteins 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008909 emotion recognition Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000007917 intracranial administration Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 210000004761 scalp Anatomy 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/14—Fourier, Walsh or analogous domain transformations, e.g. Laplace, Hilbert, Karhunen-Loeve, transforms
- G06F17/141—Discrete Fourier transforms
- G06F17/142—Fast Fourier transforms, e.g. using a Cooley-Tukey type algorithm
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Discrete Mathematics (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses an electroencephalogram classification method based on a space-time combined MLP network and application thereof, comprising the following steps: 1, preprocessing original electroencephalogram data, including data selection to be classified, sliding window slicing, data up-sampling and data input shape selection; 2, establishing a deep learning model of the multi-layer perceptron network; 3, in a training stage, inputting data and continuously optimizing model parameters through cross entropy loss to obtain a final classification model for classifying the electroencephalogram signals to be tested; and 4, calibrating the predicted result sequence of the model by using a moving average filtering algorithm. The invention applies the space-time information of the brain electrical data to the multi-layer perceptron network, and can obviously improve the accuracy of the classification of the brain electrical signals, thereby increasing the application value of the brain electrical signals in the fields of medical treatment and the like.
Description
Technical Field
The invention relates to the field of electroencephalogram signal classification, in particular to a method for predicting and classifying electroencephalogram signals by an MLP (multi-layer pulse plate) network combining time and space information of multichannel electroencephalogram signals.
Background
Electroencephalography (EEG) is a physiological technique for recording brain physiological electrical signals. The recognition and prediction of physiological and psychological states from patterns of neural activity observed in scalp and intracranial electroencephalograms is widely used in the field of brain-computer interfaces for emotion recognition, motor imagery, medical health, etc. The linear or nonlinear features, such as autoregressive coefficients and the li-apuno index, are manually extracted using conventional machine learning methods, which have met with some success in a tightly controlled experimental environment. However, these manually extracted features often require a researcher to have a rich expertise and to conduct a large number of experimental attempts. In addition, in a real electroencephalogram recording affected by various artifacts, manually extracted features often cover only part of the electroencephalogram information, resulting in poor robustness of the system.
The deep learning algorithm is stimulated to be widely applied to electroencephalogram signal classification prediction due to the excellent generalization capability and the strong capability of automatically learning high-efficiency characteristics. At present, most deep learning methods for classifying electroencephalogram signals perform feature preprocessing, such as short-time fourier transform, public space mode and the like. These preprocessing operations on the raw brain electrical, while allowing for more "clean" data, may also lose some important information. Models using feature preprocessing and direct use of the original electroencephalogram signals in recent years generally have more complex architectures and larger cores, resulting in greater memory resource consumption and computational power.
Currently, most deep learning algorithms for electroencephalogram classification are generally used as feature classifiers. Researchers extract time domain features, frequency domain features or time-frequency domain features from the electroencephalogram signals through the existing expertise, and then use a deep learning algorithm to carry out classification tasks. While this approach also achieves good classification performance, it requires deep mathematical knowledge for feature extraction and it ignores the powerful data driving capabilities of deep learning algorithms. The simultaneously extracted feature, while somewhat a better data representation, also loses much of the spatial and temporal correlation information present in the original multi-channel electroencephalogram data. There are also few deep learning algorithms using end-to-end architecture, but none of them fully exploit the spatial and temporal correlation information present in multichannel electroencephalograms. Due to the limitation of various conditions, the total data amount of the electroencephalogram signals is seriously insufficient, and the development of a classification method of the electroencephalogram signals is greatly limited. And the development of classification methods is greatly limited due to the serious data imbalance problem of different types of electroencephalogram data.
Disclosure of Invention
In order to overcome the defects, the invention provides an electroencephalogram signal classification method based on a space-time combined MLP network and application thereof, so that space-related information and time-related information of multi-channel electroencephalogram signals can be extracted from original electroencephalogram signals and applied to a multi-layer perceptron network, thereby remarkably improving the accuracy of electroencephalogram signal classification and increasing the application value of the electroencephalogram signals in the fields of medical treatment and the like.
In order to achieve the aim of the invention, the invention adopts the following technical scheme:
the invention relates to an electroencephalogram classification method based on a space-time combined MLP network, which is characterized by comprising the following steps of:
step 1, acquiring an electroencephalogram data set with labeling category information, performing sliding slice processing on electroencephalogram signals of each category, reconstructing the input shape of the sliced electroencephalogram signals, obtaining N segments of electroencephalogram signal samples with total duration of T, and marking the N segments of electroencephalogram signal samples as a training sample set X= { X 1 ,X 2 ,...,X i ,...,X N -and the tag set of training sample X is noted y= { Y 1 ,Y 2 ,...,Y i ,...,Y N -a }; wherein X is i ∈R C×1×L Representing an ith section of brain electrical signal sample after input shape reconstruction, wherein C represents the channel number of the brain electrical signal sample, and L represents the length of the brain electrical signal sample; y is Y i For the ith section of electroencephalogram signal sample X i Corresponding toIs a label of (2);
step 2, establishing a space-time joint MLP (multi-layer protocol) network, comprising the following steps: the system comprises a denoising weighting module, a space-time joint MLP module and a classification module;
step 2.1, the denoising weighting module includes: the device comprises a denoising layer, a weighting layer and a dimension reduction layer;
the denoising layer comprises a manually set matrix filter with randomly initialized element values, and firstly trains a sample set X= { X through fast Fourier transform 1 ,X 2 ,...,X i ,...,X N Converting the time domain into the frequency domain, multiplying the training sample set converted into the frequency domain with a matrix filter capable of learning to obtain a denoised training sample set, and converting the denoised training sample set into the time domain by inverse fast Fourier transform to obtain a time domain denoised electroencephalogram sample sequenceWherein (1)>Representing the denoised ith section of time domain brain signal sample;
sequencing brain electrical signalsConverting from three dimensions to two dimensions, thereby obtaining a two-dimensional denoising brain electricity sample sequence +.>Wherein (1)>Representing a two-dimensional ith section of electroencephalogram signal sample;
the weighting layer comprises a channel weight matrix which is manually preset and has a learnable diagonal element value. The weighting layer firstly carries out two-dimensional electroencephalogram sample sequenceAnd the channel weightAfter matrix multiplication, a weighted sequence is obtained>Wherein (1)>Representing an ith section of electroencephalogram signal sample after channel weighting;
the dimension reduction layer comprises a group of 1 xk convolution kernels and weights the electroencephalogram sample sequence of the channelRemoving redundant information in time dimension (length dimension) to obtain redundancy-removed brain electricity sample sequence +.>Wherein,representing an ith section of electroencephalogram sample after redundant information is removed;
step 2.2, the space-time combined MLP module comprises: an inter-channel MLP layer and an intra-channel MLP layer;
the inter-channel MLP layer sequentially comprises: layer norm layer, transform full connection layer, gel nonlinear activation function and restore full connection layer;
the layer norm layer pair redundancy-removing brain electricity sample sequenceAfter normalization processing, obtaining a space-related electroencephalogram sample sequence ∈after processing of transforming the full-connection layer, GELU activation function and restoring the full-connection layer in sequence>Wherein (1)>Representing an ith section of electroencephalogram signal sample which is subjected to extraction, integration and channel space correlation;
the structure of the intra-channel MLP layer is the same as that of the inter-channel MLP layer, and the brain electricity sample sequence is related to the spaceAfter normalization processing, obtaining a time information electroencephalogram sample sequence +.>Wherein (1)>Representing an ith section of electroencephalogram signal sample extracted through time information in a channel;
step 2.3, the classification module includes: an averaging pooling layer, a full connectivity layer and a Softmax layer;
time information electroencephalogram sample sequenceAfter the treatment of the average pooling layer and the full connection layer in sequence, obtaining the score of each section of electroencephalogram signal sample corresponding to each category, and finally converting the score of each section of electroencephalogram signal sample corresponding to each category into a probability value of each category through the Softmax layer, and selecting the maximum probability value as a prediction classification result of each section of electroencephalogram signal sample;
step 3, model training:
based on the training sample set X and the label set Y, cross entropy is adopted as a loss function, an ADAM optimizer is utilized to train the space-time combined MLP network, and gradient of the loss function is calculated to update network parameters until the maximum iteration number or the loss function convergence is reached, so that a trained electroencephalogram signal classification model is obtained;
and 4, calibrating a predicted result sequence of the model by using a moving average filtering algorithm:
taking an ith section of electroencephalogram sample X i And the following M-1 section of brain electricitySample { X i+1 ,X i+2 ,...,X i+M-1 The average value of each kind of probability value corresponding to each section of electroencephalogram signal sample in the } is correspondingly taken as X of the ith section of sample i Is a probability value for each category of (a).
The invention provides an electronic device, which comprises a memory and a processor, and is characterized in that the memory is used for storing a program for supporting the processor to execute the electroencephalogram classification method, and the processor is configured to execute the program stored in the memory.
The invention relates to a computer readable storage medium, which is stored with a computer program, characterized in that the computer program is executed by a processor to execute the steps of the electroencephalogram classification method.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention provides a new end-to-end electroencephalogram signal classification method based on an MLP network, which designs a denoising weighting module before electroencephalogram signal classification, and comprises three independent functional layers: the denoising layer is used for reducing various artifact noises in the original electroencephalogram signals; the weighting layer automatically learns Xi Chugeng channels from multi-channel electroencephalogram to be beneficial to making correct inferences and strengthens the effect of the channels beneficial to correct classification by giving larger weights; the dimension reduction layer is used for eliminating redundant information in the electroencephalogram signals, so that the robustness of the electroencephalogram signal classification result is improved.
2. Aiming at the characteristics of multichannel electroencephalogram signals, the invention designs a space-time combined MLP module which comprises two independent functional layers, wherein the two independent functional layers comprise: the inter-channel MLP layer is used for extracting and integrating spatial correlation among electroencephalogram signals of different channels; the intra-channel MLP layer is used for extracting time information of the electroencephalogram data of each channel, and the generalization performance of the model is remarkably improved by extracting time and space associated information of the integrated electroencephalogram signals.
Drawings
FIG. 1 is a schematic diagram of a network architecture according to the present invention;
FIG. 2 is a schematic diagram of a denoising weighting module according to the present invention;
FIG. 3 is a schematic diagram of a space-time interactive MLP module according to the present invention;
fig. 4 is a schematic of the input and output of the denoising layer of the present invention (data from CHB-MIT dataset subject 5).
Detailed Description
In the embodiment, in the electroencephalogram signal classification method based on the space-time interaction MLP network, firstly, noise in the electroencephalogram signal is reduced through a noise removal weighting module, the effect of an important channel electroencephalogram signal is enhanced, and redundant information in the electroencephalogram signal is removed; extracting and integrating spatial association and time information of the multi-channel electroencephalogram through a space-time interaction MLP module; the data sequence is converted into a prediction result through a classification module, as shown in fig. 1, specifically, the method comprises the following steps:
step 1, acquiring an electroencephalogram data set with labeling category information, selecting data of the electroencephalogram data to obtain electroencephalogram data of C channels, slicing the electroencephalogram data of the C channels through a sliding window, reconstructing the input shape of the sliced electroencephalogram data to obtain N segments of electroencephalogram data samples with total duration of T, and marking the N segments of electroencephalogram data as a training sample set X= { X 1 ,X 2 ,...,X i ,...,X N And training sample X's tag set y= { Y 1 ,Y 2 ,...,Y i ,...,Y N -a }; wherein X is i ∈R C ×1×L Representing an ith section of brain electrical signal sample after input shape reconstruction, wherein C represents the channel number of brain electrical signals, and L represents the length of the brain electrical signal sample; y is Y i For the ith section of electroencephalogram signal sample X i The corresponding label, in this embodiment, has an electroencephalogram signal channel number of 22, a sliding window length of 30s, and an electroencephalogram signal sampling rate of 256Hz per second; the method uses two published electroencephalogram datasets: a CHB-MIT data set and a Kaggle data set;
step 2, establishing a space-time joint MLP (multi-layer protocol) network, wherein the structure is shown in fig. 1 and comprises the following steps: the system comprises a denoising weighting module, a space-time joint MLP module and a classification module;
as shown in fig. 2, the denoising weighting module includes: the system comprises a denoising layer, a weighting layer and a dimension reduction layer.
The denoising layer comprises fast Fourier transform, inverse fast Fourier transform and a learnable matrix filter; the weighting layer comprises a learnable channel weight matrix; the dimension reduction layer comprises a group of convolution kernels;
as shown in fig. 3, the spatio-temporal joint MLP module includes: inter-channel MLP layers and intra-channel MLP layers.
The inter-channel MLP layer and the intra-channel MLP layer have the same architecture and comprise a layer norm layer, a conversion full-connection layer, a GELU nonlinear activation function and a restoration full-connection layer, wherein the difference is that the inter-channel MLP layer acts on the space dimension of the electroencephalogram data, and the intra-channel MLP layer acts on the time dimension of the electroencephalogram signal;
the classification module comprises: an averaging pooling layer, a full connectivity layer and a Softmax layer.
Step 2.1, initializing model parameters:
before training starts, randomly initializing all weight parameters in a network;
step 2.2, the reconstructed slice sample X i ∈R C×1×L Inputting the data into a network, and obtaining the data after processing by a denoising weighting moduleWherein C is r And L r Respectively representing the channel number and the length of the processed electroencephalogram sample;
in this embodiment, the implementation process of the denoising weighting module is as follows:
first, electroencephalogram data x= { X 1 ,X 2 ,...,X i ,...,X N The frequency domain brain signal sequence is obtained by the conversion of the Fast Fourier Transform (FFT) to the frequency domainWherein (1)>An ith section of electroencephalogram signal sample representing a frequency domain, multiplying the frequency domain electroencephalogram data by a manually preset element valueLearned matrix filter W 1 Obtaining the frequency domain denoising sequence->Wherein (1)>The matrix filter can effectively reduce high-frequency noise artifacts in the brain electrical signal by representing the frequency domain denoising brain electrical signal sample of the ith section, and as shown in fig. 4, the output of the denoising layer is smoother than the input, and high-frequency components are effectively removed.
Converting the denoised frequency domain data into the time domain by Inverse Fast Fourier Transform (IFFT) to obtain a time domain denoised sequenceWherein->Representing an ith section of time domain denoising brain electric signal sample; three-dimensional +.>Is removed to obtain two dimensions +.>Furthermore, there is a two-dimensional denoising sequence->Inputting the two-dimensional noise reduction sequence into a weighting layer, multiplying a manually preset channel weight diagonal matrix with a learnable diagonal element value (each diagonal element value corresponds to a channel weight value of a channel) to obtain a channel weighting sequenceWherein (1)>Representing channel weighting of the ith segmentIn the electroencephalogram signal sample, the weighting layer strengthens the role of the electroencephalogram channel containing important information in electroencephalogram classification by giving the electroencephalogram channel with greater weight, and table 3 shows the part of the channel weight values learned by the weighting layer, and the channel with the learned channel weight value of more than or equal to 0.6 is defined as an important channel, so that 6 important channels are totally available for 21 subjects in the CHB-MIT data set, and the method has more important significance for model inference.
Table 3 shows the weight values of the weighting layers of the present invention (data from CHB-MIT dataset subject 21).
| channel | LOOCV-1 | LOOCV-2 | LOOCV-3 | LOOCV-4 |
| u’FP1-F7’ | 0.233 | 0.230 | 0.225 | 0.213 |
| u’F7-T7’ | 0.212 | 0.210 | 0.206 | 0.202 |
| u’T7-P7’ | 0.202 | 0.200 | 0.198 | 0.191 |
| u’P7-O1’ | 0.600 | 0.594 | 0.594 | 0.591 |
| u’FP1-F3’ | 0.519 | 0.512 | 0.507 | 0.501 |
| u’F3-C3’ | 0.543 | 0.536 | 0.533 | 0.533 |
| u’C3-P3’ | 0.899 | 0.899 | 0.895 | 0.894 |
| u’P3-O1’ | 0.215 | 0.216 | 0.219 | 0.220 |
| u’FP2-F4’ | 0.294 | 0.290 | 0.289 | 0.286 |
| u’F4-C4’ | 0.302 | 0.300 | 0.300 | 0.297 |
| u’C4-P4’ | 0.658 | 0.657 | 0.655 | 0.653 |
| u’P4-O2’ | 0.122 | 0.136 | 0.142 | 0.146 |
| u’FP2-F8’ | 0.429 | 0.418 | 0.413 | 0.402 |
| u’F8-T8’ | 0.207 | 0.206 | 0.201 | 0.198 |
| u’T8-P8’ | 0534 | 0.533 | 0.533 | 0.531 |
| u’P8-O2’ | 0.407 | 0.420 | 0.426 | 0.432 |
| u’FZ-CZ’ | 0.788 | 0.797 | 0.806 | 0.818 |
| u’CZ-PZ’ | 0.433 | 0.433 | 0.434 | 0.435 |
| u’P7-T7’ | 0.948 | 0.944 | 0.937 | 0.929 |
| u’T7-FT9’ | 0.317 | 0.318 | 0.316 | 0.315 |
| u’FT9-FT10’ | 0.811 | 0.811 | 0.804 | 0.803 |
| u’FT10-T8’ | 0.508 | 0.509 | 0.502 | 0.492 |
Finally, inputting the electroencephalogram signal samples weighted by the channels into a dimension reduction layer, removing a large amount of redundant information in electroencephalogram data by using one-dimensional convolution, improving the robustness of a model, and obtaining a redundancy removal sequenceWherein->An electroencephalogram signal sample representing the ith section to remove redundant information is calculated as shown in formula (1):
in the formula (1), w 1 ,w 2 ,…,w n Representing the learnable channel weights, conv1D represents a one-dimensional convolution.
Step 2.3, removing redundant sequencesInput into a spatio-temporal joint MLP module to extract integrated spatio-temporal features, the implementation is as follows: firstly inputting the redundancy removing sequence into the inter-channel MLP layer, firstly standardizing the redundancy removing sequence through layer standardization (LN) operation, then expanding the channel number through converting the full-connection layer, then using a nonlinear activation function GELU to carry out nonlinear mapping on data, improving the nonlinear fitting capability of the network to the electroencephalogram signals,finally, restoring the number of the extended channels to the number of channels before extension by restoring the full-connection layer, and extracting and integrating the spatial correlation among the channels of the multi-channel electroencephalogram signals by the operation to obtain a spatial correlation sequence +.>Wherein->An electroencephalogram signal sample representing spatial correlation between the integrated channels after the i-th section is extracted;
the intra-channel MLP layer and the inter-channel MLP layer have similar internal structures, except that the inter-channel MLP layer acts on the channel dimension (spatial dimension) of the input data and the intra-channel MLP layer acts on the length dimension (temporal dimension) of the input data. Through operation similar to the inter-channel MLP layer, the intra-channel MLP layer extracts time information in each channel of the multi-channel electroencephalogram signal to obtain a time information sequenceWherein->The brain electrical signal sample representing the information of the i-th section of the extraction time is calculated as shown in a formula (2) by the space-time combined MLP module:
in the formula (2), W 2 ,W 3 ,W 4 ,W 5 Indicating full connection, LN indicates layer normalization, σ indicates gel nonlinear activation function.
Step 2.4,After being processed by the classification module, the ith section of electroencephalogram sample X is output i To obtain a final classification result; the specific process is as follows:
the time information electroencephalogram sample sequence with the space correlation and time information extracted is extractedInputting into a classification module, and obtaining a time dimension reduction sequence after the treatment of an average pooling layer>Wherein,electroencephalogram sample for representing dimension reduction of ith time dimension (length dimension) and time dimension reduction sequence through full-connection layerAfter the space dimension reduction, obtaining a category score sequence +.>Wherein,representing each class score of the ith section of electroencephalogram sample, and finally outputting the ith section of electroencephalogram sample X through a Softmax layer i Selecting one category with the highest probability as a final classification result, wherein the calculation process is shown in a formula (3):
in the formula (3), f averagepooling (. Cndot.) represents average pooling, f softmax (. Cndot.) indicates Softmax
Step 3, model training:
in the model training stage, cross entropy is adopted as a loss function, an ADAM optimizer is utilized to optimize a network, and a loss function gradient is calculated to update a network weight parameter until the maximum iteration number is reached; in the embodiment, the number of samples trained each time is set to be 16, the initial learning rate of the ADAM optimizer is set to be 0.001, and a model with the lowest verification loss in a test set is selected as an optimal model;
step 4, model calibration:
in practical situations, no other type of electroencephalogram signal appears in the continuous type of electroencephalogram signal. But this situation where an isolated other type of electroencephalogram signal appears in a continuous type of electroencephalogram signal is easily seen in an electroencephalogram signal classification task. To address this problem, a moving average filtering algorithm is used to calibrate the predicted result sequence of the model. Taking an ith section of electroencephalogram sample X i M-1 section electroencephalogram sample { X }, behind i+1 ,X i+2 ,...,X i+M-1 Mean value of probability of each class as X of ith segment sample i Is calculated as shown in formula (4).
In the formula (4), p i+k Representing the class probabilities for the i + k sample,representing the calculated class probabilities for the ith sample.
The application example of the space-time interaction MLP network is electroencephalogram type prediction based on electroencephalogram signals, experiments are respectively developed on two public data sets of CHB-MIT and Kaggle, a representative CNN model and a traditional MLP model are used as comparison models, and the performance of the proposed network is verified. In this example, four widely used evaluation indicators were used to measure model performance. Sensitivity (Sn) refers to the ratio of the number of times that the electroencephalogram is correctly predicted to the total number of times that the electroencephalogram is generated by adopting a k-threshold method (k continuous predictions of a certain class are considered to be detected); false Positive Rate (FPR) is defined as the number of mispredictions per hour; AUC is an important index for accurately balancing the prediction performance of the model, the AUC value of the random classifier is 0.5, and the AUC value of the perfect classifier can be 1; the p-value can measure whether the model is superior to the random classifier, and when the p-value is less than 0.05, the model is obviously superior to the random classifier under the significance of 0.05. Tables 1 and 2 show the behavior of the spatio-temporal interaction MLP network of the present invention and two comparative models on the CHB-MIT data set and the Kaggle data set, respectively. Compared with the traditional CNN model, the space-time combined MLP network provided by the invention realizes better prediction results with approximately equal parameter and shallower network structure, and compared with the traditional MLP network, the space-time combined MLP network provided by the invention realizes better prediction results with parameter less than 1/4, which fully shows the effectiveness of the space-time interaction MLP network provided by the invention, fully extracts spatial relationship and time information, improves classification prediction performance, and obtains very excellent performance on both CHB-MIT data set and kagle data set, thereby indicating that the space-time interaction MLP network provided by the invention has better recognition capability and stronger generalization performance on the brain electrical category of a subject.
TABLE 1 average Performance of comparative models on the CHB-MIT database for classifying electroencephalograms
Table 2 average Performance of the comparison model on the Kaggle database to classify the EEG signals
In conclusion, the method can effectively reduce high-frequency noise artifacts in the original electroencephalogram signals, strengthen the effect of important channels on making correct inferences, effectively remove redundant information in the electroencephalogram signals, and improve the recognition robustness of the model on the original electroencephalogram data; meanwhile, the rich time information and spatial association in the electroencephalogram signals can be fully utilized, the accuracy of the prediction result of the model is greatly improved, the obvious prediction result is obtained in the two-class electroencephalogram type prediction on the data sets CHB-MIT and Kagle, and meanwhile, the false alarm rate of the electroencephalogram type prediction is effectively reduced; and because the MLP network has a simple structure, the space-time combined MLP network provided by the invention is easier to train and has less time for training and testing compared with the CNN network.
Claims (3)
1. An electroencephalogram classification method based on a space-time combined MLP network is characterized by comprising the following steps:
step 1, acquiring an electroencephalogram data set with labeling category information, performing sliding slice processing on electroencephalogram signals of each category, reconstructing the input shape of the sliced electroencephalogram signals, obtaining N segments of electroencephalogram signal samples with total duration of T, and marking the N segments of electroencephalogram signal samples as a training sample set X= { X 1 ,X 2 ,...,X i ,...,X N -and the tag set of training sample X is noted y= { Y 1 ,Y 2 ,...,Y i ,...,Y N -a }; wherein X is i ∈R C×1×L Representing an ith section of brain electrical signal sample after input shape reconstruction, wherein C represents the channel number of the brain electrical signal sample, and L represents the length of the brain electrical signal sample; y is Y i For the ith section of electroencephalogram signal sample X i The corresponding label;
step 2, establishing a space-time joint MLP (multi-layer protocol) network, comprising the following steps: the system comprises a denoising weighting module, a space-time joint MLP module and a classification module;
step 2.1, the denoising weighting module includes: the device comprises a denoising layer, a weighting layer and a dimension reduction layer;
the denoising layer comprises a manually set matrix filter with randomly initialized element values, and firstly trains a sample set X= { X through fast Fourier transform 1 ,X 2 ,...,X i ,...,X N Converting the time domain into the frequency domain, multiplying the training sample set converted into the frequency domain with a matrix filter capable of learning to obtain a denoised training sample set, and converting the denoised training sample set into the time domain by inverse fast Fourier transform to obtain a time domain denoised electroencephalogram sample sequenceWherein X is i d ∈R C×1×L Representing the de-noised firsti sections of time domain brain signal samples;
sequencing brain electrical signalsConverting from three dimensions to two dimensions, thereby obtaining a two-dimensional denoising brain electricity sample sequence +.>Wherein (1)>Representing a two-dimensional ith section of electroencephalogram signal sample;
the weighting layer comprises a channel weight matrix which is manually preset and has a learnable diagonal element value; the weighting layer firstly carries out two-dimensional electroencephalogram sample sequenceMultiplying the channel weight matrix to obtain a weighted sequenceWherein (1)>Representing an ith section of electroencephalogram signal sample after channel weighting;
the dimension reduction layer comprises a group of 1 xk convolution kernels and weights the electroencephalogram sample sequence of the channelRemoving redundant information in time dimension (length dimension) to obtain redundancy-removed brain electricity sample sequence +.>Wherein,representing the first of the redundant information removedi sections of electroencephalogram samples;
step 2.2, the space-time combined MLP module comprises: an inter-channel MLP layer and an intra-channel MLP layer;
the inter-channel MLP layer sequentially comprises: layer norm layer, transform full connection layer, gel nonlinear activation function and restore full connection layer;
the layer norm layer pair redundancy-removing brain electricity sample sequenceAfter normalization processing, obtaining a space-related electroencephalogram sample sequence ∈after processing of transforming the full-connection layer, GELU activation function and restoring the full-connection layer in sequence>Wherein (1)>Representing an ith section of electroencephalogram signal sample which is subjected to extraction, integration and channel space correlation;
the structure of the intra-channel MLP layer is the same as that of the inter-channel MLP layer, and the brain electricity sample sequence is related to the spaceAfter normalization processing, obtaining a time information electroencephalogram sample sequence +.>Wherein (1)>Representing an ith section of electroencephalogram signal sample extracted through time information in a channel;
step 2.3, the classification module includes: an averaging pooling layer, a full connectivity layer and a Softmax layer;
time information electroencephalogramSample sequenceAfter the treatment of the average pooling layer and the full connection layer in sequence, obtaining the score of each section of electroencephalogram signal sample corresponding to each category, and finally converting the score of each section of electroencephalogram signal sample corresponding to each category into a probability value of each category through the Softmax layer, and selecting the maximum probability value as a prediction classification result of each section of electroencephalogram signal sample;
step 3, model training:
based on the training sample set X and the label set Y, cross entropy is adopted as a loss function, an ADAM optimizer is utilized to train the space-time combined MLP network, and gradient of the loss function is calculated to update network parameters until the maximum iteration number or the loss function convergence is reached, so that a trained electroencephalogram signal classification model is obtained;
and 4, calibrating a predicted result sequence of the model by using a moving average filtering algorithm:
taking an ith section of electroencephalogram sample X i M-1 section electroencephalogram sample { X }, behind i+1 ,X i+2 ,...,X i+M-1 The average value of each kind of probability value corresponding to each section of electroencephalogram signal sample in the } is correspondingly taken as X of the ith section of sample i Is a probability value for each category of (a).
2. An electronic device comprising a memory and a processor, wherein the memory is configured to store a program that supports the processor to perform the electroencephalogram classification method of claim 1, the processor being configured to execute the program stored in the memory.
3. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, performs the steps of the electroencephalogram classification method according to claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210849649.5A CN115017960B (en) | 2022-07-19 | 2022-07-19 | Electroencephalogram signal classification method based on space-time combined MLP network and application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210849649.5A CN115017960B (en) | 2022-07-19 | 2022-07-19 | Electroencephalogram signal classification method based on space-time combined MLP network and application |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115017960A CN115017960A (en) | 2022-09-06 |
| CN115017960B true CN115017960B (en) | 2024-02-13 |
Family
ID=83081323
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210849649.5A Active CN115017960B (en) | 2022-07-19 | 2022-07-19 | Electroencephalogram signal classification method based on space-time combined MLP network and application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115017960B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117547286B (en) * | 2023-12-29 | 2024-05-28 | 中国人民解放军东部战区总医院 | Electroencephalogram signal data analysis management system based on intelligent repair material |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114124260A (en) * | 2022-01-11 | 2022-03-01 | 中国人民解放军国防科技大学 | Spectrum prediction method, apparatus, medium, and device based on composite 2D-LSTM network |
| CN114564991A (en) * | 2022-02-28 | 2022-05-31 | 合肥工业大学 | Electroencephalogram signal classification method based on Transformer guide convolution neural network |
| CN114564990A (en) * | 2022-02-28 | 2022-05-31 | 合肥工业大学 | Electroencephalogram signal classification method based on multi-channel feedback capsule network |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210138249A1 (en) * | 2016-04-22 | 2021-05-13 | Newton Howard | System and method for neural stimulation using spike frequency modulation |
| US11786694B2 (en) * | 2019-05-24 | 2023-10-17 | NeuroLight, Inc. | Device, method, and app for facilitating sleep |
-
2022
- 2022-07-19 CN CN202210849649.5A patent/CN115017960B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114124260A (en) * | 2022-01-11 | 2022-03-01 | 中国人民解放军国防科技大学 | Spectrum prediction method, apparatus, medium, and device based on composite 2D-LSTM network |
| CN114564991A (en) * | 2022-02-28 | 2022-05-31 | 合肥工业大学 | Electroencephalogram signal classification method based on Transformer guide convolution neural network |
| CN114564990A (en) * | 2022-02-28 | 2022-05-31 | 合肥工业大学 | Electroencephalogram signal classification method based on multi-channel feedback capsule network |
Non-Patent Citations (2)
| Title |
|---|
| 单声道语音降噪与去混响研究综述;蓝天;彭川;李森;叶文政;李萌;惠国强;吕忆蓝;钱宇欣;刘峤;;计算机研究与发展(05);全文 * |
| 语音任务下声学特征提取综述;郑纯军;王春立;贾宁;;计算机科学(05);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115017960A (en) | 2022-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107844755B (en) | Electroencephalogram characteristic extraction and classification method combining DAE and CNN | |
| Ashokkumar et al. | RETRACTED: Implementation of deep neural networks for classifying electroencephalogram signal using fractional S‐transform for epileptic seizure detection | |
| CN114947883B (en) | Deep learning electroencephalogram noise reduction method based on time-frequency domain information fusion | |
| CN114564990A (en) | Electroencephalogram signal classification method based on multi-channel feedback capsule network | |
| CN114648048A (en) | Electrocardiosignal noise reduction method based on variational self-coding and PixelCNN model | |
| CN110543831A (en) | brain print identification method based on convolutional neural network | |
| CN117193537A (en) | Double-branch convolutional neural network motor imagery intention decoding method based on self-adaptive transfer learning | |
| CN115017960B (en) | Electroencephalogram signal classification method based on space-time combined MLP network and application | |
| CN115414051A (en) | Emotion classification and recognition method of electroencephalogram signal self-adaptive window | |
| CN114595725B (en) | Electroencephalogram signal classification method based on addition network and supervised contrast learning | |
| CN113128384B (en) | Brain-computer interface software key technical method of cerebral apoplexy rehabilitation system based on deep learning | |
| CN114626408B (en) | Electroencephalogram signal classification method and device, electronic equipment, medium and product | |
| CN115813410A (en) | Electroencephalogram depression identification method based on improved time sequence convolution network | |
| CN116746947A (en) | Cross-subject electroencephalogram signal classification method based on online test time domain adaptation | |
| CN116662782A (en) | MSFF-SENET-based motor imagery electroencephalogram decoding method | |
| CN117503157A (en) | Electroencephalogram signal emotion recognition method based on SGCRNN model | |
| CN116595434A (en) | Lie detection method based on dimension and classification algorithm | |
| CN115462803A (en) | BG-Attention-based electroencephalogram signal denoising method, device and storage medium | |
| CN111493864B (en) | EEG signal mixed noise processing method, equipment and storage medium | |
| CN112149724B (en) | Electroencephalogram data feature extraction method based on intra-class compactness | |
| CN117725490B (en) | Cross-test passive pitch-aware EEG automatic classification method and system | |
| CN118013352B (en) | EEG-fNIRS motor imagery identification method and device based on heterogram network | |
| Shang et al. | EEG Signal Classifiaction with Optimized LSTM: A Meta-Heuristic Approach | |
| CN117122281A (en) | Epileptic seizure prediction system and method based on space-time multi-scale attention mechanism | |
| CN118245870A (en) | Heart sound classifying algorithm based on double spectrum characteristic extraction and CNN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |