Individual-Specific Classification of Mental Workload Levels Via an Ensemble Heterogeneous Extreme Learning Machine for EEG Modeling
<p>The framework of assessing MW levels.</p> "> Figure 2
<p>The raw EEG data in (<b>a</b>) a 2-s segment of 11 channels and (<b>b</b>) the corresponding 137 EEG features.</p> "> Figure 3
<p>The architecture of a typical single-hidden-layer feedforward neural networks (SLFN) training by ELM.</p> "> Figure 4
<p>Framework of the heterogeneous extreme learning machine (ELM) ensemble for individual dependent mental workload (MW) classification.</p> "> Figure 5
<p>Training and testing data splits for mental workload classifiers: (<b>a</b>) Case 1: For each participant, two-session EEG data of 3600 instances were divided into training and testing sets of 2400 and 1200 instances, respectively; (<b>b</b>) Case 2: Two mutually exclusive training and testing sets with the sample size of 19,200 and a testing 9600 were used.</p> "> Figure 6
<p>Training and testing accuracy of the ELM classifier under Case 1 (<b>a</b>–<b>c</b>) and Case 2 (<b>d</b>) vs. the variation of the number of hidden neurons. For Case 1, the performance of ELM on the EEG feature sets from participant S1, S2 and S3 is presented. The labels of hardlim, sigmoid, and sine indicating the hard limit, sigmoid, and sine activation function were employed, respectively.</p> "> Figure 7
<p>Training and testing time of an ELM classifier: (<b>a</b>,<b>b</b>) Case 1: Training and testing time for subjects (<b>a</b>,<b>b</b>) S1, (<b>c</b>,<b>d</b>) S2, and (<b>e</b>,<b>f</b>) S3. Case 2: (<b>g</b>,<b>h</b>) Training and testing time on the datasets from all subjects.</p> "> Figure 8
<p>Testing accuracy of individual-specific MW classifiers for HE-ELM, deep ELM, and classical ELM.</p> "> Figure 9
<p>MW classification testing accuracy on 4 subjects for (<b>a</b>) ELM; (<b>b</b>) K-nearest neighbor (KNN); (<b>c</b>) artificial neural network with single hidden layer (ANN); (<b>d</b>) denoising autoencoder (DAE); (<b>e</b>) deep ELM; (<b>f</b>) local preserving projection (LPP)-KNN; (<b>g</b>) LPP-ANN and (<b>h</b>) LPP-DAE vs. different model hyper-parameters. The hyper-parameters of ELM, ANN, DAE, deep ELM, LPP-ANN and LPP-DAE are the number of hidden neurons. The hyper-parameter of KNN and LPP-KNN is the number of the nearest neighbors (denoted by <span class="html-italic">k</span>).</p> "> Figure 10
<p>Box plots for 17 classifiers performance for individual dependent MW classification. The statistics in each column data are computed from the testing classification accuracy for all eight subjects.</p> "> Figure 11
<p>3-D scatter plots of the EEG feature abstractions for low and high MW classes extracted from EEG features from two subjects, (<b>a</b>–<b>c</b>) S2, (<b>d</b>–<b>f</b>) S3. Subfigures (<b>a</b>,<b>d</b>) visualize the EEG features; Subfigures (<b>b</b>,<b>e</b>) depict the outputs of the first hidden layer in HE-ELM; Subfigures (<b>c</b>,<b>f</b>) show the outputs of the activations in the second hidden layer.</p> ">
Abstract
:1. Introduction
2. Materials and Methods
2.1. Experimental Tasks and Subjects
2.2. EEG Feature Extraction
- (1)
- All EEG signals were filtered through a three-order low-pass IIR filter with a cutoff frequency of 40 Hz. The related works [41,42,43] indicated that removing EOG artifacts can improve the EEG classification rate, the blink artifacts in EEG signals were eliminated by the coherence method in this study. According to our previous work [33], the blink artifact was removed by the following equationIn the equation, the EEG signal and the synchronized EOG signal at the time instant are denoted by and , respectively. The transfer coefficient is defined by
- (2)
- The filtered EEG was divided into 2-s segments and processed with a high-pass IIR filter (cutoff frequency of 1 Hz) to remove respiratory artifacts.
- (3)
- Fast Fourier transform was adopted to compute the power spectral density (PSD) features of the EEG signals. For each channel, four features were obtained by the calculated PSD within theta (4–8 Hz), alpha (8–13 Hz), beta (14–30 Hz), and gamma (31–40 Hz) frequency bands. Based on the PSD features from F3, F4, C3, C4, P3, P4, O1, and O2, we further computed sixteen power differences between the right and left hemispheres of the scalp. That is, 60 frequency domain features were extracted. Then, 77-time domain features were elicited via mean, variance, zero crossing rate, Shannon entropy, spectral entropy, kurtosis, and skewness of 11 channels. The indices and notations of 137 EEG features are shown in Table 1.
2.3. Extreme Learning Machine
2.4. Adaboost Based ELM Ensemble Classifier
2.5. Heterogeneous Ensemble ELM
3. Results
3.1. Model Selection for HE-ELM
3.2. Accuracy Comparison between HE-ELM and Different MW Classifiers
3.3. Computational Cost Comparison
3.4. Visualization of the Intermediate EEG Feature Representations
4. Discussion
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Giraudet, L.; Imbert, J.P.; Berenger, M.; Tremblay, S.; Causse, M. The neuroergonomic evaluation of human machine interface design in air traffic control using behavioral and EGG/ERP measures. Behav. Brain Res. 2015, 291, 246–253. [Google Scholar] [CrossRef] [PubMed]
- Sanjram, P.K. Attention and intended action in multitasking: An understanding of workload. Displays 2013, 32, 283–291. [Google Scholar] [CrossRef]
- Aricò, P.; Borghini, G.; Di, F.G.; Sciaraffa, N.; Babiloni, F. Passive BCI beyond the lab: Current trends and future directions. Physiol. Meas. 2018, 39, 1361–6579. [Google Scholar] [CrossRef] [PubMed]
- Byrne, E.A.; Parasuraman, R. Psychophysiology and adaptive automation. Biol. Psychol. 1996, 42, 249–268. [Google Scholar] [CrossRef]
- Cannon, J.; Krokhmal, P.A.; Lenth, R.V.; Murphey, R. An algorithm for online detection of temporal changes in operator cognitive state using real-time psychophysiological data. Biomed. Signal Process. Control 2010, 5, 229–236. [Google Scholar] [CrossRef]
- Cannon, J.; Krokhmal, P.A.; Chen, Y.; Murphey, R. Detection of temporal changes in psychophysiological data using statistical process control methods. Comput. Methods Programs Biomed. 2012, 107, 367–381. [Google Scholar] [CrossRef]
- Ting, C.; Mahfouf, M.; Nassef, A.; Linkens, D.; Panoutsos, G.; Niclkel, P.; Roberts, A.; Hockey, G.R.J. Real-time adaptive automation system based on identification of operator functional state in simulated process control operations. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2010, 40, 251–262. [Google Scholar] [CrossRef]
- Majumdar, K. Human scalp EEG processing: Various soft computing approaches. Appl. Soft Comput. 2011, 11, 4433–4447. [Google Scholar] [CrossRef]
- Gundel, A.; Wilson, G.F. Topographical changes in the ongoing EEG related to the diffificulty of mental tasks. Brain Topogr. 1992, 5, 17–25. [Google Scholar] [CrossRef]
- Wilson, G.F.; Fisher, F. Cognitive task classification based upon topographic EEG data. Biol. Psychol. 1995, 40, 239–250. [Google Scholar] [CrossRef]
- Aricò, P.; Borghini, G.; Di, F.G.; ColosimoA, P.S.; Babiloni, F. A passive brain-computer interface application for the mental workload assessment on professional air traffic controllers during realistic air traffic control tasks. Prog. Brain Res. 2016, 228, 295–328. [Google Scholar]
- Moon, B.S.; Lee, H.C.; Lee, Y.H.; Park, J.C. Fuzzy systems to process ECG and EEG signals for quantification of the mental workload. Inform. Sci. 2002, 142, 23–35. [Google Scholar] [CrossRef]
- Goro, O.; Satoru, T.; Naoki, S. Mental workloads can be objectively quantified in real-time using VOR (Vestibulo-Ocular Reflex). IFAC Pro. 2008, 41, 15094–15099. [Google Scholar]
- Ryu, K.; Myung, R. Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. Int. J. Ind. Ergonom. 2005, 35, 991–1009. [Google Scholar] [CrossRef]
- Dimitrakopoulos, G.N.; Kakko, I.; Dai, Z.X.; Lim, J.L.; De Souza, J.J.; Bezerianos, A.; Sun, Y. Task-Independent mental workload classification based upon common multiband EEG cortical connectivity. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 1940–1949. [Google Scholar] [CrossRef]
- Ayaz, H.; Shewokis, P.A.; Bunce, S.; Izzetoglu, K.; Willems, B.; Onaral, B. Optical brain monitoring for operator training and mental workload assessment. Neuroimage 2012, 59, 36–47. [Google Scholar] [CrossRef]
- Subasi, A.; Ercelebi, E. Classification of EEG signals using neural network and logistic regression. Comput. Meth. Pro. Biomed. 2005, 78, 87–99. [Google Scholar] [CrossRef]
- Camps-Valls, G.; Bruzzone, L. Kernel-based methods for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2005, 43, 1351–1362. [Google Scholar] [CrossRef]
- Mazaeva, N.; Ntuen, C.; Lebby, G. Self-Organizing Map (SOM) model for mental workload classification. IEEE Conf. 2001, 3, 1822–1825. [Google Scholar]
- Antelis, J.M.; Gudiño-Mendoza, B.; Falcón, L.E.; Sanchez-Ante, G.; Sossa, H. Dendrite morphological neural networks for motor task recognition from electroencephalographic signals. Biomed. Signal Proces. 2018, 44, 12–24. [Google Scholar] [CrossRef]
- Yin, Z.; Zhang, J.H. Cross-session classification of mental workload levels using EEG and an adaptive deep learning model. Biomed. Signal Proces. 2017, 33, 30–47. [Google Scholar] [CrossRef]
- Zhao, G.Z.; Liu, Y.J.; Shi, Y.C. Real-Time assessment of the cross-task mental workload using physiological measures during anomaly detection. IEEE Trans. Hum. Mach. Syst. 2018, 48, 149–160. [Google Scholar] [CrossRef]
- Huang, G.B.; Zhu, Y.Q.; Siew, C.K. Extreme learning machine: A new learning scheme of feedforward neural networks. Proc. Int. Jt. Conf. Neural Netw. 2004, 2, 985–990. [Google Scholar]
- Huang, G.B. An insight into extreme learning machines: Random neurons, random features and kernels. Cogn. Comput. 2014, 6, 376–390. [Google Scholar] [CrossRef]
- Nicolas-Alonso, L.F.; Gomez-Gil, J. Brain Computer Interfaces: A Review. Sensors 2012, 12, 1211–1279. [Google Scholar] [CrossRef]
- Huang, G.B.; Siew, C. Extreme learning machine: RBF network case. ICARCV 2004, 2, 1029–1036. [Google Scholar]
- Huang, G.B.; Zhu, Q.Y.; Siew, C.K. Extreme learning machine: Theory and applications. Neurocomputing 2006, 70, 489–501. [Google Scholar] [CrossRef]
- Zhang, R.; Lan, Y.; Huang, G.B.; Xu, Z.B. Universal approximation of extreme learning machine with adaptive growth of hidden nodes. IEEE Trans. Neur. Net. Lear. 2012, 23, 365–371. [Google Scholar] [CrossRef]
- Qi, Y.; Zhou, W.D. Epileptic EEG classification based on extreme learning machine and nonlinear features. Epilesy Res. 2011, 96, 29–38. [Google Scholar]
- Huang, G.B.; Wang, D. Advances in extreme learning machines (ELM2011). Neurocomputing 2011, 74, 2411–2412. [Google Scholar] [CrossRef]
- Huang, G.B.; Song, S.J.; You, K.Y. Trends in extreme learning machines: A review. Neural Netw. 2015, 61, 32–48. [Google Scholar] [CrossRef]
- Shi, L.C.; Lu, B.L. EEG-based vigilance estimation using extreme learning machines. Neurocomputing 2013, 102, 135–143. [Google Scholar] [CrossRef]
- Zhang, J.; Yin, Z.; Wang, R. Recognition of mental workload levels under complex human-machine collaboration by using physiological features and adaptive support vector machines. IEEE Trans. Hum. Mach. Syst. 2015, 45, 200–214. [Google Scholar] [CrossRef]
- Sauer, J.; Nickel, P.; Wastell, D. Designing automation for complex work environments under different levels of stress. Appl. Ergon. 2013, 44, 119–127. [Google Scholar] [CrossRef]
- Sauer, J.; Wastell, D.G.; Hockey, G.R.J. A conceptual framework for designing micro-worlds for complex work domains: A case study on the cabin air management system. Comput. Hum. Behav. 2000, 16, 45–58. [Google Scholar] [CrossRef]
- Yin, Z.; Zhang, J. Identification of temporal variations in mental workload using locally-linear-embedding-based EEG feature reduction and support-vector-machine-based clustering and classification techniques. Comput. Methods Programs Biomed. 2014, 115, 119–134. [Google Scholar] [CrossRef]
- Slobounov, S.M.; Fukada, K.; Simon, R.; Rearick, M.; Ray, W. Neurophysiologicaland behavioral indices oftime pressure effects on visuomotor task performance. Cogn. Brain Res. 2000, 9, 287–298. [Google Scholar] [CrossRef]
- Fairclough, S.H.; Venables, L.; Tattersall, A. The inflfluence of task demand and learning on the psychophysiological response. Int. J. Psychophysiol. 2005, 56, 171–184. [Google Scholar] [CrossRef]
- Fairclough, S.H.; Venables, L. Prediction of subjective states from psychophysiology: A multivariate approach. Biol. Psychol. 2006, 71, 100–110. [Google Scholar] [CrossRef]
- Zhao, C.; Zhao, M.; Liu, J.; Zheng, C. Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. Accid. Anal. Prev. 2012, 45, 83–90. [Google Scholar] [CrossRef]
- Yang, B.H.; Duan, K.; Fan, C.C.; Hu, C.X.; Wang, J.L. Automatic ocular artifacts removal in EEG using deep learning. Biomed. Signal Proces. 2018, 43, 148–158. [Google Scholar] [CrossRef]
- Borowicz, A. Using a multichannel wiener filter to remove eye-blink artifacts from EEG data. Biomed. Signal Process. 2018, 45, 246–255. [Google Scholar] [CrossRef]
- Manganotti, P.; Gerloff, C.; Toro, C.; Katsua, H.; Sadato, N.; Zhuang, P.; Leocani, L.; Hallet, M. Task-related coherence and task-related spectral power changes during sequential finger movements. Electroencephalogr. Clin. Neurophysiol. 1998, 109, 50–62. [Google Scholar] [CrossRef]
- Huang, G.B.; Wang, D.; Lan, Y. Extreme learning machines: A survey. Int. J. Mach. Learn. Cyb. 2011, 2, 107–122. [Google Scholar] [CrossRef]
- Huang, G.B.; Chen, L. Enhanced random search based incremental extreme learning machine. Neurocomputing 2008, 71, 3460–3468. [Google Scholar] [CrossRef]
- Cormen, T.H.; Leiserson, C.E.; Rivest, R.L.; Stein, C. Introduction to Algorithms, 2nd ed.; MIT Press: Cambridge, MA, USA, 2001. [Google Scholar]
- Freund, Y. An adaptive version of the boost by majority algorithm. Mach. Learn. 2001, 43, 293–318. [Google Scholar] [CrossRef]
- Friedman, J.H. Greedy function aproximation: A gradient boosting machine. Ann. Stat. 2001, 29, 1189–1232. [Google Scholar] [CrossRef]
- Zhang, Z.; Zhao, M.; Chow, T.W.S. Constrained large margin local projection algorithms and extensions for multimodal dimensionality reduction. Pattern Recogn. 2012, 45, 4466–4493. [Google Scholar] [CrossRef]
- Tang, J.; Deng, C.; Huang, G.B. Extreme learning machine for multilayer perceptron. IEEE Trans. Neural Netw. Learn. Syst. 2017, 27, 809–821. [Google Scholar] [CrossRef]
- Cover, T.M.; Hart, P.E. Nearest neighbor pattern classification. IEEE Trans. Inform. Theory 2006, 13, 21–27. [Google Scholar] [CrossRef]
- Li, J.H.; Struzik, Z.; Zhang, L.Q.; Cichocki, A. Feature learning from incomplete EEG with denoising autoencoder. Neurocomputing 2015, 165, 23–31. [Google Scholar] [CrossRef] [Green Version]
- Lombardo, L.; Cama, M.; Conoscenti, C.; Märker, M.; Rotigliano, E. Binary logistic regression versus stochastic gradient boosted decision trees in assessing landslide susceptibility for multiple-occurring landslide events: Application to the 2009 storm event in Messina (Sicily, southern Italy). Nat. Hazards 2015, 79, 1621–1648. [Google Scholar] [CrossRef]
- Yin, Z.; Zhang, J. Operator functional state classification using least-square support vector machine based recursive feature elimination technique. Comput. Meth. Prog. Biomed. 2014, 113, 101–115. [Google Scholar] [CrossRef]
- Babiloni, F.; Astolfi, L. Social neuroscience and hyperscanning techniques: Past, present and future. Neurosci. Biobehav. Rev. 2014, 44, 76–93. [Google Scholar] [CrossRef] [Green Version]
Feature Index | Feature Notations |
---|---|
No. 1–11 | Centroid frequencies of 11 channels |
No. 12–22 | Log energy entropies of 11 channels |
No. 23–33 | Means of 11 channels |
No. 34–77 | Average PSDs in theta (4–8 Hz), alpha (8–13 Hz), beta (14–30 Hz) and gamma (31–40 Hz) frequency bands of 11 channels |
No. 78–93 | Power differences between the right and left hemispheres of the scalp |
No. 94–04 | Shannon entropies of 11 channels |
No. 105–115 | Sums of energy of 11 channels |
No. 116–126 | Variances of 11 channels |
No. 127–137 | Zero-crossing rates of 11 channels |
ELM_Train | |
1 | Randomly assigned |
2 | for = 1 to , = 1 to |
3 | |
4 | Compute according to Equation (5) |
5 | Compute the generalized inverse of |
6 | |
7 | return |
ELM_Adaboost | |
1 | |
2 | for = 1 to |
3 | ELM_train |
4 | return |
5 | Compute predictive output as |
6 | |
7 | |
8 | Update according to Equation (9) |
9 | |
10 | return |
HE_ELM | |
1 | Compute the input weights via Equation (18) |
2 | |
3 | , |
4 | ELM_train |
5 | return |
6 | for = 1 to |
7 | |
8 | |
9 | Update according to Equation (10) |
10 | |
11 | Compute on via Equation (14) |
12 | if |
13 | Compute according to Equation (20) |
14 | else ELM_train |
15 | return a deep ELM |
16 | |
17 | return |
Subjects | Size of Hidden Neurons | Accuracy | Activation Function | ||
---|---|---|---|---|---|
Training | Testing | Training | Testing | ||
S1 | 1621 | 261 | 1.0000 | 0.8967 | hardlim |
1421 | 331 | 1.0000 | 0.9117 | sigmoid | |
2191 | 1541 | 1.0000 | 0.5517 | sine | |
S2 | 1581 | 461 | 1.0000 | 0.9333 | hardlim |
1321 | 281 | 1.0000 | 0.9525 | sigmoid | |
2081 | 211 | 1.0000 | 0.5392 | sine | |
S3 | 1881 | 701 | 1.0000 | 0.8650 | hardlim |
1791 | 351 | 1.0000 | 0.8808 | sigmoid | |
2141 | 1271 | 1.0000 | 0.5417 | sine | |
S4 | 2001 | 651 | 1.0000 | 0.7367 | hardlim |
2011 | 391 | 1.0000 | 0.7600 | sigmoid | |
2141 | 2061 | 1.0000 | 0.5492 | sine | |
S5 | 1921 | 211 | 1.0000 | 0.8200 | hardlim |
1881 | 221 | 1.0000 | 0.8325 | sigmoid | |
2171 | 1771 | 1.0000 | 0.5425 | sine | |
S6 | 1611 | 391 | 1.0000 | 0.8942 | hardlim |
1581 | 511 | 1.0000 | 0.9142 | sigmoid | |
2141 | 1851 | 1.0000 | 0.5375 | sine | |
S7 | 1671 | 471 | 1.0000 | 0.9108 | hardlim |
1671 | 231 | 1.0000 | 0.9167 | sigmoid | |
2171 | 1371 | 1.0000 | 0.5333 | sine | |
S8 | 1761 | 491 | 1.0000 | 0.8317 | hardlim |
1851 | 571 | 1.0000 | 0.8558 | sigmoid | |
2171 | 491 | 1.0000 | 0.5492 | sine |
Number of Iterations | Accuracy |
---|---|
10 | 0.6683 |
20 | 0.6694 |
30 | 0.6773 |
40 | 0.6846 |
41 | 0.6854 |
42 | 0.6857 |
43 | 0.6847 |
44 | 0.6848 |
45 | 0.6859 |
46 | 0.6865 |
47 | 0.6850 |
48 | 0.6853 |
49 | 0.6855 |
50 | 0.6852 |
60 | 0.6844 |
Number of Hidden Neurons | Accuracy | Activation Function |
---|---|---|
201 | 0.6722 | hardlim |
301 | 0.6683 | sigmoid |
1301 | 0.5120 | sine |
Subject Index | Number of Iterations | Number and Proportion of Deep ELM Classifiers | Number and Proportion of NB Classifiers |
---|---|---|---|
S1 | 6 | 4 (66.6%) | 2 (33.3%) |
S2 | 6 | 4 (66.6%) | 2 (33.3%) |
S3 | 11 | 10 (90.9%) | 1 (0.9%) |
S4 | 6 | 6 (100%) | 0 (0%) |
S5 | 6 | 4 (66.6%) | 2 (33.3%) |
S6 | 1 | 1 (100%) | 0 (0) |
S7 | 4 | 2 (50%) | 2 (50%) |
S8 | 11 | 9 (81.8%) | 2 (18.1%) |
MW Classifier | Mean | s.d. | ||||
---|---|---|---|---|---|---|
Accuracy | Precision | Recall | Accuracy | Precision | Recall | |
ELM | 0.878 | 0.8763 | 0.8797 | 0.0609 | 0.0582 | 0.0662 |
DAE | 0.8767 | 0.8758 | 0.8804 | 0.0616 | 0.0688 | 0.0618 |
ANN | 0.8182 | 0.8177 | 0.8235 | 0.1124 | 0.1126 | 0.1048 |
NB | 0.7778 | 0.8435 | 0.6785 | 0.1002 | 0.0348 | 0.2345 |
LR | 0.9124 | 0.9135 | 0.9119 | 0.0404 | 0.0445 | 0.0361 |
KNN | 0.8056 | 0.7911 | 0.8416 | 0.0758 | 0.0921 | 0.0861 |
SDAE | 0.8876 | 0.8862 | 0.8889 | 0.0561 | 0.0497 | 0.0759 |
Adaboost | 0.8803 | 0.8785 | 0.8831 | 0.0653 | 0.0655 | 0.0644 |
Deep ELM | 0.928 | 0.9314 | 0.9237 | 0.0423 | 0.0379 | 0.0513 |
LPP-DAE | 0.9062 | 0.9073 | 0.9087 | 0.055 | 0.0672 | 0.0621 |
LPP-NB | 0.8885 | 0.9158 | 0.8564 | 0.0918 | 0.0692 | 0.1505 |
LPP-LR | 0.9282 | 0.9293 | 0.9275 | 0.0391 | 0.0425 | 0.0406 |
LPP-ANN | 0.9279 | 0.9249 | 0.9314 | 0.0425 | 0.0427 | 0.0438 |
LPP-KNN | 0.7711 | 0.7795 | 0.8062 | 0.1153 | 0.1431 | 0.1124 |
LPP-AD | 0.9322 | 0.9272 | 0.9392 | 0.0389 | 0.0383 | 0.0485 |
LPP-SDAE | 0.8798 | 0.8781 | 0.8917 | 0.1214 | 0.1304 | 0.0996 |
HE-ELM | 0.9384 | 0.9348 | 0.9433 | 0.0386 | 0.0401 | 0.0372 |
MW Classifier | Accuracy | Precision | Recall | |||
---|---|---|---|---|---|---|
HE-ELM vs. ELM | t = 5.9112 | p = 0.0006 | t = 5.8759 | p = 0.0006 | t = 4.7131 | p = 0.0022 |
HE-ELM vs. DAE | t = 6.1614 | p = 0.0005 | t = 3.9249 | p = 0.0057 | t = 4.3161 | p = 0.0035 |
HE-ELM vs. ANN | t = 2.9515 | p = 0.0213 | t = 2.8119 | p = 0.0261 | t = 3.4224 | p = 0.0111 |
HE-ELM vs. NB | t = 6.8741 | p = 0.0002 | t = 8.0088 | p = <0.0001 | t = 3.6874 | p = 0.0078 |
HE-ELM vs. LR | t = 5.3195 | p = 0.0011 | t = 4.1021 | p = 0.0046 | t = 6.3117 | p = 0.0004 |
HE-ELM vs. KNN | t = 4.8887 | p = 0.0018 | t = 4.2814 | p = 0.0036 | t = 3.1671 | p = 0.0158 |
HE-ELM vs. SDAE | t = 5.6262 | p = 0.0008 | t = 4.6278 | p = 0.0024 | t = 3.0531 | p = 0.0185 |
HE-ELM vs. Adaboost | t = 4.9370 | p = 0.0017 | t = 4.8494 | p = 0.0019 | t = 4.9756 | p = 0.0016 |
HE-ELM vs. Deep ELM | t = 3.2168 | p = 0.0147 | t = 0.7537 | p = 0.4756 | t = 1.9432 | p = 0.0931 |
HE-ELM vs. LPP-DAE | t = 3.6243 | p = 0.0085 | t = 1.5724 | p = 0.1599 | t = 2.490 | p = 0.0416 |
HE-ELM vs. LPP-NB | t = 2.1636 | p = 0.0472 | t = 1.504 | p = 0.1763 | t = 1.8093 | p = 0.1133 |
HE-ELM vs. LPP-LR | t = 9.8706 | p = <0.0001 | t = 1.7792 | p = 0.1184 | t = 3.161 | p = 0.0159 |
HE-ELM vs. LPP-ANN | t = 6.8041 | p = 0.0003 | t = 3.9848 | p = 0.0053 | t = 3.2927 | p = 0.0132 |
HE-ELM vs. LPP-KNN | t = 4.9900 | p = 0.0016 | t = 3.5837 | p = 0.0089 | t = 3.8710 | p = 0.0061 |
HE-ELM vs. LPP-AD | t = 4.2052 | p = 0.004 | t = 1.6532 | p = 0.1423 | t = 0.6349 | p = 0.5455 |
HE-ELM vs. LPP-SDAE | t = 1.6693 | p = 0.1389 | t = 1.4507 | p = 0.1902 | t = 1.9213 | p = 0.0961 |
Subject index | ELM | DAE | ANN | KNN | SDAE | Adaboost | Deep ELM | LPP-DAE | LPP-ANN | LPP-KNN | LPP-SDAE | LPP-AD | HE-ELM | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 2 | 1 | 2 | ||||||||||||
S1 | 331 | 121 | 1261 | 23 | 90 | 150 | 58 | 301 | 121 | 61 | 13 | 130 | 30 | 1 | 6 (66.6%) |
S2 | 281 | 351 | 551 | 6 | 110 | 150 | 64 | 101 | 291 | 231 | 30 | 50 | 70 | 19 | 6 (66.6%) |
S3 | 351 | 111 | 1041 | 4 | 130 | 130 | 51 | 321 | 111 | 461 | 12 | 70 | 130 | 29 | 11 (90.9%) |
S4 | 391 | 101 | 451 | 76 | 150 | 130 | 62 | 161 | 101 | 371 | 35 | 70 | 130 | 65 | 6 (100%) |
S5 | 221 | 151 | 101 | 14 | 130 | 10 | 63 | 161 | 151 | 71 | 38 | 130 | 130 | 21 | 6 (66.6%) |
S6 | 511 | 261 | 421 | 10 | 130 | 10 | 64 | 131 | 261 | 361 | 27 | 30 | 110 | 17 | 1 (100%) |
S7 | 231 | 61 | 1051 | 35 | 130 | 90 | 60 | 201 | 191 | 931 | 95 | 50 | 90 | 18 | 4 (50.0%) |
S8 | 571 | 121 | 951 | 36 | 110 | 150 | 65 | 161 | 231 | 741 | 16 | 110 | 90 | 23 | 11 (81.8%) |
Average | 361 | 160 | 729 | 26 | 58 | 48 | 29 | 192 | 182 | 403 | 33 | 38 | 46 | 11 | 6 (77.8%) |
Classifier | Training | Testing | ||
---|---|---|---|---|
Mean | s.d. | Mean | s.d. | |
ELM | 0.3045 | 0.0273 | 0.5744 | 0.0280 |
NB | 0.1694 | 0.1795 | 0.0243 | 0.0155 |
KNN | 0.0162 | 0.0396 | 0.5725 | 0.0322 |
DAE | 1.7840 | 0.1250 | 0.1740 | 0.0379 |
ANN | 3.2071 | 1.1786 | 0.0356 | 0.0126 |
LR | 3.3983 | 0.1305 | 0.0168 | 0.0359 |
SDAE | 11.275 | 0.6258 | 0.1325 | 0.0231 |
Adaboost | 31.504 | 0.5583 | 0.0234 | 0.0211 |
Deep ELM | 0.2162 | 0.0447 | 0.1891 | 0.0292 |
LPP-NB | 0.0371 | 0.0117 | 0.0090 | 0.0141 |
LPP-KNN | 0.0103 | 0.0112 | 0.1279 | 0.0148 |
LPP-DAE | 1.8040 | 0.0810 | 0.1690 | 0.0240 |
LPP-ANN | 1.3790 | 0.4569 | 0.0624 | 0.0221 |
LPP-LR | 0.0440 | 0.0186 | 0.0044 | 0.0151 |
LPP-SDAE | 12.142 | 1.3520 | 0.1277 | 0.0323 |
LPP-AD | 5.6332 | 0.0651 | 0.0109 | 0.0105 |
HE-ELM | 9.030 | 0.1970 | 0.2552 | 0.0709 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Tao, J.; Yin, Z.; Liu, L.; Tian, Y.; Sun, Z.; Zhang, J. Individual-Specific Classification of Mental Workload Levels Via an Ensemble Heterogeneous Extreme Learning Machine for EEG Modeling. Symmetry 2019, 11, 944. https://doi.org/10.3390/sym11070944
Tao J, Yin Z, Liu L, Tian Y, Sun Z, Zhang J. Individual-Specific Classification of Mental Workload Levels Via an Ensemble Heterogeneous Extreme Learning Machine for EEG Modeling. Symmetry. 2019; 11(7):944. https://doi.org/10.3390/sym11070944
Chicago/Turabian StyleTao, Jiadong, Zhong Yin, Lei Liu, Ying Tian, Zhanquan Sun, and Jianhua Zhang. 2019. "Individual-Specific Classification of Mental Workload Levels Via an Ensemble Heterogeneous Extreme Learning Machine for EEG Modeling" Symmetry 11, no. 7: 944. https://doi.org/10.3390/sym11070944