Abstract
Deep Learning methods have been rising in popularity in the past few years, and are now used as a fundamental component in various application domains such as computer vision, natural language processing, bioinformatics. Supervised learning with Convolutional Neural Networks has become the state of the art approach in many image related works. However, despite the great success of deep learning methods in other areas they remain relatively unexplored in the brain imaging field. In this paper we make an overview of recent achievements of Deep Learning to automatically extract features from brain signals that enable building Brain-Machine Interfaces (BMI). Major challenge in the BMI research is to find common subject-independent neural signatures due to the high brain data variability across multiple subjects. To address this problem we propose a Deep Neural Autoencoder with sparsity constraint as a promising approach to extract hidden features from Electroencephalogram data (in-dept feature learning) and build a subject-independent noninvasive BMI in the affective neuro computing framework. Future direction for research are also outlined.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. Cambridge, MA, USA: MIT (2016)
Le Cun, Y., Boser, B., Denker, J.S., Howard, R.E., Habbard, W., Jackel, L.D., Henderson, D.: Handwritten digit recognition with a back-propagation network. In: Proc. Advances in Neural Information Processing Systems, pp. 396–404 (1990)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. IEEE Proceedings 86, 2278–2324 (1998)
Hinton, G., Li Deng, D. Y. u., Dahl, G., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., Kingsbury, B.: Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process. Mag. 29, 82–97 (2012)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems (NIPS), pp. 1097–1105 (2012)
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei, L.: Large-scale video classification with convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
Graves, A., Mohamed, A.-R., Hinton, G.E.: Speech recognition with deep recurrent neural networks.. s. Acoustics. In: IEEE International Conference on Speech and Signal Processing (ICASSP), Vancouver, Canadas (2013)
Zhang, X., Zhao, J., LeCun, Y.: Character-level Convolutional Networks for Text Classification. Advances in Neural Information Processing Systems 28 (NIPS) (2015)
Hermann, K.M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., Read, P. Blunsom.: Comprehend teaching machines to advances in neural information processing systems (NIPS) Vol. 28 (2015)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)
Ng, J.Y., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., Toderici, G.: Beyond short snippets: deep networks for video classification computer vision and pattern recognition (2015)
McFarland, D., Wolpaw, J.: Brain-computer interface operation of robotic and prosthetic devices. computer, vol 41 (10) (2008)
Mak, J.N., Wolpaw, J.R.: Clinical applications of brain-computer interfaces - Current State and Future Prospects. IEEE Rev. Biomed. Eng. 2, 187–199 (2009)
Mirowski, P.W., Madhavan, D., LeCun, Y.: Time-delay neural networks and independent component analysis for EEG-based prediction of epileptic seizures propagation. AAAI (2007)
Mirowski, P., LeCun, Y., Madhavan, D., Kuzniecky, R.: Comparing SVM and convolutional networks for epileptic seizure prediction from intracranial, EEG IEEE Workshop on Machine Learning for Signal Processing, 2008. MLSP 2008. pp. 244–249 (2008)
Mirowski, P., Madhavan, D., LeCun, Y., Kuzniecky, R.: Classification of patterns of EEG synchronization for seizure prediction. Clin. Neurophysiol. 120(11), 1927–1940 (2009)
An, X., Kuang, D., Guo, X., Zhao, Y., He, L.: A deep learning method for classication of EEG data based on motor imagery, Intelligent Computing in Bioinformatics (D.-S. huang, K. Han, and M. Gromiha, eds.), vol. 8590 of Lec- ture Notes in Computer Science, pp. 203–210 (2014)
Walker, I.: Deep Convolutional Neural Networks for Brain Computer Interface using Motor Imagery, MSc thesis, Imperial College London, Department of Computing (2015)
Stober, S., Sternin, A., Owen, A.M., Grahn, J.A.: Deep feature learning for EEG recordings. In: International Conference on Learning Representations (ICLR), San Juan, Puerto Rico (2016)
Bashivan, P., Rish, I., Yeasin, M., Codella, N.: Learning representations from EEG with deep Recurrent-Convolutional neural networks. In: International Conference on Learning Representations (ICLR), San juan, Puerto Rico (2016)
Plis, S.M., Hjelm, D.R., Salakhutdinov, R., Allen, E.A., Bockholt, H.J., Long, J.D., Johnson, H.J., Paulsen, J.S., Turner, J.A., Calhoun, V.D.: Deep learning for neuroimaging: a validation study. Frontiers in Neuroscience (2014)
Bozhkov, L. , Georgieva, P., Santos, I., Pereira, A., Silva, A.: EEG-based subject independent affective computing models. Procedia Computer Science, Elsevier, pp. 375-382, https://doi.org/10.1016/j.procs.2015.07.314 (2015)
Jatupaiboon, N., Panngum, S., Israsena, P.: Real-Time EEG-Based Happiness Detection System, The Scientific World Journal, Vol. 2013, Hindawi, Article ID 618649
Georgieva, O., Milanov, S., Georgieva, P., Santos, I.M., Pereira, A.T., da Silva, C.F.: Learning to decode human emotions from ERPs, Neural Computing and Applications, Springer, vol. 26 (3) . https://doi.org/10.1007/s00521-014-1653-6 ISSN 0941-0643 (2015)
Zhuang, N., Zeng, Y., Tong, L., Zhang, C., Zhang, H., Yan: Emotion Recognition from EEG Signals Using Multidimensional Information in EMD Domain, BioMed Research Int, Vol Hindawi, Article ID 8317357. https://doi.org/10.1155/2017/8317357 (2017)
Li, K., Li, X., Zhang, Y., Zhang, A.: Affective state recognition from EEG with deep belief networks. In: IEEE International Conference on Bioinformatics and Biomedicin, pp. 18–21 (2013)
Koelstra, S., Mühl, C., Soleymani, M., et al.: DEAP: A database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)
Jirayucharoensak, S., Pan-Ngum, S., Israsena, P.: EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation, The Scientific World Journal, Vol Hindawi, Article ID 627892. https://doi.org/10.1155/2014/627892 (2014)
Zheng, W.L., Zhu, J.Y., Peng, Y., Lu, B.L.: EEG-based emotion classification using deep belief networks. In: IEEE International Conference on Multimedia and Expo, Chengdu, China, 14–18 (2014)
Koprinkova-Hristova, P., Bozhkov, L., Georgieva, P.: Echo State Networks for feature selection in affective computing. In: Proceedings of the 13th Int. Conf. on Practical Applications of Agents and Multi-Agent Systems (PAAMS), Spain (2015)
Bozhkov, L., Koprinkova-Hristova, P., Georgieva, P.: Learning to decode human emotions with Echo State Networks. Neural Netw. 78, 112–119 (2016)
Bozhkov, L., Koprinkova-Hristova, P., Georgieva, P.: Reservoir computing for emotion valence discrimination from EEG signals. Neurocomputing 231, 28–40 (2017)
Lia, J., Struzik, Z., Zhang, L., Cichocki, A.: Feature learning from incomplete EEG with denoising autoencoder. Neurocomputing, Vol. 165 (2015)
Mauss, I.B., Robinson, M.D.: Measures of emotion: a review. Cognit. Emot. 23, 209–237 (2209)
Coan, J.A., Allen, J.J., Harmon-Jones, E.: Voluntary facial expression and hemispheric asymmetry over the frontal cortex. Psychophysiology 38, 912–925 (2001)
Lee, Y.-Y., Hsieh, S.: Classifying different emotional states by means of EEG-Based functional connectivity patterns. PLoS ONE 9(4). https://doi.org/10.1371/journal.pone.0095415 (2014)
Sammler, D., Grigutsch, M., Fritz, T., Koelsch, S.: Music and emotion: Electrophysiological correlates of the processing of pleasant and unpleasant music. Psychophysiology 44, 293–304 (2007)
Hanson, L., Dack, C., Airhart, S., Kestel, K.: The effects of emotion on alpha wave amplitude. BSHS anatomy and physiology (2014)
Olofsson, J.K., Nordin, S., Sequeira, H., Polich, J.: Affective picture processing: an integrative review of ERP findings. Biol. Psychol. 77(3), 247–265 (2008). https://doi.org/10.1016/j.biopsycho.2007.11.006
Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks?. Proceedings of Annual Conference on Neural Information Processing Systems (NIPS’2014) (2014)
Acknowledgements
This Research work was funded by National Funds through the FCT - Foundation for Science and Technology, in the context of the project UID /CEC/00127/2013.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Bozhkov, L., Georgieva, P. Deep learning models for brain machine interfaces. Ann Math Artif Intell 88, 1175–1190 (2020). https://doi.org/10.1007/s10472-019-09668-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10472-019-09668-0