Abstract
The new perspective in visual classification aims to decode the feature representation of visual objects from human brain activities. Recording electroencephalogram (EEG) from the brain cortex has been seen as a prevalent approach to understand the cognition process of an image classification task. In this study, we proposed a deep learning framework guided by the visual evoked potentials, called the Event-Related Potential (ERP)-Long short-term memory (LSTM) framework, extracted by EEG signals for visual classification. In specific, we first extracted the ERP sequences from multiple EEG channels to response image stimuli-related information. Then, we trained an LSTM network to learn the feature representation space of visual objects for classification. In the experiment, 10 subjects were recorded by over 50,000 EEG trials from an image dataset with 6 categories, including a total of 72 exemplars. Our results showed that our proposed ERP-LSTM framework could achieve classification accuracies of cross-subject of 66.81% and 27.08% for categories (6 classes) and exemplars (72 classes), respectively. Our results outperformed that of using the existing visual classification frameworks, by improving classification accuracies in the range of 12.62%–53.99%. Our findings suggested that decoding visual evoked potentials from EEG signals is an effective strategy to learn discriminative brain representations for visual classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahmed, H., Wilbur, R.B., Bharadwaj, H.M., Siskind, J.M.: Object classification from randomized EEG trials. arXiv preprint arXiv:2004.06046 (2020)
Bashivan, P., Rish, I., Yeasin, M., Codella, N.: Learning representations from EEG with deep recurrent-convolutional neural networks. arXiv preprint arXiv:1511.06448 (2015)
Gazzaniga, M., Ivry, R., Mangun, G.: Cognitive Neuroscience: The Biology of the Mind, 3rd ed., Chap. 6 (2008)
Geirhos, R., Temme, C.R., Rauber, J., Schütt, H.H., Bethge, M., Wichmann, F.A.: Generalisation in humans and deep neural networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 7538–7550 (2018)
Gu, X., et al.: EEG-based brain-computer interfaces (BCIs): a survey of recent studies on signal sensing technologies and computational intelligence approaches and their applications. arXiv preprint arXiv:2001.11337 (2020)
Hanson, S.J., Matsuka, T., Haxby, J.V.: Combinatorial codes in ventral temporal lobe for object recognition: haxby (2001) revisited: is there a “face” area? Neuroimage 23(1), 156–166 (2004)
Haynes, J.D., Rees, G.: Decoding mental states from brain activity in humans. Nat. Rev. Neurosci. 7(7), 523–534 (2006)
Hosseini, H., Xiao, B., Jaiswal, M., Poovendran, R.: On the limitation of convolutional neural networks in recognizing negative images. In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 352–358. IEEE (2017)
Kaneshiro, B., Guimaraes, M.P., Kim, H.S., Norcia, A.M., Suppes, P.: A representational similarity analysis of the dynamics of object processing using single-trial EEG classification. PLoS One 10(8), e0135697 (2015)
Li, R., et al.: Training on the test set? An analysis of Spampinato et al. [31]. arXiv preprint arXiv:1812.07697 (2018)
Pascalis, V.D.: Chapter 16 - on the psychophysiology of extraversion. In: Stelmack, R.M. (ed.) On the Psychobiology of Personality, pp. 295–327. Elsevier, Oxford (2004)
Peelen, M.V., Downing, P.E.: The neural basis of visual body perception. Nat. Rev. Neurosci. 8(8), 636–648 (2007)
Philiastides, M.G., Ratcliff, R., Sajda, P.: Neural representation of task difficulty and decision making during perceptual categorization: a timing diagram. J. Neurosci. 26(35), 8965–8975 (2006)
Philiastides, M.G., Sajda, P.: Temporal characterization of the neural correlates of perceptual decision making in the human brain. Cereb. Cortex 16(4), 509–518 (2005)
Simanova, I., Van Gerven, M., Oostenveld, R., Hagoort, P.: Identifying object categories from event-related EEG: toward decoding of conceptual representations. PLoS One 5(12), e14465 (2010)
Spampinato, C., Palazzo, S., Kavasidis, I., Giordano, D., Souly, N., Shah, M.: Deep learning human mind for automated visual classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6809–6817 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zheng, X., Cao, Z., Bai, Q. (2020). An Evoked Potential-Guided Deep Learning Brain Representation for Visual Classification. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1333. Springer, Cham. https://doi.org/10.1007/978-3-030-63823-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-63823-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63822-1
Online ISBN: 978-3-030-63823-8
eBook Packages: Computer ScienceComputer Science (R0)