Abstract
In the current study, a novel brain-machine interaction was proposed, which incorporates action observation decoding into the traditional control circuit of a brain-machine interface. In this new brain-machine interaction, the machine can actively decode the user’s action observation and stop immediately if it detects that the user does not understand the intention of the action correctly. We measured brain activation using electroencephalography (EEG)-functional near-infrared spectroscopy (fNIRS) bimodal measurement while 16 healthy participants observed three action tasks: drinking, moving a cup, and action with unclear intention. Complex brain networks were constructed for EEG and fNIRS data separately, and four network measures were chosen as features for classification. The obtained results revealed that the classification of three action observation tasks achieved accuracy of 72.3% using EEG-fNIRS confusion features, which was higher than that using fNIRS features (52.7%) or EEG features (68.6%) alone. Thus, the current findings suggested that our proposed method could provide a promising direction for brain-machine interface systems design.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lebedev, M.A., Nicolelis, M.A.L.: Brain–machine interfaces: past, present and future. Trends Neurosci. 29, 536–546 (2006)
Klobassa, D.S., et al.: Toward a high-throughput auditory P300-based brain–computer interface. Clin. Neurophysiol. 120, 1252–1261 (2009)
Chen, X., Wang, Y., Nakanishi, M., Gao, X., Jung, T.-P., Gao, S.: High-speed spelling with a noninvasive brain–computer interface. Proc. Natl. Acad. Sci. 112, E6058 (2015)
Patil, P.G., Turner, D.A.: The development of brain-machine interface neuroprosthetic devices. Neurotherapeutics 5, 137–146 (2008)
Cui, X., Bray, S., Bryant, D.M., Glover, G.H., Reiss, A.L.: A quantitative comparison of NIRS and fMRI across multiple cognitive tasks. NeuroImage 54, 2808–2821 (2011)
Khan, M.J., Hong, M.J., Hong, K.-S.: Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface. Front. Hum. Neurosci. 8, 244 (2014)
Ge, S., et al.: A Brain-Computer Interface Based on a Few-Channel EEG-fNIRS Bimodal System. IEEE Access 5, 208–218 (2017)
Rubinov, M., Sporns, O.: Complex network measures of brain connectivity: uses and interpretations. NeuroImage 52, 1059–1069 (2010)
Bullmore, E., Sporns, O.: Complex brain networks: graph theoretical analysis of structural and functional systems. Nat. Rev. Neurosci. 10, 186 (2009)
Ge, S., et al.: Temporal-spatial features of intention understanding based on EEG-fNIRS bimodal measurement. IEEE Access 5, 14245–14258 (2017)
Gardner, T., Goulden, N., Cross, E.S.: Dynamic modulation of the action observation network by movement familiarity. J. Neurosci. 35, 1561 (2015)
Watts, D.J., Strogatz, S.H.: Collective dynamics of ‘small-world’ networks. Nature 393, 440–442 (1998)
Wang, J., et al.: Parcellation-dependent small-world brain functional networks: a resting-state fMRI study. Hum. Brain Mapp. 30, 1511–1523 (2009)
Freeman, L.C.: Centrality in social networks conceptual clarification. Soc. Netw. 1, 215–239 (1978)
Kononenko, I.: Estimating attributes: analysis and extensions of RELIEF. In: Bergadano, F., De Raedt, L. (eds.) ECML 1994. LNCS, vol. 784, pp. 171–182. Springer, Heidelberg (1994). https://doi.org/10.1007/3-540-57868-4_57
Robnik-Šikonja, M., Kononenko, I.: Theoretical and empirical analysis of ReliefF and RReliefF. Mach. Learn. 53, 23–69 (2003)
Chang, C.-C., Lin, C.-J.: LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 1–27 (2011)
Schippers, M.B., Keysers, C.: Mapping the flow of information within the putative mirror neuron system during gesture observation. NeuroImage 57, 37–44 (2011)
Buxbaum, L.J., Kyle, K.M., Tang, K., Detre, J.A.: Neural substrates of knowledge of hand postures for object grasping and functional object use: evidence from fMRI. Brain Res. 1117, 175–185 (2006)
Song, T., Zheng, W., Song, P., Cui, Z.: EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans. Affect. Comput. 1 (2018)
Acknowledgments
This work was supported in part by the National Basic Research Program of China under Grant 2015CB351704, the National Nature Science Foundation of China (61473221, 61773408), and in part by the Fundamental Research Funds for the Central Universities of China (CZY18047).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, Yc., Wang, P., Liu, H., Ge, S. (2019). Decoding Action Observation Using Complex Brain Networks from Simultaneously Recorded EEG-fNIRS Signals. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Communications in Computer and Information Science, vol 1142. Springer, Cham. https://doi.org/10.1007/978-3-030-36808-1_61
Download citation
DOI: https://doi.org/10.1007/978-3-030-36808-1_61
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36807-4
Online ISBN: 978-3-030-36808-1
eBook Packages: Computer ScienceComputer Science (R0)