Abstract
We present a deep neural network method that enables learning of a personal representation from samples acquired while subjects are performing a self neuro-feedback task, guided by functional MRI (fMRI). The neurofeedback task (watch vs. regulate) provides the subjects with continuous feedback, contingent on the down-regulation of their Amygdala signal. The representation is learned by a self-supervised recurrent neural network that predicts the Amygdala activity in the next fMRI frame given recent fMRI frames and is conditioned on the learned individual representation. We show that our personal representation, learned solely using fMRI images, improves the next-frame prediction considerably and, more importantly, yields superior performance in linear prediction of psychiatric traits, compared to performing such predictions based on clinical data and personality tests. Our code is attached as supplementary and the data would be shared subject to ethical approvals.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bagby, R.M., Parker, J.D., Taylor, G.J.: The twenty-item Toronto Alexithymia Scale-I. Item selection and cross-validation of the factor structure. J. Psychosom. Res. 38(1), 23–32 (1994)
Bouckaert, R.R., Frank, E.: Evaluating the replicability of significance tests for comparing learning algorithms. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 3–12. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24775-3_3
Calhoun, V.D., Miller, R., Pearlson, G., Adalı, T.: The chronnectome: time-varying connectivity networks as the next frontier in fMRI data discovery. Neuron 84(2), 262–274 (2014)
Dvornek, N.C., Ventola, P., Pelphrey, K.A., Duncan, J.S.: Identifying autism from resting-state fMRI using long short-term memory networks. In: Wang, Q., Shi, Y., Suk, H.-I., Suzuki, K. (eds.) MLMI 2017. LNCS, vol. 10541, pp. 362–370. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67389-9_42
Gondara, L.: Medical image denoising using convolutional denoising autoencoders. In: 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), pp. 241–246. IEEE (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Lubianiker, N., et al.: Process-based framework for precise neuromodulation. Nat. Hum. Behav. 3(5), 436–445 (2019)
Marxen, M., et al.: Amygdala regulation following fMRI-neurofeedback without instructed strategies. Front. Hum. Neurosci. 10, 183 (2016)
Oksuz, I., et al.: Magnetic resonance fingerprinting using recurrent neural networks. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1537–1540. IEEE (2019)
Paret, C., et al.: Current progress in real-time functional magnetic resonance-based neurofeedback: methodological challenges and achievements. NeuroImage 202, 116107 (2019)
Spielberger, C.D., Gorsuch, R.L.: State-Trait Anxiety Inventory for Adults: Manual and Sample: Manual, Instrument and Scoring Guide. Consulting Psychologists Press, Palo Alto (1983)
Sulzer, J., et al.: Real-time fMRI neurofeedback: progress and challenges. Neuroimage 76, 386–399 (2013)
Weathers, F., Blake, D., Schnurr, P., Kaloupek, D., Marx, B., Keane, T.: The clinician-administered PTSD scale for DSM-5 (CAPS-5). Interview Available from the National Center for PTSD (2013)
Whitfield-Gabrieli, S., Nieto-Castanon, A.: Conn: a functional connectivity toolbox for correlated and anticorrelated brain networks. Brain Connect. 2(3), 125–141 (2012)
Yan, W., et al.: Discriminating schizophrenia using recurrent neural network applied on time courses of multi-site fMRI data. EBioMedicine 47, 543–552 (2019)
Acknowledgments
This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant ERC CoG 725974), the BRAINTRAIN consortium, 7th Framework Programme, under Grant Agreement no. 602186, and US Department of Defense grant agreement no. W81XWH-11–2–0008.
We thank Shira Reznik-Balter for insightful discussions on the analytic approach and helpful comments on the manuscript.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Osin, J. et al. (2020). Learning Personal Representations from fMRI by Predicting Neurofeedback Performance. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12267. Springer, Cham. https://doi.org/10.1007/978-3-030-59728-3_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-59728-3_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59727-6
Online ISBN: 978-3-030-59728-3
eBook Packages: Computer ScienceComputer Science (R0)