Abstract
The difficulty of acquiring resting-state fMRI of early developing children under the same condition leads to a dedicated protocol, i.e., scanning younger infants during sleep and older children during being awake, respectively. However, the obviously different brain activities of sleep and awake states arouse a new challenge of awake-to-sleep connectome prediction/translation, which remains unexplored despite its importance in the longitudinally-consistent delineation of brain functional development. Due to the data scarcity and huge differences between natural images and geometric data (e.g., brain connectome), existing methods tailored for image translation generally fail in predicting functional connectome from awake to sleep. To fill this critical gap, we unprecedentedly propose a novel reference-relation guided autoencoder with deep CCA restriction (R2AE-dCCA) for awake-to-sleep connectome prediction. Specifically, 1) A reference-autoencoder (RAE) is proposed to realize a guided generation from the source domain to the target domain. The limited paired data are thus greatly augmented by including the combinations of all the age-restricted neighboring subjects as the references, while the target-specific pattern is fully learned; 2) A relation network is then designed and embedded into RAE, which utilizes the similarity in the source domain to determine the belief-strength of the reference during prediction; 3) To ensure that the learned relation in the source domain can effectively guide the generation in the target domain, a deep CCA restriction is further employed to maintain the neighboring relation during translation; 4) New validation metrics dedicated for connectome prediction are also proposed. Experimental results showed that our proposed R2AE-dCCA produces better prediction accuracy and well maintains the modular structure of brain functional connectome in comparison with state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lyall, A.E., et al.: Dynamic development of regional cortical thickness and surface area in early childhood. Cereb. Cortex 25(8), 2204–2212 (2015)
Gilmore, J.H., et al.: Longitudinal development of cortical and subcortical gray matter from birth to 2 years’. Cereb. Cortex 22(11), 2478–2485 (2012)
Li, G., et al.: Cortical thickness and surface area in neonates at high risk for schizophrenia. Brain Struct. Funct. 221(1), 447–461 (2016). https://doi.org/10.1007/s00429-014-0917-3
Zhang, H., Shen, D., Lin, W.: Resting-state functional MRI studies on infant brains: a decade of gap-filling efforts. Neuroimage 185, 664–684 (2019)
Cao, M., Huang, H., He, Y.: Developmental connectomics from infancy through early childhood. Trends Neurosci. 40(8), 494–506 (2017)
Howell, B.R., et al.: The UNC/UMN Baby Connectome Project (BCP): an overview of the study design and protocol development. Neuroimage 185, 891–905 (2019)
Alotaibi, A.: Deep generative adversarial networks for image-to-image translation: a review. Symmetry 12(10), 1705 (2020)
Armanious, K., et al.: MedGAN: medical image translation using GANs. Comput. Med. Imaging Graph. 79, 101684 (2020)
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., Choo, J.: StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789–8797 (2018)
Zhu, M., Rekik, I.: Multi-view brain network prediction from a source view using sample selection via CCA-based multi-kernel connectomic manifold learning. In: Rekik, I., Unal, G., Adeli, E., Park, S.H. (eds.) PRIME 2018. LNCS, vol. 11121, pp. 94–102. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00320-3_12
Bessadok, A., Mahjoub, M.A., Rekik, I.: Brain graph synthesis by dual adversarial domain alignment and target graph prediction from a source graph. Med. Image Anal. 68, 101902 (2021)
Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., Frey, B.: Adversarial autoencoders. In: International Conference on Learning (2015)
Van den Heuvel, M.P., de Lange, S.C., Zalesky, A., Seguin, C., Yeo, B.T., Schmidt, R.: Proportional thresholding in resting-state fMRI functional connectivity networks and consequences for patient-control connectome studies: Issues and recommendations. Neuroimage 152, 437–449 (2017)
Garrison, K.A., Scheinost, D., Finn, E.S., Shen, X., Constable, R.T.: The (in)stability of functional brain network measures across thresholds. Neuroimage 118, 651–661 (2015)
Wen, X., Wang, R., Yin, W., Lin, W., Zhang, H., Shen, D.: Development of dynamic functional architecture during early infancy. Cereb. Cortex 30(11), 5626–5638 (2020)
Meunier, D., Achard, S., Morcom, A., Bullmore, E.: Age-related changes in modular organization of human brain functional networks. Neuroimage 44(3), 715–723 (2009)
Meilă, M.: Comparing clusterings-an information based distance. J. Multivar. Anal. 98(5), 873–895 (2007)
Venkataraman, A., Van Dijk, K.R., Buckner, R.L., Golland, P.: Exploring functional connectivity in fMRI via clustering. In: 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 441–444 (2009)
Tzourio-Mazoyer, N., et al.: Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 15(1), 273–289 (2002)
Tavor, I., Jones, O.P., Mars, R.B., Smith, S., Behrens, T., Jbabdi, S.: Task-free MRI predicts individual differences in brain activity during task performance. Science 352(6282), 216–220 (2016)
Wang, B., Zhu, J., Pierson, E., Ramazzotti, D., Batzoglou, S.: Visualization and analysis of single-cell RNA-seq data by kernel-based similarity learning. Nat. Methods 14(4), 414–416 (2017)
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Li, G., Wang, L., Yap, P.-T., et al.: Computational neuroanatomy of baby brains: a review. Neuroimage 185, 906–925 (2019)
Wang, L., et al.: Volume-based analysis of 6-month-old infant brain MRI for autism biomarker identification and early diagnosis. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 411–419. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_47
Li, G., Wang, L., Shi, F., Gilmore, J.H., Lin, W., Shen, D.: Construction of 4D high-definition cortical surface atlases of infants: methods and applications. Med. Image Anal. 25(1), 22–36 (2015)
Li, G., Wang, L., Shi, F., Lin, W., Shen, D.: Simultaneous and consistent labeling of longitudinal dynamic developing cortical surfaces in infants. Med. Image Anal. 18(8), 1274–1289 (2014)
Yin, W., et al.: The emergence of a functionally flexible brain during early infancy. Proc. Natl. Acad. Sci. 117(38), 23904–23913 (2020)
Hu, D., Zhang, H., Wu, Z., Wang, et al.: Disentangled-multimodal adversarial autoencoder: application to infant age prediction with incomplete multimodal neuroimages. IEEE Trans. Med. Imaging 39(12), 4137–4149 (2020)
Hu, D., et al.: Disentangled intensive triplet autoencoder for infant functional connectome fingerprinting. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12267, pp. 72–82. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_8
Acknowledgments
This work was partially supported by NIH grants (MH127544, MH116225, MH117943, MH109773, and MH123202). This work also utilizes approaches developed by an NIH grant(1U01MH110274) and the efforts of the UNC/UMN Baby Connectome Project Consortium.
Author information
Authors and Affiliations
Consortia
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Hu, D. et al. (2021). Reference-Relation Guided Autoencoder with Deep CCA Restriction for Awake-to-Sleep Brain Functional Connectome Prediction. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12903. Springer, Cham. https://doi.org/10.1007/978-3-030-87199-4_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-87199-4_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87198-7
Online ISBN: 978-3-030-87199-4
eBook Packages: Computer ScienceComputer Science (R0)