Abstract
Disease progression models are crucial to understanding degenerative diseases. Mixed-effects models have been consistently used to model clinical assessments or biomarkers extracted from medical images, allowing missing data imputation and prediction at any timepoint. However, such progression models have seldom been used for entire medical images. In this work, a Variational Auto Encoder is coupled with a temporal linear mixed-effect model to learn a latent representation of the data such that individual trajectories follow straight lines over time and are characterised by a few interpretable parameters. A Monte Carlo estimator is devised to iteratively optimize the networks and the statistical model. We apply this method on a synthetic data set to illustrate the disentanglement between time dependant changes and inter-subjects variability, as well as the predictive capabilities of the method. We then apply it to 3D MRI and FDG-PET data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) to recover well documented patterns of structural and metabolic alterations of the brain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Allassonnière, S., Kuhn, E., Trouvé, A.: Construction of Bayesian deformable models via a stochastic approximation algorithm: a convergence study. Bernoulli 16(3), 641–678 (2010)
Ashman, M., So, J., Tebbutt, W., Fortuin, V., Pearce, M., Turner, R.E.: Sparse gaussian process variational autoencoders. arXiv preprint arXiv:2010.10177 (2020)
Banerjee, M., Chakraborty, R., Ofori, E., Okun, M.S., Viallancourt, D.E., Vemuri, B.C.: A nonlinear regression technique for manifold valued data with applications to medical image analysis. In: Proceedings of the IEEE Conference On Computer Vision and Pattern Recognition, pp. 4424–4432 (2016)
Bauer, S., May, C., Dionysiou, D., Stamatakos, G., Buchler, P., Reyes, M.: Multiscale modeling for image analysis of brain tumor studies. IEEE Trans. Biomed. Eng. 59(1), 25–29 (2011)
Bône, A., Colliot, O., Durrleman, S.: Learning distributions of shape trajectories from longitudinal datasets: a hierarchical model on a manifold of diffeomorphisms. In: CVPR 2018 - Computer Vision and Pattern Recognition 2018, Salt Lake City, United States, June 2018. https://hal.archives-ouvertes.fr/hal-01744538
Bône, A., et al.: Prediction of the progression of subcortical brain structures in Alzheimer’s disease from baseline. In: Cardoso, M.J., et al. (eds.) GRAIL/MFCA/MICGen -2017. LNCS, vol. 10551, pp. 101–113. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67675-3_10
Casale, F.P., Dalca, A.V., Saglietti, L., Listgarten, J., Fusi, N.: Gaussian process prior variational autoencoders. arXiv preprint arXiv:1810.11738 (2018)
Coffey, C.E., et al.: Sex differences in brain aging: a quantitative magnetic resonance imaging study. Arch. Neurol. 55(2), 169–179 (1998)
Couronné, R., Vernhet, P., Durrleman, S.: Longitudinal self-supervision to disentangle inter-patient variability from disease progression. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12902, pp. 231–241. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87196-3_22
Couronné, R., Vernhet, P.: Starmen longitudinal (2021). https://doi.org/10.5281/zenodo.5081988
Cui, R., Liu, M., et al.: RNN-based longitudinal analysis for diagnosis of Alzheimer’s disease. Comput. Med. Imaging Graph. 73, 1–10 (2019)
Fonteijn, H.M., et al.: An event-based model for disease progression and its application in familial Alzheimer’s disease and Huntington’s disease. NeuroImage 60(3), 1880–1889 (2012)
Fortuin, V., Baranchuk, D., Rätsch, G., Mandt, S.: GP-VAE: deep probabilistic time series imputation. In: International Conference On Artificial Intelligence and Statistics, pp. 1651–1661. PMLR (2020)
Gruffaz, S., Poulet, P.E., Maheux, E., Jedynak, B., Durrleman, S.: Learning Riemannian metric for disease progression modeling. Adv. Neural Inf. Process. Syst. 34, 23780–23792 (2021)
Gurvich, C., Hoy, K., Thomas, N., Kulkarni, J.: Sex differences and the influence of sex hormones on cognition through adulthood and the aging process. Brain Sci. 8(9), 163 (2018)
Higgins, I., et al.: BETA-VAE: Learning basic visual concepts with a constrained variational framework (2016)
Jack, C.R., et al.: Age, sex, and apoe \(\varepsilon \)4 effects on memory, brain structure, and \(\beta \)-amyloid across the adult life span. JAMA Neurol. 72(5), 511–519 (2015)
Jedynak, B.M., et al.: A computational neurodegenerative disease progression score: method and results with the Alzheimer’s disease neuroimaging initiative cohort. Neuroimage 63(3), 1478–1486 (2012)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Koval, I., et al.: Statistical learning of spatiotemporal patterns from longitudinal manifold-valued networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 451–459. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_52
Koval, I., et al.: AD Course Map charts Alzheimer’s disease progression. Sc. Rep. 11(1), -1-6 (2021). https://doi.org/10.1038/s41598-021-87434-1, https://hal.inria.fr/hal-01964821
Kuhn, E., Lavielle, M.: Coupling a stochastic approximation version of EM with an MCMC procedure. ESAIM: Probabil. Statist 8, 115–131 (2004)
Liu, M., Cheng, D., Yan, W., Initiative, A.D.N., et al.: Classification of Alzheimer’s disease by combination of convolutional and recurrent neural networks using FDG-pet images. Front. Neuroinform. 12, 35 (2018)
Louis, M., Couronné, R., Koval, I., Charlier, B., Durrleman, S.: Riemannian geometry learning for disease progression modelling. In: Chung, A., Gee, J., Yushkevich, P., Bao, S. (eds.) Information Processing in Medical Imaging. IPMI 2019. LNCS, vol. 11492, pp. 542–553. Springer, Cham (2019)
Niethammer, M., Huang, Y., Vialard, F.-X.: Geodesic regression for image time-series. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6892, pp. 655–662. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23629-7_80
Ramchandran, S., Tikhonov, G., Kujanpää, K., Koskinen, M., Lähdesmäki, H.: Longitudinal variational autoencoder. In: International Conference on Artificial Intelligence and Statistics, pp. 3898–3906. PMLR (2021)
Routier, A., et al.: Clinica: an open-source software platform for reproducible clinical neuroscience studies. Front. Neuroinform. 15, 689675 (2021)
Sauty, B., Durrleman, S.: Riemannian metric learning for progression modeling of longitudinal datasets. In: ISBI 2022-International Symposium on Biomedical Imaging (2022)
Schiratti, J.B., Allassonniere, S., Colliot, O., Durrleman, S.: Learning spatiotemporal trajectories from manifold-valued longitudinal data. In: Neural Information Processing Systems, vol. 28, Advances in Neural Information Processing Systems, Montréal, Canada, December 2015. https://hal.archives-ouvertes.fr/hal-01163373
Zhao, Q., Liu, Z., Adeli, E., Pohl, K.M.: Longitudinal self-supervised learning. Med. Image Anal. 71, 102051 (2021)
Acknowledgments
This work was funded in part by grant number 678304 (ERC) and 826421 (TVB-Cloud) from H2020 programme, and ANR-10-IAIHU-06 (IHU ICM), ANR-19-P3IA-0001 (PRAIRIE) and ANR-19-JPW2-000 (E-DADS) from ANR.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sauty, B., Durrleman, S. (2022). Progression Models for Imaging Data with Longitudinal Variational Auto Encoders. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13431. Springer, Cham. https://doi.org/10.1007/978-3-031-16431-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-16431-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16430-9
Online ISBN: 978-3-031-16431-6
eBook Packages: Computer ScienceComputer Science (R0)