Abstract
The growing availability of novel interpretation techniques opened the way to the application of deep learning models in the clinical field, including neuroimaging, where their use is still largely underexploited. In this framework, we focus the stratification of Multiple Sclerosis (MS) patients in the Primary Progressive versus the Relapsing-Remitting state of the disease using a 3D Convolutional Neural Network trained on structural MRI data. Within this task, the application of Layer-wise Relevance Propagation visualization allowed detecting the voxels of the input data mostly involved in the classification decision, potentially bringing to light brain regions which might reveal disease state.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Achiron, A., Chapman, J., Tal, S., Bercovich, E., Gil, H., Achiron, A.: Superior temporal gyrus thickness correlates with cognitive performance in multiple sclerosis. Brain Struct. Funct. 218(4), 943–950 (2013)
Alber, M., et al.: Innvestigate neural networks!. J. Mach. Learn. Res. 20(93), 1–8 (2019)
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS One 10(7), e0130140 (2015)
Binder, A., Montavon, G., Lapuschkin, S., Müller, K.-R., Samek, W.: Layer-wise relevance propagation for neural networks with local renormalization layers. In: Villa, A.E.P., Masulli, P., Pons Rivero, A.J. (eds.) ICANN 2016, Part II. LNCS, vol. 9887, pp. 63–71. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44781-0_8
Böhle, M., Eitel, F., Weygandt, M., Ritter, K.: Layer-wise relevance propagation for explaining deep neural network decisions in MRI-based Alzheimer’s disease classification. Front. Aging Neurosci. 11, 194 (2019). https://doi.org/10.3389/fnagi.2019.00194
Calabrese, M., Castellaro, M.: Cortical gray matter MR imaging in multiple sclerosis. Neuroimaging Clin. 27(2), 301–312 (2017)
Calabrese, M., et al.: Regional distribution and evolution of gray matter damage in different populations of multiple sclerosis patients. PloS One 10(8), e0135428 (2015)
Calabrese, M., et al.: Basal ganglia and frontal/parietal cortical atrophy is associated with fatigue in relapsing-remitting multiple sclerosis. Mult. Scler. J. 16(10), 1220–1228 (2010). https://doi.org/10.1177/1352458510376405. pMID: 20670981
Compston, A., Coles, A.: Multiple sclerosis. The Lancet 372(9648), 1502–1517 (2008). https://doi.org/10.1016/S0140-6736(08)61620-7
De Santis, S., et al.: Characterizing microstructural tissue properties in multiple sclerosis with diffusion MRI at 7 t and 3 t: the impact of the experimental design. Neuroscience 403, 17–26 (2019)
Eitel, F., et al.: Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation. arXiv preprint arXiv:1904.08771 (2019)
Eshaghi, A., et al.: Progression of regional grey matter atrophy in multiple sclerosis. Brain 141(6), 1665–1677 (2018)
Fischl, B.: FreeSurfer. Neuroimage 62(2), 774–781 (2012)
Geurts, J.J., Barkhof, F.: Grey matter pathology in multiple sclerosis. The Lancet Neurol. 7(9), 841–851 (2008)
Geurts, J.J., Calabrese, M., Fisher, E., Rudick, R.A.: Measurement and clinical effect of grey matter pathology in multiple sclerosis. The Lancet Neurol. 11(12), 1082–1092 (2012)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (June 2016)
Huang, W.J., Chen, W.W., Zhang, X.: Multiple sclerosis: pathology, diagnosis and treatments. Exp. Ther. Med. 13(6), 3163–3166 (2017)
Hulst, H.E., Geurts, J.J.: Gray matter imaging in multiple sclerosis: what have we learned? BMC Neurol. 11(1), 153 (2011)
Hurwitz, B.J.: The diagnosis of multiple sclerosis and the clinical subtypes. Ann. Indian Acad. Neurol. 12(4), 226 (2009)
Inojosa, H., Proschmann, U., Akgün, K., Ziemssen, T.: A focus on secondary progressive multiple sclerosis (SPMS): challenges in diagnosis and definition. J. Neurol. 1–12 (2019). https://doi.org/10.1007/s00415-019-09489-5
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Korolev, S., Safiullin, A., Belyaev, M., Dodonova, Y.: Residual and plain convolutional neural networks for 3D brain MRI classification. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pp. 835–838. IEEE (2017)
Lassmann, H.: Multiple sclerosis pathology. Cold Spring Harb. Perspect. Med. 8(3), a028936 (2018)
Lublin, F.D., et al.: Defining the clinical course of multiple sclerosis: the 2013 revisions. Neurology 83(3), 278–286 (2014)
Lucchinetti, C., Brück, W., Parisi, J., Scheithauer, B., Rodriguez, M., Lassmann, H.: Heterogeneity of multiple sclerosis lesions: implications for the pathogenesis of demyelination. Ann. Neurol. Off. J. Am. Neurol. Assoc. Child Neurol. Soc. 47(6), 707–717 (2000)
Magliozzi, R., et al.: Inflammatory intrathecal profiles and cortical damage in multiple sclerosis. Ann. Neurol. 83(4), 739–755 (2018). https://doi.org/10.1002/ana.25197
Manca, R., Sharrack, B., Paling, D., Wilkinson, I.D., Venneri, A.: Brain connectivity and cognitive processing speed in multiple sclerosis: a systematic review. J. Neurol. Sci. 388, 115–127 (2018)
Marzullo, A., et al.: Classification of multiple sclerosis clinical profiles via graph convolutional neural networks. Front. Neurosci. 13, 594 (2019)
Miller, D., Thompson, A., Filippi, M.: Magnetic resonance studies of abnormalities in the normal appearing white matter and grey matter in multiple sclerosis. J. Neurol. 250(12), 1407–1419 (2003)
Montavon, G., Samek, W., Müller, K.R.: Methods for interpreting and understanding deep neural networks. Digit. Signal Process. 73, 1–15 (2018)
Nourbakhsh, B., Mowry, E.M.: Multiple sclerosis risk factors and pathogenesis. CONTINUUM: Lifelong Learn. Neurol. 25(3), 596–610 (2019)
Paszke, A., et al.: Automatic differentiation in PyTorch. In: Conference on Neural Information Processing Systems (NIPS) (2017)
Popescu, V., et al.: Brain atrophy and lesion load predict long term disability in multiple sclerosis. J. Neurol. Neurosurg. Psychiatry 84(10), 1082–1091 (2013). https://doi.org/10.1136/jnnp-2012-304094
Rieke, J., Eitel, F., Weygandt, M., Haynes, J.-D., Ritter, K.: Visualizing convolutional networks for MRI-based diagnosis of Alzheimer’s disease. In: Stoyanov, D., et al. (eds.) MLCN/DLF/IMIMIC -2018. LNCS, vol. 11038, pp. 24–31. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02628-8_3
Schmidt, P., et al.: An automated tool for detection of flair-hyperintense white-matter lesions in multiple sclerosis. Neuroimage 59(4), 3774–3783 (2012)
Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. arXiv preprint arXiv:1704.02685 (2017)
Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806 (2014)
Vercellino, M., et al.: Demyelination, inflammation, and neurodegeneration in multiple sclerosis deep gray matter. J. Neuropathol. Exp. Neurol. 68(5), 489–502 (2009)
Xie, N., Ras, G., van Gerven, M., Doran, D.: Explainable deep learning: A field guide for the uninitiated. arXiv preprint arXiv:2004.14545 (2020)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part I. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
Zintgraf, L.M., Cohen, T.S., Adel, T., Welling, M.: Visualizing deep neural network decisions: Prediction difference analysis. arXiv preprint arXiv:1702.04595 (2017)
Acknowledgements
This work has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (ERC Advanced Grant agreement no. 694665: CoBCoM - Computational Brain Connectivity Mapping) and from the French government, through the 3IA Côte d’Azur Investments in the Future project managed by the National Research Agency (ANR) with the reference number ANR-10-P3IA-0002.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Cruciani, F. et al. (2021). Explainable 3D-CNN for Multiple Sclerosis Patients Stratification. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12663. Springer, Cham. https://doi.org/10.1007/978-3-030-68796-0_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-68796-0_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68795-3
Online ISBN: 978-3-030-68796-0
eBook Packages: Computer ScienceComputer Science (R0)