Abstract
The hippocampus has been recognized as an important biomarker for the diagnosis and assessment of neurological diseases. Convenient and accurate automated segmentation of the hippocampus facilitates the analysis of large-scale neuroimaging studies. This work describes a novel technique for hippocampus segmentation in magnetic resonance images, in which interactive neural network (Inter-Net) is based on 3D convolutional operations. Inter-Net achieves the interaction through two aspects: one is the compartments, which builds an exponential ensemble network that integrates numerous short networks together when forward propagation. The other is the pathways, which realizes inter-connection between feature extraction and restoration. In addition, a multi-target architecture is proposed by designing multiple objective functions in terms of evaluation index, information theory, and data distribution. The proposed architecture is validated in fivefold cross-validation on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, where the mean Dice similarity indices of 0.919 (± 0.023) and precision of 0.926 (± 0.032) for the hippocampus segmentation. The running time is approximately 42.1 s from reading the image to outputting the segmentation result in our computer configuration. We compare the experimental results of a variety of methods to prove the effectiveness of the Inter-Net and contrast integrated architectures with different objective functions to illustrate the robustness of the fusion. The proposed framework is general and can be easily extended to numerous tissue segmentation tasks while it is tailored for the hippocampus.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Czepielewski LS, Wang L, Gama CS, et al. The relationship of intellectual functioning and cognitive performance to brain structure in schizophrenia. Schizophr Bull. 2017;43(2):355–64.
Steiger VR, Brühl AB, Weidt S, Delsignore A, Rufer M, Jäncke L, et al. Pattern of structural brain changes in social anxiety disorder after cognitive behavioral group therapy: a longitudinal multimodal MRI study. Mol Psychiatry. 2017;22(8):1164–71.
den Heijer T, van der Lijn F, Vernooij MW, et al. Structural and diffusion MRI measures of the hippocampus and memory performance. Neuroimage. 2012;63(4):1782–9.
Wixted JT, Squire LR. The medial temporal lobe and the attributes of memory. Trends Cogn Sci. 2011;15(5):210–7.
Jeneson A, Squire LR. Working memory, long-term memory, and medial temporal lobe function. Learn Mem. 2012;19(1):15–25.
Bobinski M, Wegiel J, Wisniewski HM, Tarnawski M, Bobinski M, Reisberg B, et al. Neurofibrillary pathology—correlation with hippocampal formation atrophy in Alzheimer disease. Neurobiol Aging. 1996;17(6):909–19.
Geuze E, Vermetten E, Bremner JD. MR-based in vivo hippocampal volumetrics: 1. Review of methodologies currently employed. Mol Psychiatry. 2005;10(2):147–59.
Knickmeyer RC, Gouttard S, Kang C, Evans D, Wilber K, Smith JK, et al. A structural MRI study of human brain development from birth to 2 years. J Neurosci. 2008;28(47):12176–82.
Filippi M, Rocca MA, Ciccarelli O, De Stefano N, Evangelou N, Kappos L, et al. MRI criteria for the diagnosis of multiple sclerosis: MAGNIMS consensus guidelines. Lancet Neurol. 2016;15(3):292–303.
Jacobsen C, Hagemeier J, Myhr KM, Nyland H, Lode K, Bergsland N, et al. Brain atrophy and disability progression in multiple sclerosis patients: a 10-year follow-up study. J Neurol Neurosurg Psychiatry. 2014;85(10):1109–15.
Andreasen NC, Liu D, Ziebell S, Vora A, Ho BC. Relapse duration, treatment intensity, and brain tissue loss in schizophrenia: a prospective longitudinal MRI study. Am J Psychiatr. 2013;170(6):609–15.
Scheenstra AEH, van de Ven RCG, van der Weerd L, van den Maagdenberg AM, Dijkstra J, Reiber JH. Automated segmentation of in vivo and ex vivo mouse brain magnetic resonance images. Mol Imaging. 2009;8(1):35–44.
Carmichael OT, Aizenstein HA, Davis SW, Becker JT, Thompson PM, Meltzer CC, et al. Atlas-based hippocampus segmentation in Alzheimer’s disease and mild cognitive impairment. Neuroimage. 2005;27(4):979–90.
Chupin M, Mukuna-Bantumbakulu AR, Hasboun D, Bardinet E, Baillet S, Kinkingnéhun S, et al. Anatomically constrained region deformation for the automated segmentation of the hippocampus and the amygdala: method and validation on controls and patients with Alzheimer’s disease. Neuroimage. 2007;34(3):996–1019.
Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Haselgrove C, et al. Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron. 2002;33(3):341–55.
Zandifar A, Fonov V, Coupé P, Pruessner J, Collins DL, Alzheimer’s Disease Neuroimaging Initiative. A comparison of accurate automatic hippocampal segmentation methods. NeuroImage. 2017;155:383–93.
Hosseini MP, Nazem Zadeh MR, Pompili D, Jafari-Khouzani K, Elisevich K, Soleanian-Zadeh H. Comparative performance evaluation of automated segmentation methods of hippocampus from magnetic resonance images of temporal lobe epilepsy patients. Med Phys. 2016;43(1):538–53.
Dill V, Franco AR, Pinho MS. Automated methods for hippocampus segmentation: the evolution and a review of the state of the art. Neuroinformatics. 2015;13(2):133–50.
Birenbaum A, Greenspan H. Multi-view longitudinal CNN for multiple sclerosis lesion segmentation. Eng Appl Artif Intell. 2017;65:111–8.
Kwak K, Yoon U, Lee DK, Kim GH, Seo SW, Na DL. Fully-automated approach to hippocampus segmentation using a graph-cuts algorithm combined with atlas-based segmentation and morphological opening. Magn Reson Imaging. 2013;31(7):1190–6.
Pipitone J, Park MTM, Winterburn J, Lett TA, Lerch JP, Pruessner JC, et al. Multi-atlas segmentation of the whole hippocampus and subfields using multiple automatically generated templates. Neuroimage. 2014;101:494–512.
Sabuncu MR, Yeo BTT, Van Leemput K, Fischl B, Golland P. A generative model for image segmentation based on label fusion. IEEE Trans Med Imaging. 2010;29(10):1714–29.
Van der Lijn F, De Bruijne M, Klein S, Den Heijer T, Hoogendam YY, Van der Lugt A, et al. Automated brain structure segmentation based on atlas registration and appearance models. IEEE Trans Med Imaging. 2012;31(2):276–86.
Kim M, Wu G, Li W, Wang L, Son YD, Cho ZH, et al. Automatic hippocampus segmentation of 7.0 Tesla MR images by combining multiple atlases and auto-context models. NeuroImage. 2013;83:335–45.
Hao Y, Wang T, Zhang X, Duan Y, Yu C, Jiang T, et al. Local label learning (LLL) for subcortical structure segmentation: application to hippocampus segmentation. Hum Brain Mapp. 2014;35(6):2674–97.
Moghaddam MJ, Soltanian-Zadeh H. Automatic segmentation of brain structures using geometric moment invariants and artificial neural networks//International conference on Information Processing in Medical Imaging. Berlin: Springer; 2009. p. 326–37.
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, 2015. pp. 3431–3440.
Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. International conference on medical image computing and computer-assisted intervention. Cham: Springer; 2015. p. 234–41.
Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal. 2017;36:61–78.
Liu X, Deng Z. Segmentation of drivable road using deep fully convolutional residual network with pyramid pooling. Cogn Comput. 2017:1–10.
Liu W, Tao D. Multiview Hessian regularization for image annotation. IEEE Trans Image Process. 2013;22(7):2676–87.
Liu W, Yang X, Tao D, Cheng J, Tang Y. Multiview dimension reduction via Hessian multiset canonical correlations. Information Fusion. 2018;41:119–28.
Yuan Y, Xun G, Ma F, et al. Muvan: a multi-view attention network for multivariate temporal data. 2018 IEEE International Conference on Data Mining (ICDM). Piscataway: IEEE; 2018. p. 717–26.
Kang G, Liu K, Hou B, Zhang N. 3D multi-view convolutional neural networks for lung nodule classification. PloS one, Public Library of Science. 2017;12(11):e0188290.
Setio AAA, Ciompi F, Litjens G, Gerke P, Jacobs C, van Riel SJ, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging. 2016;35(5):1160–9.
Fischl B. FreeSurfer. Neuroimage. 2012;62(2):774–81.
Chen Y, Shi B, Wang Z, Zhang P, Smith CD, Liu J. Hippocampus segmentation through multi-view ensemble ConvNets[C]//Biomedical Imaging (ISBI 2017), 2017 IEEE 14th International Symposium on. IEEE, 2017. pp. 192–196.
Jack CR Jr, Bernstein MA, Fox NC, et al. The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J Magn Reson Imaging. 2008;27(4):685–91.
Wen G, Hou Z, Li H, Li D, Jiang L, Xun E. Ensemble of deep neural networks with probability-based fusion for facial expression recognition. Cogn Comput. 2017;9(5):597–610.
Brosch T, Tang LY, Yoo Y, Li DK. Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation. IEEE Trans Med Imaging. 2016;35(5):1229–39.
Veit A, Wilber M, Belongie S. Residual networks are exponential ensembles of relatively shallow networks. arXiv preprint. arXiv preprint arXiv:1605.06431. 2016;1(2):3.
He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. European Conference on Computer Vision. Cham: Springer; 2016. p. 630–45.
Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th international conference on machine learning (ICML-10). 2010. pp. 807–814.
Zeiler MD. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. 2012.
Kingma DP, Ba J. Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.
Dauphin Y, de Vries H, Bengio Y. Equilibrated adaptive learning rates for non-convex optimization[C]. Adv Neural Inf Proces Syst. 2015:1504–12.
Srivastava N, Hinton G, Krizhevsky A, Stuskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15(1):1929–58.
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014.
Zeng D, Zhao F, Shen W, Ge S. Compressing and accelerating neural network for facial point localization. Cogn Comput. 2017:1–9.
Dice LR. Measures of the amount of ecologic association between species. Ecology. 1945;26(3):297–302.
Cabezas M, Oliver A, Lladó X, Freixenet J, Cuadra MB. A review of atlas-based segmentation for magnetic resonance brain images. Comput Methods Prog Biomed. 2011;104(3):e158–77.
Ghanei A, Soltanian-Zadeh H, Windham JP. A 3D deformable surface model for segmentation of objects from volumetric data in medical images. Comput Biol Med. 1998;28(3):239–2.
Lötjönen JMP, Wolz R, Koikkalainen JR, Thurfjell L, Waldemar G, Soininen H, et al. Fast and robust multi-atlas segmentation of brain magnetic resonance images. Neuroimage. 2010;49(3):2352–65.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. pp. 770–778.
Wolz R, Aljabar P, Hajnal JV, Hammers A, Rueckert D. Alzheimer’s Disease Neuroimaging Initiative. LEAP: learning embeddings for atlas propagation. NeuroImage. 2010;49(2):1316–25.
Funding
This work was supported by National Natural Science Foundation of China (61471064), National Science and Technology Major Project of China (No.2017ZX03001022), and BUPT Excellent Ph.D. Students Foundation (No.CX2019309).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Ethical Approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Hou, B., Kang, G., Zhang, N. et al. Multi-target Interactive Neural Network for Automated Segmentation of the Hippocampus in Magnetic Resonance Imaging. Cogn Comput 11, 630–643 (2019). https://doi.org/10.1007/s12559-019-09645-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-019-09645-z