Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Multi-level Kronecker Convolutional Neural Network (ML-KCNN) for Glioma Segmentation from Multi-modal MRI Volumetric Data

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

The development of an automated glioma segmentation system from MRI volumes is a difficult task because of data imbalance problem. The ability of deep learning models to incorporate different layers for data representation assists medical experts like radiologists to recognize the condition of the patient and further make medical practices easier and automatic. State-of-the-art deep learning algorithms enable advancement in the medical image segmentation area, such a segmenting the volumes into sub-tumor classes. For this task, fully convolutional network (FCN)-based architectures are used to build end-to-end segmentation solutions. In this paper, we proposed a multi-level Kronecker convolutional neural network (MLKCNN) that captures information at different levels to have both local and global level contextual information. Our ML-KCNN uses Kronecker convolution, which overcomes the missing pixels problem by dilated convolution. Moreover, we used a post-processing technique to minimize false positive from segmented outputs, and the generalized dice loss (GDL) function handles the data-imbalance problem. Furthermore, the combination of connected component analysis (CCA) with conditional random fields (CRF) used as a post-processing technique achieves reduced Hausdorff distance (HD) score of 3.76 on enhancing tumor (ET), 4.88 on whole tumor (WT), and 5.85 on tumor core (TC). Dice similarity coefficient (DSC) of 0.74 on ET, 0.90 on WT, and 0.83 on TC. Qualitative and visual evaluation of our proposed method shown effectiveness of the proposed segmentation method can achieve performance that can compete with other brain tumor segmentation techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Louis, David N., et al: The 2016 World Health Organization classification of tumors of the central nervous system: a summary. Acta Neuropathologica 131.6:803–820,2016

  2. Centers for Disease Control and Prevention: Data collection of primary central nervous system tumors. National Program of Cancer Registries Training Materials. Atlanta, Georgia: Department of Health and Human Services, Centers for Disease Control and Prevention, 2004

  3. Brain Tumor - Diagnosis. 18 Mar. 2019, www.cancer.net/cancer-types/brain-tumor/diagnosis. Last Accessed: 25 April 2020

  4. Galanaud, Damien, et al: Noninvasive diagnostic assessment of brain tumors using combined in vivo MR imaging and spectroscopy. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 55.6:1236–1245,2006

  5. Rees, J: Advances in magnetic resonance imaging of brain tumours. Current Opinion in Neurology 16.6:643–650,2003

    Article  Google Scholar 

  6. UCSF Department of Radiology & Biomedical Imaging. Exploring the Brain: Is CT or MRI Better for Brain Imaging? UCSF Radiology, 16 Nov. 2015, https://radiology.ucsf.edu/blog/neuroradiology/exploring-the-brain-is-ct-or-mri-better-for-brain-imaging

  7. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, et al: The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS), IEEE Transactions on Medical Imaging 34(10):1993–2024,2015. https://doi.org/10.1109/TMI.2014.2377694

  8. Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems. 2012.

  9. Keçeli AS, Aydın K, Ahmet BC: Combining 2D and 3D deep models for action recognition with depth information. Signal, Image and Video Processing 12.6:1197–1205,2018

  10. van Harten, Louis, et al: Automatic segmentation of organs at risk in thoracic CT scans by combining 2D and 3D convolutional neural networks. SegTHOR@ ISBI. 2019

  11. Yu, Fisher, and Vladlen Koltun: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 2015

  12. Wu, Tianyi, et al. Tree-structured kronecker convolutional network for semantic segmentation. 2019 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2019

  13.  Zheng S, Jayasumana S, Romera-Paredes B, Vineet V, Su S, Du D, Huang C, Torr PH: Conditional random fields as recurrent neural networks. In Proceedings of the IEEE International Conference on Computer Vision, 2015, pp 1529–1537

  14. Peng C, et al: Large kernel matters—improve semantic segmentation by global convolutional network. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017

  15. Adams A, Baek J, Davis MA: Fast high-dimensional filtering using the permutohedral lattice. In Computer Graphics Forum, volume 29, pp 753–762. Wiley Online Library, 2010

  16. Liu Z, Li X, Luo P, Loy C-C, Tang X: Semantic image segmentation via deep parsing network. In Proceedings of the IEEE International Conference on Computer Vision, 2015, pages 1377–1385

  17. Lin G, Shen C, van den Hengel A, Reid I: Efficient piecewise training of deep structured models for semantic segmentation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

  18. Barron JT, Poole B: The fast bilateral solver. ECCV, 2016

  19. Sudre CH, Li W, Vercauteren T, Ourselin S, Jorge Cardoso M: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Cardoso M. et al. (eds) Deep learning in medical image analysis and multimodal learning for clinical decision support. DLMIA 2017, ML-CDS 2017. Lecture Notes in Computer Science, vol 10553. Springer, Cham, 2017

  20. Sargur N. Srihari. Machine learning: generative and discriminative models. Cedar University of Befallo, University of Befallo, https://cedar.buffalo.edu/~srihari/CSE574/Discriminative-Generative.pdf

  21. Lee C-H, et al: Segmenting brain tumors with conditional random fields and support vector machines. International Workshop on Computer Vision for Biomedical Image Applications. Springer, Berlin, Heidelberg, 2005

  22. Bauer S, Nolte L-P, Reyes M: Fully automatic segmentation of brain tumor images using support vector machine classification in combination with hierarchical conditional random field regularization. international conference on medical image computing and computer-assisted intervention. Springer, Berlin, Heidelberg, 2011

  23. He K, et al: Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, 2016

  24. Long J, Shelhamer E, Darrell T: Fully convolutional networks for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, 2015

  25. Mlynarski P, et al: 3D convolutional neural networks for tumor segmentation using long-range 2D context. Computerized Medical Imaging and Graphics 73:60–72,2019

  26. Pereira S, et al: Adaptive feature recombination and recalibration for semantic segmentation with Fully Convolutional Networks. IEEE transactions on medical imaging, 2019

  27. Chen H, et al: Brain tumor segmentation with deep convolutional symmetric neural network. Neurocomputing, 2019

  28. Isensee F, et al: No new-net. International MICCAI Brainlesion Workshop. Springer, Cham, 2018

  29. Xu, Hai, et al. Deep cascaded attention network for multi-task brain tumor segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2019

  30. Akil M, Saouli R, Kachouri R: Fully automatic brain tumor segmentation with deep learning-based selective attention using overlapping patches and multi-class weighted cross-entropy. Medical Image Analysis 101692,2020

  31. Ding Yi, et al: A stacked multi-connection simple reducing net for brain tumor segmentation. IEEE Access 7:104011–104024,2019

  32. Chen C, Liu X, Ding M, Zheng J, Li J: 3D Dilated multi-fiber network for real-time brain tumor segmentation in MRI. In: Shen D. et al Eds .Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science, vol 11766. Springer, Cham, 2019

  33. Rafi A et al: U-Net Based Glioblastoma Segmentation with Patient’s Overall Survival Prediction. International Symposium on Intelligent Computing Systems. Springer, Cham, 2020

  34. Wang G, et al: Automatic brain tumor segmentation using convolutional neural networks with test-time augmentation. International MICCAI Brainlesion Workshop. Springer, Cham, 2018

  35. Zhao X, et al: Brain tumor segmentation using a fully convolutional neural network with conditional random fields. International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Springer, Cham, 2016

  36. Wang F, Jiang R, Zheng L, Meng C, Biswal B: 3D U-Net based brain tumor segmentation and survival days prediction. In: Crimi A., Bakas S. (eds) Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries. BrainLes 2019. Lecture Notes in Computer Science, vol 11992. Springer, Cham, 2019

  37. Hamghalamm M, Lei B, Wang T: Brain tumor synthetic segmentation in 3D multimodal MRI scans. arXiv preprint arXiv:1909.13640 2019

  38. Agravat R, Raval MS: Brain tumor segmentation and survival prediction. arXiv preprint arXiv:1909.09399 2019

  39. Amian M, Soltaninejad M: Multi-resolution 3D CNN for MRI brain tumor segmentation and survival prediction. arXiv preprint arXiv:1911.08388 2019

  40. Murugesan GK, et al: Multidimensional and multiresolution ensemble networks for brain tumor segmentation. bioRxiv 760124,2019

  41. Jiang Z, et al: Two-stage cascaded U-Net: 1st place solution to BraTS challenge 2019 segmentation task. International MICCAI Brainlesion Workshop. Springer, Cham, 2019

  42. Vu MH, Nyholm T, Löfstedt T: TuNet: end-to-end hierarchical brain tumor segmentation using cascaded networks. arXiv preprint arXiv:1910.05338 2019

  43. McKinley R, et al: Triplanar ensemble of 3D-to-2D CNNs with label-uncertainty for brain tumor segmentation. International MICCAI Brainlesion Workshop. Springer, Cham, 2019

  44. Milletari F, Navab N, Ahmadi S-Ahmad: V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 fourth international conference on 3D vision (3DV). IEEE, 2016

  45. Bernal J, et al: Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review. Artificial intelligence in medicine 95:64–81, 2019

Download references

Funding

This work has been supported by Higher Education Commission under Grant # 2(1064) and is carried out at Medical Imaging and Diagnostics (MID) Lab at COMSATS University Islamabad, under the umbrella of National Center of Artificial Intelligence (NCAI), Pakistan.

Author information

Authors and Affiliations

  1. Medical Imaging and Diagnostic Lab, National Centre of Artificial Intelligence, Department of Computer Science, COMSATS University Islamabad (CUI), 45550, Islamabad, Pakistan

    Muhammad Junaid Ali, Basit Raza & Ahmad Raza Shahid

Authors
  1. Muhammad Junaid Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Basit Raza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmad Raza Shahid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Basit Raza.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ali, M.J., Raza, B. & Shahid, A.R. Multi-level Kronecker Convolutional Neural Network (ML-KCNN) for Glioma Segmentation from Multi-modal MRI Volumetric Data. J Digit Imaging 34, 905–921 (2021). https://doi.org/10.1007/s10278-021-00486-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-021-00486-7

Keywords

Search

Navigation