research-article

Review of Deep Learning Models for Spine Segmentation

Authors:

Longfei ZhouAuthors Info & Claims

ICMR '22: Proceedings of the 2022 International Conference on Multimedia Retrieval

Pages 498 - 507

https://doi.org/10.1145/3512527.3531356

Published: 27 June 2022 Publication History

Abstract

Medical image segmentation has been a long-standing chal- lenge due to the limitation in labeled datasets and the exis- tence of noise and artifacts. In recent years, deep learning has shown its capability in achieving successive progress in this field, making its automatic segmentation performance gradually catch up with that of manual segmentation. In this paper, we select twelve state-of-the-art models and compare their performance in the spine MRI segmentation task. We divide them into two categories. One of them is the U-Net family, including U-Net, Attention U-Net, ResUNet++, TransUNet, and MiniSeg. The architectures of these models often ultimately include the encoder-decoder structure, and their innovation generally lies in the way of better fusing low-level and high-level information. Models in the other category, named Models Using Backbone often use ResNet, Res2Net, or other pre-trained models on ImageNet as the backbone to extract information. These models pay more attention capturing multi-scale and rich contextual information. All models are trained and tested on the open-source spine M- RI dataset with 20 labels and no pre-training. Through the comparison, the models using backbone exceed U-Net family, and DeepLabv3+ works best. We suppose it is also necessary to extract multi-scale information in a multi-label medical segmentation task.

Supplementary Material

MP4 File (presentation_LF.mp4)

Presentation video

Download
274.86 MB

References

[1]

A. Kulkarni, R. Shevgaonkar, and S. Sahasrabudhe, "Edge detection using scale space knowledge," in Proceedings of TENCON'93. IEEE Region 10 International Conference on Computers, Communications and Automation, vol. 2. IEEE, 1993, pp. 986--990.

[2]

J. K. Leader, B. Zheng, R. M. Rogers, F. C. Sciurba, A. Perez, B. E. Chapman, S. Patel, C. R. Fuhrman, and D. Gur, "Automated lung segmentation in x-ray computed tomography: development and evaluation of a heuristic threshold-based scheme1," Academic radiology, vol. 10, no. 11, pp. 1224--1236, 2003.

[3]

H. Gao, L. Dou, W. Chen, and G. Xie, "The applications of image segmentation techniques in medical ct images," in Proceedings of the 30th Chinese Control Conference. IEEE, 2011, pp. 3296--3299.

[4]

D. Gawel, P. Glówka, T. Kotwicki, and M. Nowak, "Automatic spine tissue segmentation from mri data based on cascade of boosted classifiers and active appearance model," BioMed research international, vol. 2018, 2018.

[5]

Y. Tian, T. Guan, C. Wang, L. Li, and W. Liu, "Interactive foreground segmentation method using mean shift and graph cuts," Sensor Review, 2009.

[6]

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. Van Der Laak, B. Van Ginneken, and C. I. Sánchez, "A survey on deep learning in medical image analysis," Medical image analysis, vol. 42, pp. 60--88, 2017.

[7]

A. Suzani, A. Rasoulian, A. Seitel, S. Fels, R. N. Rohling, and P. Abolmaesumi, "Deep learning for automatic localization, identification, and segmentation of vertebral bodies in volumetric mr images," in Medical Imaging 2015: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 9415. International Society for Optics and Photonics, 2015, p. 941514.

[8]

A. Arpitha and L. Rangarajan, "Computational techniques to segment and classify lumbar compression fractures," La radiologia medica, vol. 125, no. 6, pp. 551--560, 2020.

[9]

L. Silvoster and M. Retnaswami, "Spine mri segmentation and efficient detection of lumbar intervertebral disc degeneration," IET Image Processing, vol. 14, 2020.

[10]

A. S. Al-Kafri, S. Sudirman, A. Hussain, D. Al-Jumeily, F. Natalia, H. Meidia, N. Afriliana, W. Al-Rashdan, M. Bashtawi, and M. Al-Jumaily, "Boundary delineation of mri images for lumbar spinal stenosis detection through semantic segmentation using deep neural networks," IEEE Access, vol. 7, pp. 43 487--43 501, 2019.

[11]

A. Suzani, A. Rasoulian, S. Fels, R. N. Rohling, and P. Abolmaesumi, "Semi-automatic segmentation of vertebral bodies in volumetric mr images using a statistical shape+ pose model," in Medical Imaging 2014: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 9036. International Society for Optics and Photonics, 2014, p. 90360P.

[12]

R. Zhang, X. Xiao, Z. Liu, Y. Li, and S. Li, "Mrln: Multi-task relational learning network for mri vertebral localization, identification, and segmentation," IEEE journal of biomedical and health informatics, vol. 24, no. 10, pp. 2902--2911, 2020.

[13]

J. Huang, H. Shen, J. Wu, X. Hu, Z. Zhu, X. Lv, Y. Liu, and Y. Wang, "Spine explorer: a deep learning based fully automated program for efficient and reliable quantifications of the vertebrae and discs on sagittal lumbar spine mr images," The Spine Journal, vol. 20, no. 4, pp. 590--599, 2020.

[14]

O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in International Conference on Medical image computing and computer-assisted intervention. Springer, 2015, pp. 234--241.

[15]

O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y. Hammerla, B. Kainz et al., "Attention u-net: Learning where to look for the pancreas," arXiv preprint arXiv:1804.03999, 2018.

[16]

D. Jha, P. H. Smedsrud, M. A. Riegler, D. Johansen, T. De Lange, P. Halvorsen, and H. D. Johansen, "Resunet++: An advanced architecture for medical image segmentation," in 2019 IEEE International Symposium on Multimedia (ISM). IEEE, 2019, pp. 225--2255.

[17]

J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, and Y. Zhou, "Transunet: Transformers make strong encoders for medical image segmentation," arXiv preprint arXiv:2102.04306, 2021.

[18]

Y. Qiu, Y. Liu, S. Li, and J. Xu, "Miniseg: An extremely minimum network for efficient covid-19 segmentation," arXiv preprint arXiv:2004.09750, 2020.

[19]

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in 2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248--255.

[20]

K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770--778.

[21]

S. Gao, M.-M. Cheng, K. Zhao, X.-Y. Zhang, M.-H. Yang, and P. H. Torr, "Res2net: A new multi-scale backbone architecture," IEEE transactions on pattern analysis and machine intelligence, 2019.

[22]

S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, "Aggregated residual transformations for deep neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1492--1500.

[23]

F. Yu, D. Wang, E. Shelhamer, and T. Darrell, "Deep layer aggregation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2403--2412.

[24]

H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, "Pyramid scene parsing network," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2881--2890.

[25]

L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, "Encoder-decoder with atrous separable convolution for semantic image segmentation," in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 801--818.

[26]

M. Yang, K. Yu, C. Zhang, Z. Li, and K. Yang, "Denseaspp for semantic segmentation in street scenes," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3684--3692.

[27]

J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu, "Dual attention network for scene segmentation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 3146--3154.

[28]

X. Li, Z. Zhong, J. Wu, Y. Yang, Z. Lin, and H. Liu, "Expectation-maximization attention networks for semantic segmentation," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9167--9176.

[29]

D.-P. Fan, T. Zhou, G.-P. Ji, Y. Zhou, G. Chen, H. Fu, J. Shen, and L. Shao, "Infnet: Automatic covid-19 lung infection segmentation from ct images," IEEE Transactions on Medical Imaging, vol. 39, no. 8, pp. 2626--2637, 2020.

[30]

H. Zhao, Y. Zhang, S. Liu, J. Shi, C. C. Loy, D. Lin, and J. Jia, "Psanet: Point-wise spatial attention network for scene parsing," in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 267--283.

[31]

F. Pérez-García, R. Sparks, and S. Ourselin, "Torchio: a python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning," Computer Methods and Programs in Biomedicine, p. 106236, 2021.

Digital Library

[32]

F. Milletari, N. Navab, and S.-A. Ahmadi, "V-net: Fully convolutional neural networks for volumetric medical image segmentation," in 2016 fourth international conference on 3D vision (3DV). IEEE, 2016, pp. 565--571.

[33]

J. M. J. Valanarasu, P. Oza, I. Hacihaliloglu, and V. M. Patel, "Medical transformer: Gated axial-attention for medical image segmentation," arXiv preprint arXiv:2102.10662, 2021.

[34]

A. Hatamizadeh, Y. Tang, V. Nath, D. Yang, A. Myronenko, B. Landman, H. Roth, and D. Xu, "Unetr: Transformers for 3d medical image segmentation," arXiv preprint arXiv:2103.10504, 2021.

[35]

H. Cao, Y. Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, "Swinunet: Unet-like pure transformer for medical image segmentation," arXiv preprint arXiv:2105.05537, 2021.

[36]

F. Isensee, P. F. Jäger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, "Automated design of deep learning methods for biomedical image segmentation," arXiv preprint arXiv:1904.08128, 2019.

Cited By

Nohel MJakubicek RBlazkova LValek VDostal MOurednicek PChmelik J(2024)Comparison of Spine Segmentation Algorithms on Clinical Data from Spectral CT of Patients with Multiple MyelomaMEDICON’23 and CMBEBIH’2310.1007/978-3-031-49062-0_34(309-317)Online publication date: 4-Jan-2024
https://doi.org/10.1007/978-3-031-49062-0_34
Bansal TGupta SJindal N(2023)Segmentation Techniques for Detection of Tuberculosis Using Deep Learning: A Review2023 3rd International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON)10.1109/SMARTGENCON60755.2023.10442736(1-6)Online publication date: 29-Dec-2023
https://doi.org/10.1109/SMARTGENCON60755.2023.10442736
Lai YChang IYang TKuo LLeong S(2023)A Multi-views Models Ensemble Method for Thumb Trapeziometacarpal Joint Segmentation from Computed Tomography Images2023 IEEE 12th Global Conference on Consumer Electronics (GCCE)10.1109/GCCE59613.2023.10315468(1033-1037)Online publication date: 10-Oct-2023
https://doi.org/10.1109/GCCE59613.2023.10315468
Show More Cited By

Index Terms

Review of Deep Learning Models for Spine Segmentation
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Phonology / morphology

Recommendations

Knowledge-based segmentation of spine and ribs from bone scintigraphy
ICONIP'11: Proceedings of the 18th international conference on Neural Information Processing - Volume Part I

We propose a novel method for the segmentation of spine and ribs from posterior whole-body bone scintigraphy images. The knowledge-based method is first applied to determine the thoracic region. An adaptive thresholding method is then used to extract ...
Review on deep learning fetal brain segmentation from Magnetic Resonance images
Abstract
Brain segmentation is often the first and most critical step in quantitative analysis of the brain for many clinical applications, including fetal imaging. Different aspects challenge the segmentation of the fetal brain in magnetic resonance ...
Highlights
- We reviewed 39 DL studies for fetal brain segmentation from MR images
- U-Net backbone is the dominant method for automatic fetal brain segmentation
- There is a segmentation performances convergence for the currently available ...
A comprehensive review: Segmentation of MRI images-brain tumor

Segmentation of tumors in human brain aims to classify different abnormal tissues necrotic core, edema, active cells from normal tissues cerebrospinal fluid, gray matter, white matter of the brain. In existence, detection of abnormal tissues is easy for ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '22: Proceedings of the 2022 International Conference on Multimedia Retrieval

June 2022

714 pages

ISBN:9781450392389

DOI:10.1145/3512527

General Chairs:
Vincent Oria
New Jersey Institute of Technology, USA
,
Maria Luisa Sapino
Università degli Studi di Torino, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Brigitte Kerhervé
Université du Québec à Montréal, Canada
,
Program Chairs:
Wen-Huang Cheng
National Yang Ming Chao Tung University, Taiwan
,
Ichiro Ide
Nagoya University, Japan
,
Vivek Singh
Rutgers University, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICMR '22

Sponsor:

SIGMM

ICMR '22: International Conference on Multimedia Retrieval

June 27 - 30, 2022

NJ, Newark, USA

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
221
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)3

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nohel MJakubicek RBlazkova LValek VDostal MOurednicek PChmelik J(2024)Comparison of Spine Segmentation Algorithms on Clinical Data from Spectral CT of Patients with Multiple MyelomaMEDICON’23 and CMBEBIH’2310.1007/978-3-031-49062-0_34(309-317)Online publication date: 4-Jan-2024
https://doi.org/10.1007/978-3-031-49062-0_34
Bansal TGupta SJindal N(2023)Segmentation Techniques for Detection of Tuberculosis Using Deep Learning: A Review2023 3rd International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON)10.1109/SMARTGENCON60755.2023.10442736(1-6)Online publication date: 29-Dec-2023
https://doi.org/10.1109/SMARTGENCON60755.2023.10442736
Lai YChang IYang TKuo LLeong S(2023)A Multi-views Models Ensemble Method for Thumb Trapeziometacarpal Joint Segmentation from Computed Tomography Images2023 IEEE 12th Global Conference on Consumer Electronics (GCCE)10.1109/GCCE59613.2023.10315468(1033-1037)Online publication date: 10-Oct-2023
https://doi.org/10.1109/GCCE59613.2023.10315468
Saha PNadeem SComellas A(2023)A survey on artificial intelligence in pulmonary imagingWIREs Data Mining and Knowledge Discovery10.1002/widm.151013:6Online publication date: 7-Jul-2023
https://doi.org/10.1002/widm.1510

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents