Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Deep learning-based magnetic resonance image super-resolution: a survey

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Magnetic resonance imaging (MRI) is a medical imaging technique used to show anatomical structures and physiological processes of the human body. Due to limitations like image acquisition time, hardware capabilities, or uncooperative patients, the resolution of MR images is insufficient. Super-resolution (SR) is a crucial method to enhance the resolution of images without expensive scanning equipment. Recent years have witnessed significant progress in MR image super-resolution. Therefore, this survey presents a thorough overview of current developments in deep learning-based MR image super-resolution methods. In general, we can roughly divide the MRI super-resolution methods into single-contrast MR image SR methods and multi-contrast MR image SR methods. Additionally, we introduce the multi-task learning approaches about the MR image super-resolution. We also summarize other crucial topics, such as the degradation model, the definition of the super-resolution problem, the dataset, loss functions, and image quality assessment. Lastly, we indicate the challenges in the field of super-resolution and draw a conclusion to our survey.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

HR:

Ground truth MR image

SR:

Super-resolved MR image

LR:

Low-resolution MR image

Ref:

High-resolution reference image

\(\mathcal {D}\) :

Degradation model

\(\delta \) :

The parameter of the degradation model

F :

Super-resolution model

\(\theta \) :

The parameter of the super-resolution model

n :

The number of images

\(\epsilon \) :

Constant (1e−3)

\(\nabla \) :

Gradient

\(\phi \) :

The pre-trained network

\(\phi _{j}(\cdot )\) :

The feature map of the jth layer of the network \(\phi \)

p :

Feature map p

q :

Feature map q

\({\text {vec}}(\cdot )\) :

Vectorization operation

G :

Generator network

D :

Discriminator network

\(\mu _{\text{SR}}\) :

Mean value of the image

\(\sigma _{\text{SR}}\) :

Standard deviation of the image intensity

\(\mathcal {C}_{\text{l}}\) :

Luminance

\(\mathcal {C}_{\text{c}}\) :

Contrast

\(\mathcal {C}_{\text{s}}\) :

Structure

\(C_{1}\) :

Constant

\(C_{2}\) :

Constant

\(C_{3}\) :

Constant

\(\sigma _{\text{SRHR}}\) :

The covariance between SR and HR

\(\alpha \) :

Constant (1)

\(\beta \) :

Constant (1)

\(\gamma \) :

Constant (1)

\(S_{\text{seg}}\) :

Predicted segmentation

\(G_{\text{seg}}\) :

Real segmentation label

References

  1. Ramírez VM, Pinon N, Forbes F, Lartizen C, Dojat M (2021) atch versus global image-based unsupervised anomaly detection in MR brain scans of early parkinsonian patients. In: Machine learning in clinical neuroimaging: 4th international workshop, MLCN 2021, vol. 13001, pp 34–43. https://doi.org/10.1007/978-3-030-87586-2_4

  2. Zhang J, He X, Qing L, Gao F, Wang B (2022) PGAN: brain PET synthesis from MRI using generative adversarial network for multi-modal Alzheimer’s disease diagnosis. Comput Methods Programs Biomed 217:106676. https://doi.org/10.1016/j.cmpb.2022.106676

    Article  Google Scholar 

  3. Mendonça LJC, Ferrari RJ, Initiative ADN (2023) Alzheimer’s disease classification based on graph kernel SVMs constructed with 3d texture features extracted from MR images. Expert Syst Appl 211:118633. https://doi.org/10.1016/j.eswa.2022.118633

    Article  Google Scholar 

  4. Ma X, Zhao Y, Lu Y, Li P, Li X, Mei N, Wang J, Geng D, Zhao L, Yin B (2022) A dual-branch hybrid dilated CNN model for the AI-assisted segmentation of meningiomas in MR images. Comput Biol Med 151(Part):106279. https://doi.org/10.1016/j.compbiomed.2022.106279

    Article  Google Scholar 

  5. Chen C, Qin C, Ouyang C, Li Z, Wang S, Qiu H, Chen L, Tarroni G, Bai W, Rueckert D (2022) Enhancing MR image segmentation with realistic adversarial data augmentation. Medical Image Anal 82:102597. https://doi.org/10.1016/j.media.2022.102597

    Article  Google Scholar 

  6. Wei D, Ahmad S, Guo Y, Chen L, Huang Y, Ma L, Wu Z, Li G, Wang L, Lin W, Yap P, Shen D, Wang Q (2022) Recurrent tissue-aware network for deformable registration of infant brain MR images. IEEE Trans Med Imaging 41(5):1219–1229. https://doi.org/10.1109/TMI.2021.3137280

    Article  Google Scholar 

  7. Zakeri A, Hokmabadi A, Bi N, Wijesinghe I, Nix MG, Petersen SE, Frangi AF, Taylor ZA, Gooya A (2023) Dragnet: learning-based deformable registration for realistic cardiac MR sequence generation from a single frame. Medical Image Anal 83:102678. https://doi.org/10.1016/j.media.2022.102678

    Article  Google Scholar 

  8. Wang L, Du J, Gholipour A, Zhu H, He Z, Jia Y (2021) 3d dense convolutional neural network for fast and accurate single MR image super-resolution. Comput Med Imaging Graph 93:101973. https://doi.org/10.1016/j.compmedimag.2021.101973

    Article  Google Scholar 

  9. Zhu D, Qiu D (2021) Residual dense network for medical magnetic resonance images super-resolution. Comput Methods Programs Biomed 209:106330. https://doi.org/10.1016/j.cmpb.2021.106330

    Article  Google Scholar 

  10. Zhu J, Tan C, Yang J, Yang G, Lio’ P (2021) Arbitrary scale super-resolution for medical images. Int J Neural Syst 31(10):2150037–1215003720. https://doi.org/10.1142/S0129065721500374

    Article  Google Scholar 

  11. Qiu D, Cheng Y, Wang X (2022) Dual u-net residual networks for cardiac magnetic resonance images super-resolution. Comput Methods Programs Biomed 218:106707. https://doi.org/10.1016/j.cmpb.2022.106707

    Article  Google Scholar 

  12. Zhao X, Zhang Y, Qin Y, Wang Q, Zhang T, Li T (2022) Single MR image super-resolution via channel splitting and serial fusion network. Knowl Based Syst 246:108669. https://doi.org/10.1016/j.knosys.2022.108669

    Article  Google Scholar 

  13. Wang H, Hu X, Zhao X, Zhang Y (2022) Wide weighted attention multi-scale network for accurate MR image super-resolution. IEEE Trans Circuits Syst Video Technol 32(3):962–975. https://doi.org/10.1109/TCSVT.2021.3070489

    Article  Google Scholar 

  14. Li G, Lyu J, Wang C, Dou Q, Qin J (2022) Wavtrans: synergizing wavelet and cross-attention transformer for multi-contrast MRI super-resolution. In: Medical Image computing and computer assisted intervention: MICCAI 2022, vol. 13436, pp 463–473. https://doi.org/10.1007/978-3-031-16446-0_44

  15. Yang G, Zhang L, Zhou M, Liu A, Chen X, Xiong Z, Wu F (2022) Model-guided multi-contrast deep unfolding network for MRI super-resolution reconstruction. In: MM ’22: the 30th ACM international conference on multimedia, pp 3974–3982. https://doi.org/10.1145/3503161.3548068

  16. Wang R, Zhao R, Fu W, Zhang X, Zhang Y, Feng R (2022) Multi-contrast high quality MR image super-resolution with dual domain knowledge fusion. In: IEEE international conference on bioinformatics and biomedicine, BIBM 2022, pp 2127–2134. https://doi.org/10.1109/BIBM55620.2022.9995219

  17. Li G, Lv J, Tian Y, Dou Q, Wang C, Xu C, Qin J (2022) Transformer-empowered multi-scale contextual matching and aggregation for multi-contrast MRI super-resolution. In: IEEE conference on computer vision and pattern recognition, CVPR 2022, pp 20604–20613. https://doi.org/10.1109/CVPR52688.2022.01998

  18. Fang C, Zhang D, Wang L, Zhang Y, Cheng L, Han J (2022) Cross-modality high-frequency transformer for MR image super-resolution. In: MM ’22: the 30th ACM international conference on multimedia, pp 1584–1592. https://doi.org/10.1145/3503161.3547804

  19. Kang L, Liu G, Huang J, Li J (2022) Super-resolution method for MR images based on multi-resolution CNN. Biomed Signal Process Control 72(Part):103372. https://doi.org/10.1016/j.bspc.2021.103372

    Article  Google Scholar 

  20. Li Y, Iwamoto Y, Lin L, Xu R, Tong R, Chen Y (2021) Volumenet: a lightweight parallel network for super-resolution of MR and CT volumetric data. IEEE Trans Image Process 30:4840–4854. https://doi.org/10.1109/TIP.2021.3076285

    Article  MathSciNet  Google Scholar 

  21. Feng C, Wang K, Lu S, Xu Y, Li X (2021) Brain MRI super-resolution using coupled-projection residual network. Neurocomputing 456:190–199. https://doi.org/10.1016/j.neucom.2021.01.130

    Article  Google Scholar 

  22. Qiu D, Cheng Y, Wang X (2021) radual back-projection residual attention network for magnetic resonance image super-resolution. Comput Methods Programs Biomed 208:106252. https://doi.org/10.1016/j.cmpb.2021.106252

    Article  Google Scholar 

  23. Jiang M, Zhi M, Wei L, Yang X, Zhang J, Li Y, Wang P, Huang J, Yang G (2021) A-GAN: fused attentive generative adversarial networks for MRI image super-resolution. Comput Med Imaging Graph 92:101969. https://doi.org/10.1016/j.compmedimag.2021.101969

    Article  Google Scholar 

  24. Zhang Y, Li K, Li K, Fu Y (2021) R image super-resolution with squeeze and excitation reasoning attention network. In: IEEE conference on computer vision and pattern recognition, CVPR 2021, pp 13425–13434. https://doi.org/10.1109/CVPR46437.2021.01322

  25. Zeng K, Zheng H, Cai C, Yang Y, Zhang K, Chen Z (2018) imultaneous single- and multi-contrast super-resolution for brain MRI images based on a convolutional neural network. Comput Biol Med 99:133–141. https://doi.org/10.1016/j.compbiomed.2018.06.010

    Article  Google Scholar 

  26. Lyu Q, Shan H, Steber C, Helis C, Whitlow C, Chan M, Wang G (2020) Multi-contrast super-resolution MRI through a progressive network. IEEE Trans Med Imaging 39(9):2738–2749. https://doi.org/10.1109/TMI.2020.2974858

    Article  Google Scholar 

  27. Feng C, Fu H, Yuan S, Xu Y (2021) Multi-contrast MRI super-resolution via a multi-stage integration network. In: Medical image computing and computer assisted intervention—MICCAI 12906, pp 140–149. https://doi.org/10.1007/978-3-030-87231-1_14

  28. Tsiligianni E, Zerva M, Marivani I, Deligiannis N, Kondi LP (2021) Interpretable deep learning for multimodal super-resolution of medical images. In: Medical image computing and computer assisted intervention: MICCAI 12906, pp 421–429. https://doi.org/10.1007/978-3-030-87231-1_41

  29. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Computer vision: ECCV 2014—13th European conference, vol. 8692, pp 184–199. https://doi.org/10.1007/978-3-319-10593-2_13

  30. Thurnhofer-Hemsi K, López-Rubio E, Domínguez E, Baena RML, Roé-Vellvé N (2020) Deep learning-based super-resolution of 3d magnetic resonance images by regularly spaced shifting. Neurocomputing 398:314–327. https://doi.org/10.1016/j.neucom.2019.05.107

    Article  Google Scholar 

  31. Qiu D, Zhang S, Liu Y, Zhu J, Zheng L (2020) Super-resolution reconstruction of knee magnetic resonance imaging based on deep learning. Comput Methods Programs Biomed 187:105059. https://doi.org/10.1016/j.cmpb.2019.105059

    Article  Google Scholar 

  32. Zhao C, Dewey BE, Pham DL, Calabresi PA, Reich DS, Prince JL (2021) SMORE: a self-supervised anti-aliasing and super-resolution algorithm for MRI using deep learning. IEEE Trans Med Imaging 40(3):805–817. https://doi.org/10.1109/TMI.2020.3037187

    Article  Google Scholar 

  33. Zhang W, Wang L, Chen W, Jia Y, He Z, Du J (2022) 3d cross-scale feature transformer network for brain MR image super-resolution. In: IEEE international conference on acoustics, speech and signal processing, ICASSP 2022, pp 1356–1360. https://doi.org/10.1109/ICASSP43922.2022.9746092

  34. Shi J, Liu Q, Wang C, Zhang Q, Ying S, Xu H (2018) Super-resolution reconstruction of MR image with a novel residual learning network algorithm. Phys Med Biol 63(8):085011. https://doi.org/10.1088/1361-6560/aab9e9

    Article  Google Scholar 

  35. Zhu D, He H, Wang D (2023) Feedback attention network for cardiac magnetic resonance imaging super-resolution. Comput Methods Programs Biomed 231:107313. https://doi.org/10.1016/j.cmpb.2022.107313

    Article  Google Scholar 

  36. Qiu D, Cheng Y, Wang X (2023) Progressive feedback residual attention network for cardiac magnetic resonance imaging super-resolution. IEEE J Biomed Health Inform 27(7):3478–3488. https://doi.org/10.1109/JBHI.2023.3272155

    Article  Google Scholar 

  37. Chen Y, Shi F, Christodoulou AG, Xie Y, Zhou Z, Li D (2018) Efficient and accurate MRI super-resolution using a generative adversarial network and 3d multi-level densely connected network. In: Medical image computing and computer assisted intervention—MICCAI 2018 11070, pp 91–99. https://doi.org/10.1007/978-3-030-00928-1_11

  38. Chen L, Yang X, Jeon G, Anisetti M, Liu K (2020) A trusted medical image super-resolution method based on feedback adaptive weighted dense network. Artif Intell Med 106:101857. https://doi.org/10.1016/j.artmed.2020.101857

    Article  Google Scholar 

  39. Mahapatra D, Bozorgtabar B, Garnavi R (2019) Image super-resolution using progressive generative adversarial networks for medical image analysis. Comput Med Imaging Graph 71:30–39. https://doi.org/10.1016/j.compmedimag.2018.10.005

    Article  Google Scholar 

  40. Zhao M, Liu X, Liu H, Wong KKL (2020) Super-resolution of cardiac magnetic resonance images using Laplacian pyramid based on generative adversarial networks. Comput Med Imaging Graph 80:101698. https://doi.org/10.1016/j.compmedimag.2020.101698

    Article  Google Scholar 

  41. Lyu Q, Shan H, Wang G (2020) MRI super-resolution with ensemble learning and priors. IEEE Trans Comput Imaging 6:615–624. https://doi.org/10.1109/TCI.2020.2964201

    Article  Google Scholar 

  42. Guerreiro J, Tomás P, Garcia NC, Aidos H (2023) Super-resolution of magnetic resonance images using generative adversarial networks. Comput Med Imaging Graph 108:102280. https://doi.org/10.1016/j.compmedimag.2023.102280

    Article  Google Scholar 

  43. Wang L, Du J, Zhu H, He Z, Jia Y (2020) Brain MR image super-resolution using 3d feature attention network. In: IEEE international conference on bioinformatics and biomedicine, BIBM 2020, pp 1151–1155. https://doi.org/10.1109/BIBM49941.2020.9313377

  44. Wang L, Zhu H, He Z, Jia Y, Du J (2022) Adjacent slices feature transformer network for single anisotropic 3d brain MRI image super-resolution. Biomed Signal Process Control 72(Part):103339. https://doi.org/10.1016/j.bspc.2021.103339

    Article  Google Scholar 

  45. Hu X, Wang H, Cai Y, Zhao X, Zhang Y (2021) Pyramid orthogonal attention network based on dual self-similarity for accurate mr image super-resolution. In: 2021 IEEE international conference on multimedia and expo, ICME, pp 1–6. https://doi.org/10.1109/ICME51207.2021.9428112

  46. Forigua C, Escobar M, Arbelaez P (2022) Superformer: volumetric transformer architectures for MRI super-resolution. In: Simulation and synthesis in medical imaging—7th international workshop, SASHIMI 2022, vol. 13570, pp 132–141. https://doi.org/10.1007/978-3-031-16980-9_13

  47. Huang S, Liu X, Tan T, Hu M, Wei X, Chen T, Sheng B (2023) Transmrsr: transformer-based self-distilled generative prior for brain MRI super-resolution. Vis Comput 39(8):3647–3659. https://doi.org/10.1007/s00371-023-02938-3

    Article  Google Scholar 

  48. Wang J, Levman J, Pinaya WHL, Tudosiu P, Cardoso MJ, Marinescu RV (2023) Inversesr: 3d brain MRI super-resolution using a latent diffusion model. In: Medical image computing and computer assisted intervention—MICCAI 2023 14229, pp 438–447. https://doi.org/10.1007/978-3-031-43999-5_42

  49. Chung H, Lee ES, Ye JC (2023) MR image denoising and super-resolution using regularized reverse diffusion. IEEE Trans Med Imaging 42(4):922–934. https://doi.org/10.1109/TMI.2022.3220681

    Article  Google Scholar 

  50. Wu Z, Chen X, Xie S, Shen J, Zeng Y (2023) Super-resolution of brain MRI images based on denoising diffusion probabilistic model. Biomed Signal Process Control 85:104901. https://doi.org/10.1016/j.bspc.2023.104901

    Article  Google Scholar 

  51. Chen W, Zhao J, Wen Y, Xie B, Zhou X, Guo L, Yang L, Wang J, Dai Y, Zhou D (2015) Accuracy of 3-t MRI using susceptibility-weighted imaging to detect meniscal tears of the knee. Knee Surg Sports Traumatol Arthrosc 23:198–204. https://doi.org/10.1007/s00167-014-3035-0

    Article  Google Scholar 

  52. Li Y, Sixou B, Peyrin F (2021) A review of the deep learning methods for medical images super resolution problems. Irbm 42(2):120–133. https://doi.org/10.1016/j.irbm.2020.08.004

    Article  Google Scholar 

  53. Shi J, Li Z, Ying S, Wang C, Liu Q, Zhang Q, Yan P (2019) MR image super-resolution via wide residual networks with fixed skip connection. IEEE J Biomed Health Inform 23(3):1129–1140. https://doi.org/10.1109/JBHI.2018.2843819

    Article  Google Scholar 

  54. Cherukuri V, Guo T, Schiff SJ, Monga V (2020) Deep MR brain image super-resolution using spatio-structural priors. IEEE Trans Image Process 29:1368–1383. https://doi.org/10.1109/TIP.2019.2942510

    Article  MathSciNet  Google Scholar 

  55. Delannoy Q, Pham C, Cazorla C, Tor-Díez C, Dollé G, Meunier H, Bednarek N, Fablet R, Passat N, Rousseau F (2020) Segsrgan: super-resolution and segmentation using generative adversarial networks—application to neonatal brain MRI. Comput Biol Med 120:103755. https://doi.org/10.1016/j.compbiomed.2020.103755

    Article  Google Scholar 

  56. Xue X, Wang Y, Li J, Jiao Z, Ren Z, Gao X (2020) Progressive sub-band residual-learning network for MR image super resolution. IEEE J Biomed Health Inform 24(2):377–386. https://doi.org/10.1109/JBHI.2019.2945373

    Article  Google Scholar 

  57. Huang S, Li J, Mei L, Zhang T, Chen Z, Dong Y, Dong L, Liu S, Lyu M (2023) Accurate multi-contrast MRI super-resolution via a dual cross-attention transformer network. In: Medical image computing and computer assisted intervention—MICCAI 2023 14229, pp 313–322. https://doi.org/10.1007/978-3-031-43999-5_30

  58. Zhang J, Chi Y, Lyu J, Yang W, Tian Y (2023) Dual arbitrary scale super-resolution for multi-contrast MRI. In: Medical image computing and computer assisted intervention—MICCAI 2023, vol. 14229, pp 282–292. https://doi.org/10.1007/978-3-031-43999-5_27

  59. Zou B, Ji Z, Zhu C, Dai Y, Zhang W, Kui X (2023) Multi-scale deformable transformer for multi-contrast knee MRI super-resolution. Biomed Signal Process Control 79(Part):104154. https://doi.org/10.1016/j.bspc.2022.104154

    Article  Google Scholar 

  60. Zhao X, Zhang Y, Zhang T, Zou X (2019) Channel splitting network for single MR image super-resolution. IEEE Trans Image Process 28(11):5649–5662. https://doi.org/10.1109/TIP.2019.2921882

    Article  MathSciNet  Google Scholar 

  61. Lei P, Fang F, Zhang G, Xu M (2023) Deep unfolding convolutional dictionary model for multi-contrast MRI super-resolution and reconstruction. In: Proceedings of the thirty-second international joint conference on artificial intelligence, IJCAI 2023, pp 1008–1016. https://doi.org/10.24963/ijcai.2023/112

  62. Lei P, Fang F, Zhang G, Zeng T (2023) Decomposition-based variational network for multi-contrast MRI super-resolution and reconstruction. In: Proceedings of the IEEE international conference on computer vision, pp 21296–21306

  63. Mao Y, Jiang L, Chen X, Li C (2023) Disc-diff: Disentangled conditional diffusion model for multi-contrast MRI super-resolution. In: Medical image computing and computer assisted intervention—MICCAI 2023 14229, pp 387–397. https://doi.org/10.1007/978-3-031-43999-5_37

  64. Pham C, Tor-Díez C, Meunier H, Bednarek N, Fablet R, Passat N, Rousseau F (2019) Multiscale brain MRI super-resolution using deep 3d convolutional networks. Comput Med Imaging Graph. https://doi.org/10.1016/j.compmedimag.2019.101647

    Article  Google Scholar 

  65. Iwamoto Y, Takeda K, Li Y, Shiino A, Chen Y (2023) Unsupervised MRI super resolution using deep external learning and guided residual dense network with multimodal image priors. IEEE Trans Emerg Top Comput Intell 7(2):426–435. https://doi.org/10.1109/TETCI.2022.3215137

    Article  Google Scholar 

  66. Wu Q, Li Y, Sun Y, Zhou Y, Wei H, Yu J, Zhang Y (2023) An arbitrary scale super-resolution approach for 3d MR images via implicit neural representation. IEEE J Biomed Health Inform 27(2):1004–1015. https://doi.org/10.1109/JBHI.2022.3223106

    Article  Google Scholar 

  67. Zhou H, Huang Y, Li Y, Zhou Y, Zheng Y (2023) Blind super-resolution of 3d MRI via unsupervised domain transformation. IEEE J Biomed Health Inform 27(3):1409–1418. https://doi.org/10.1109/JBHI.2022.3232511

    Article  Google Scholar 

  68. Du J, He Z, Wang L, Gholipour A, Zhou Z, Chen D, Jia Y (2020) Super-resolution reconstruction of single anisotropic 3d MR images using residual convolutional neural network. Neurocomputing 392:209–220. https://doi.org/10.1016/j.neucom.2018.10.102

    Article  Google Scholar 

  69. https://fastmri.med.nyu.edu/

  70. Zbontar J, Knoll F, Sriram A, Muckley MJ, Bruno M, Defazio A, Parente M, Geras KJ, Katsnelson J, Chandarana H, Zhang Z, Drozdzal M, Romero A, Rabbat MG, Vincent P, Pinkerton J, Wang D, Yakubova N, Owens E, Zitnick CL, Recht MP, Sodickson DK, Lui YW (2018) fastmri: an open dataset and benchmarks for accelerated MRI. CoRR abs/1811.08839

  71. https://brain-development.org/ixi-dataset/

  72. http://hdl.handle.net/1926/1687

  73. https://db.humanconnectome.org/

  74. http://www.bic.mni.mcgill.ca/brainweb/

  75. Cocosco CA, Kollokian V, Kwan RK-S, Evans AC (1997) Brainweb: online interface to a 3d MRI simulated brain database. NeuroImage

  76. ...Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby JS, Burren Y, Porz N, Slotboom J, Wiest R, Lanczi L, Gerstner ER, Weber M, Arbel T, Avants BB, Ayache N, Buendia P, Collins DL, Cordier N, Corso JJ, Criminisi A, Das T, Delingette H, Demiralp Ç, Durst CR, Dojat M, Doyle S, Festa J, Forbes F, Geremia E, Glocker B, Golland P, Guo X, Hamamci A, Iftekharuddin KM, Jena R, John NM, Konukoglu E, Lashkari D, Mariz JA, Meier R, Pereira S, Precup D, Price SJ, Raviv TR, Reza SMS, Ryan MT, Sarikaya D, Schwartz LH, Shin H, Shotton J, Silva CA, Sousa N, Subbanna NK, Székely G, Taylor TJ, Thomas OM, Tustison NJ, Ünal GB, Vasseur F, Wintermark M, Ye DH, Zhao L, Zhao B, Zikic D, Prastawa M, Reyes M, Leemput KV (2015) The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging 34(10):1993–2024. https://doi.org/10.1109/TMI.2014.2377694

    Article  Google Scholar 

  77. http://www.developingconnectome.org

  78. Hughes E, Cordero-Grande L, Murgasova M, Hutter J, Price A, Gomes ADS, Allsop J, Steinweg J, Tusor N, Wurie J, et al (2017) The developing human connectome: announcing the first release of open access neonatal brain imaging. Organ HumBrain Mapp:25–29

  79. http://www.cma.mgh.harvard.edu/ibsr/

  80. Worth A (2010) MGH CMA internet brain segmentation repository (IBSR)

  81. https://github.com/npnl/ATLAS/

  82. Ahn N, Kang B, Sohn K (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Computer vision: ECCV 2018—15th European conference, vol. 11214, pp 256–272. https://doi.org/10.1007/978-3-030-01249-6_16

  83. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE conference on computer vision and pattern recognition workshops, CVPR workshops, pp 1132–1140. https://doi.org/10.1109/CVPRW.2017.151

  84. Zhao H, Gallo O, Frosio I, Kautz J (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57. https://doi.org/10.1109/TCI.2016.2644865

    Article  Google Scholar 

  85. Lai W, Huang J, Ahuja N, Yang M (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: IEEE conference on computer vision and pattern recognition, CVPR 2017, pp 5835–5843. https://doi.org/10.1109/CVPR.2017.618

  86. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: Computer vision: ECCV 2016—14th European conference, vol. 9906, pp 694–711 . https://doi.org/10.1007/978-3-319-46475-6_43

  87. Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In: IEEE conference on computer vision and pattern recognition, CVPR 2016, pp 2414–2423. https://doi.org/10.1109/CVPR.2016.265

  88. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken AP, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE conference on computer vision and pattern recognition, CVPR 2017, pp 105–114. https://doi.org/10.1109/CVPR.2017.19

  89. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  90. Sheikh HR, Bovik AC, Veciana G (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans Image Process 14(12):2117–2128. https://doi.org/10.1109/TIP.2005.859389

    Article  Google Scholar 

  91. Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, pp 6626–6637. https://proceedings.neurips.cc/paper/2017/hash/8a1d694707eb0fefe65871369074926d-Abstract.html

  92. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: IEEE conference on computer vision and pattern recognition, CVPR 2016, pp 2818–2826. https://doi.org/10.1109/CVPR.2016.308

  93. Rodrigues L, Rezende TJR, Wertheimer G, Santos Y, França M, Rittner L (2022) A benchmark for hypothalamus segmentation on t1-weighted MR images. NeuroImage 264:119741. https://doi.org/10.1016/j.neuroimage.2022.119741

    Article  Google Scholar 

  94. Zhuang Y, Liu H, Song E, Ma G, Xu X, Hung C (2022) Aprnet: a 3d anisotropic pyramidal reversible network with multi-modal cross-dimension attention for brain tissue segmentation in MR images. IEEE J Biomed Health Inform 26(2):749–761. https://doi.org/10.1109/JBHI.2021.3093932

    Article  Google Scholar 

  95. Dice LR (1945) Measures of the amount of ecologic association between species. Ecology 26(3):297–302. https://doi.org/10.2307/1932409

    Article  Google Scholar 

  96. https://iseg2017.web.unc.edu/

  97. Kempton MJ, Underwood TSA, Brunton S, Stylios F, Schmechtig A, Ettinger U, Smith MS, Lovestone S, Crum WR, Frangou S, Williams SCR, Simmons A (2011) A comprehensive testing protocol for MRI neuroanatomical segmentation techniques: evaluation of a novel lateral ventricle segmentation method. NeuroImage 58(4):1051–1059. https://doi.org/10.1016/j.neuroimage.2011.06.080

    Article  Google Scholar 

  98. https://www.smir.ch/BRATS/Start2015

  99. ...Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby JS, Burren Y, Porz N, Slotboom J, Wiest R, Lanczi L, Gerstner ER, Weber M, Arbel T, Avants BB, Ayache N, Buendia P, Collins DL, Cordier N, Corso JJ, Criminisi A, Das T, Delingette H, Demiralp Ç, Durst CR, Dojat M, Doyle S, Festa J, Forbes F, Geremia E, Glocker B, Golland P, Guo X, Hamamci A, Iftekharuddin KM, Jena R, John NM, Konukoglu E, Lashkari D, Mariz JA, Meier R, Pereira S, Precup D, Price SJ, Raviv TR, Reza SMS, Ryan MT, Sarikaya D, Schwartz LH, Shin H, Shotton J, Silva CA, Sousa NJ, Subbanna NK, Székely G, Taylor TJ, Thomas OM, Tustison NJ, Ünal GB, Vasseur F, Wintermark M, Ye DH, Zhao L, Zhao B, Zikic D, Prastawa M, Reyes M, Leemput KV (2015) The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging 34(10):1993–2024. https://doi.org/10.1109/TMI.2014.2377694

    Article  Google Scholar 

  100. Radau P, Lu Y, Connelly K, Paul G, Dick A, Wright G (2009) Evaluation framework for algorithms segmenting short axis cardiac MRI. MIDAS J-Card MR Left Ventricle Segmentation Chall 49:4. https://doi.org/10.54294/g80ruo

    Article  Google Scholar 

  101. Landman BA, Huang AJ, Gifford A, Vikram DS, Lim IAL, Farrell JAD, Bogovic JA, Hua J, Chen M, Jarso S, Smith SA, Joel S, Mori S, Pekar JJ, Barker PB, Prince JL, Zijl PCM (2011) Multi-parametric neuroimaging reproducibility: a 3-t resource study. NeuroImage 54(4):2854–2866. https://doi.org/10.1016/j.neuroimage.2010.11.047

    Article  Google Scholar 

  102. http://epipage2.inserm.fr

  103. Ancel P-Y, Goffinet F (2014) Epipage 2: a preterm birth cohort in France in 2011. BMC Pediatr 14(1):1–8. https://doi.org/10.1186/1471-2431-14-97

    Article  Google Scholar 

  104. Huang J, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, pp 5197–5206. https://doi.org/10.1109/CVPR.2015.7299156

  105. Marcus DS, Wang TH, Parker J, Csernansky JG, Morris JC, Buckner RL (2007) Open access series of imaging studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J Cogn Neurosci 19(9):1498–1507. https://doi.org/10.1162/jocn.2007.19.9.1498

    Article  Google Scholar 

  106. https://mrbrains18.isi.uu.nl/

  107. http://www.sdspeople.fudan.edu.cn/zhuangxiahai/0/mmwhs/

  108. ...Bernard O, Lalande A, Zotti C, Cervenansky F, Yang X, Heng P, Cetin I, Lekadir K, Camara O, Ballester MÁG, Sanroma G, Napel S, Petersen SE, Tziritas G, Grinias E, Khened M, Varghese A, Krishnamurthi G, Rohé M, Pennec X, Sermesant M, Isensee F, Jaeger P, Maier-Hein KH, Full PM, Wolf I, Engelhardt S, Baumgartner CF, Koch LM, Wolterink JM, Isgum I, Jang Y, Hong Y, Patravali J, Jain S, Humbert O, Jodoin P (2018) Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: Is the problem solved? IEEE Trans Med Imaging 37(11):2514–2525. https://doi.org/10.1109/TMI.2018.2837502

    Article  Google Scholar 

  109. http://www.cse.yorku.ca/~mridataset/

  110. https://digital-heart.org/

  111. http://www.cardiacatlas.org/studies/ amrg- cardiac- atlas/

  112. Snoek L, Miesen MM, Beemsterboer T, Van Der Leij A, Eigenhuis A, Steven Scholte H (2021) The Amsterdam open MRI collection, a set of multimodal MRI datasets for individual difference analyses. Sci Data 8(1):85. https://doi.org/10.1038/s41597-021-00870-6

    Article  Google Scholar 

  113. Li H, Xuan Z, Zhou J, Hu X, Yang B (2023) Fast and accurate super-resolution of MR images based on lightweight generative adversarial network. Multim Tools Appl 82(2):2465–2487. https://doi.org/10.1007/s11042-022-13326-9

    Article  Google Scholar 

  114. Feichtenhofer C, Fassold H, Schallauer P (2013) A perceptual image sharpness metric based on local edge gradient analysis. IEEE Signal Process Lett 20(4):379–382. https://doi.org/10.1109/LSP.2013.2248711

    Article  Google Scholar 

  115. Hassen R, Wang Z, Salama MMA (2013) Image sharpness assessment based on local phase coherence. IEEE Trans Image Process 22(7):2798–2810. https://doi.org/10.1109/TIP.2013.2251643

    Article  Google Scholar 

  116. Shi F, Cheng J, Wang L, Yap P-T, Shen D (2015) Lrtv: Mr image super-resolution with low-rank and total variation regularizations. IEEE Trans Med Imaging 34(12):2459–2466. https://doi.org/10.1109/TMI.2015.2437894

    Article  Google Scholar 

  117. He J, Liu Q, Christodoulou AG, Ma C, Lam F, Liang Z (2016) Accelerated high-dimensional MR imaging with sparse sampling using low-rank tensors. IEEE Trans Med Imaging 35(9):2119–2129. https://doi.org/10.1109/TMI.2016.2550204

    Article  Google Scholar 

  118. Li J, Liu H, Pan J, Yao H (2018) Training samples-optimizing based dictionary learning algorithm for MR sparse superresolution reconstruction. Biomed Signal Process Control 39:177–184. https://doi.org/10.1016/j.bspc.2017.08.007

    Article  Google Scholar 

  119. Ravishankar S, Bresler Y (2011) MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans Med Imaging 30(5):1028–1041. https://doi.org/10.1109/TMI.2010.2090538

    Article  Google Scholar 

  120. Shi W, Caballero J, Huszar F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE conference on computer vision and pattern recognition, CVPR 2016, pp 1874–1883. https://doi.org/10.1109/CVPR.2016.207

  121. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition, CVPR 2016, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  122. Wang L, Du J, Gholipour A, He Z, Jia Y (2019) Brain MRI super-resolution reconstruction using a multi-level and parallel conv-deconv network. In: 2019 IEEE international conference on bioinformatics and biomedicine, BIBM, pp 885–891. https://doi.org/10.1109/BIBM47256.2019.8983233

  123. Huang G, Liu Z, Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE conference on computer vision and pattern recognition, CVPR 2017, pp 2261–2269. https://doi.org/10.1109/CVPR.2017.243

  124. Du J, Wang L, Gholipour A, He Z, Jia Y (2018) Accelerated super-resolution MR image reconstruction via a 3d densely connected deep convolutional neural network. In: IEEE international conference on bioinformatics and biomedicine, BIBM 2018, pp 349–355. https://doi.org/10.1109/BIBM.2018.8621073

  125. He W, Hu Y, Wang L, He Z, Du J (2021) Gating feature dense network for single anisotropic MR image super-resolution. In: IEEE international conference on acoustics, speech and signal processing, ICASSP 2021, pp 1670–1674. https://doi.org/10.1109/ICASSP39728.2021.9414646

  126. Liu J, Chen F, Wang X, Liao H (2019) An edge enhanced SRGAN for MRI super resolution in slice-selection direction. In: Multimodal brain image analysis and mathematical foundations of computational anatomy—4th international workshop, MBIA 2019, vol. 11846, pp 12–20. https://doi.org/10.1007/978-3-030-33226-6_2

  127. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville AC, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems 27: annual conference on neural information processing systems 2014, pp 2672–2680. https://proceedings.neurips.cc/paper/2014/hash/5ca3e9b122f61f8f06494c97b1afccf3-Abstract.html

  128. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN. Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in Neural information processing systems 30: annual conference on neural information processing systems 2017, pp 5998–6008. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html

  129. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N (2021) An image is worth 16x16 words: Transformers for image recognition at scale. In: 9th international conference on learning representations, ICLR. https://openreview.net/forum?id=YicbFdNTTy

  130. Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: 2021 IEEE international conference on computer vision, ICCV, pp 9992–10002. https://doi.org/10.1109/ICCV48922.2021.00986

  131. Sohl-Dickstein J, Weiss EA, Maheswaranathan N, Ganguli S (2015) Deep unsupervised learning using nonequilibrium thermodynamics. In: Proceedings of the 32nd international conference on machine learning, ICML 2015, vol. 37, pp 2256–2265. http://proceedings.mlr.press/v37/sohl-dickstein15.html

  132. Aubert-Broche B, Griffin M, Pike GB, Evans AC, Collins DL (2006) Twenty new digital brain phantoms for creation of validation image data bases. IEEE Trans Med Imaging 25(11):1410–1416. https://doi.org/10.1109/TMI.2006.883453

    Article  Google Scholar 

  133. Lesjak Z, Galimzianova A, Koren A, Lukin M, Pernus F, Likar B, Spiclin Z (2018) A novel public MR image dataset of multiple sclerosis patients with lesion segmentations based on multi-rater consensus. Neuroinformatics 16(1):51–63. https://doi.org/10.1007/s12021-017-9348-7

    Article  Google Scholar 

  134. https://www.med.upenn.edu/cbica/brats-2019/

  135. https://www.med.upenn.edu/cbica/brats2020/data.html

  136. https://www.med.upenn.edu/sbia/brats2018/data.html

  137. McGinnis J, Shit S, Li HB, Sideri-Lampretsa V, Graf R, Dannecker M, Pan J, Ansó NS, Mühlau M, Kirschke JS, Ruecker, D, Wiestler B (2023) Single-subject multi-contrast MRI super-resolution via implicit neural representations. In: Medical image computing and computer assisted intervention—MICCAI 2023, vol. 14227, pp 173–183. https://doi.org/10.1007/978-3-031-43993-3_17

  138. Commowick O, Cervenansky F, Ameli R (2016) Msseg challenge proceedings: Multiple sclerosis lesions segmentation challenge using a data management and processing infrastructure. In: Medical image computing and computer assisted intervention—MICCAI . https://api.semanticscholar.org/CorpusID:51996766

  139. Lyu M, Mei L, Huang S, Liu S, Li Y, Yang K, Liu Y, Dong Y, Dong L, Wu EX (2023) M4raw: a multi-contrast, multi-repetition, multi-channel MRI k-space dataset for low-field MRI research. Sci Data 10(1):264. https://doi.org/10.1038/s41597-023-02181-4

    Article  Google Scholar 

  140. Akansu AN, Haddad RA (2001) Multiresolution signal decomposition: transforms, subbands, and wavelets

  141. Corona V, Avilés-Rivero AI, Debroux N, Guyader CL, Schönlieb C (2021) Variational multi-task MRI reconstruction: joint reconstruction, registration and super-resolution. Med Image Anal 68:101941. https://doi.org/10.1016/j.media.2020.101941

    Article  Google Scholar 

  142. Ebner M, Wang G, Li W, Aertsen M, Patel PA, Aughwane R, Melbourne A, Doel T, Dymarkowski S, Coppi PD, David AL, Deprest J, Ourselin S, Vercauteren T (2020) An automated framework for localization, segmentation and super-resolution reconstruction of fetal brain MRI. NeuroImage. https://doi.org/10.1016/j.neuroimage.2019.116324

    Article  Google Scholar 

  143. Feng C, Yan Y, Fu H, Chen L, Xu Y (2021) Task transformer network for joint MRI reconstruction and super-resolution. In: Medical image computing and computer assisted intervention—MICCAI 2021, vol. 12906, pp 307–317 . https://doi.org/10.1007/978-3-030-87231-1_30

  144. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE conference on computer vision and pattern recognition, CVPR 2016, pp 1646–1654. https://doi.org/10.1109/CVPR.2016.182

  145. Zhang S, Liang G, Pan S, Zheng L (2019) A fast medical image super resolution method based on deep learning network. IEEE Access 7:12319–12327. https://doi.org/10.1109/ACCESS.2018.2871626

    Article  Google Scholar 

  146. Feng C, Yan Y, Liu C, Fu H, Xu Y, Shao L (2021) Exploring separable attention for multi-contrast MR image super-resolution. CoRR abs/2109.01664

  147. Sánchez I, Vilaplana V (2018) Brain MRI super-resolution using 3d generative adversarial networks. arXiv preprint arXiv:1812.11440

  148. Chitty-Venkata KT, Somani AK (2023) Neural architecture search survey: a hardware perspective. ACM Comput Surv 55(4):78–17836. https://doi.org/10.1145/3524500

    Article  Google Scholar 

  149. Sui Y, Afacan,O, Gholipour A, Warfield SK (2021) MRI super-resolution through generative degradation learning. In: Medical image computing and computer assisted intervention—MICCAI 2021, vol. 12906, pp 430–440. https://doi.org/10.1007/978-3-030-87231-1_42

  150. Xuan K, Xiang L, Huang X, Zhang L, Liao S, Shen D, Wang Q (2022) Multimodal MRI reconstruction assisted with spatial alignment network. IEEE Trans Med Imaging 41(9):2499–2509. https://doi.org/10.1109/TMI.2022.3164050

    Article  Google Scholar 

Download references

Funding

The work was supported by the National Key R &D Program of China (No. 2018AAA0102100); the National Natural Science Foundation of China (No. U22A2034, 62177047); the Key Research and Development Program of Hunan Province (No. 2022SK2054); Central South University Research Programme of Advanced Interdisciplinary Studies (No. 2023QYJC020).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Central South University, Changsha, 410083, China

    Zexin Ji, Beiji Zou, Xiaoyan Kui, Chengzhang Zhu, Peishan Dai & Yulan Dai

  2. Hunan Engineering Research Center of Machine Vision and Intelligent Medicine, Central South University, Changsha, 410083, China

    Zexin Ji, Beiji Zou, Xiaoyan Kui, Chengzhang Zhu, Peishan Dai & Yulan Dai

  3. Department of Radiology, The Second Xiangya Hospital, Central South University, Changsha, 410011, China

    Jun Liu & Wei Zhao

  4. The College of Literature and Journalism, Central South University, Changsha, 410083, China

    Chengzhang Zhu

Authors
  1. Zexin Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Beiji Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyan Kui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chengzhang Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Peishan Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yulan Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoyan Kui.

Ethics declarations

Conflict of interest

The authors have no Conflict of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, Z., Zou, B., Kui, X. et al. Deep learning-based magnetic resonance image super-resolution: a survey. Neural Comput & Applic 36, 12725–12752 (2024). https://doi.org/10.1007/s00521-024-09890-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09890-w

Keywords

Search

Navigation