An Attention-Based, Dual-Domain Adaptive Convolutional Neural Network for Fast MRI Reconstruction
Authors: 
Rui Hu, Shufan Chen, Jinxu TaoAuthors Info & Claims
Pages 242 - 249
Abstract
In recent years, deep learning has demonstrated great potential in accelerating Magnetic Resonance Imaging (MRI). However, most existing works simply consider MRI reconstruction as an image-to-image task, or typically use a single and identical network for both frequency and image domain reconstruction simultaneously, without taking into consideration the data characteristics and internal interactions of the two domains, which leads to ineffective learning of the image reconstruction models. In this study, we proposed an attention-based, dual-domain reconstruction network to facilitate fast and accurate MRI reconstruction by adaptively learning different data characteristics in the frequency domain and image domain. In response to the inhomogeneous energy distribution in K-space, we designed a densely connected multi-attention network for frequency domain reconstruction. For image aliasing at low sampling rates, we developed a residual encoder-decoder network with an attention module for the image domain reconstruction. The overall network contains two parallel and interactive branches that simultaneously perform on K-space and image data, with the data fusion modules to fuse dual domain information in each reconstruction layer, and the long skip connections to transfer inter-layer information. We have evaluated our method on two datasets. A series of comparative and ablation experiments demonstrate that our method effectively reconstructs images from undersampled K-space data, achieving better image reconstruction performance and restoring significant textures of brain slices.
References
[1]
Aaron Defazio, Tullie Murrell, and Michael Recht. 2020. MRI banding removal via adversarial training. Advances in Neural Information Processing Systems 33 (2020), 7660–7670.
[2]
David L Donoho. 2006. Compressed sensing. IEEE Transactions on information theory 52, 4 (2006), 1289–1306.
[3]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).
[4]
Tianming Du, Honggang Zhang, Yuemeng Li, Stephen Pickup, Mark Rosen, Rong Zhou, Hee Kwon Song, and Yong Fan. 2021. Adaptive convolutional neural networks for accelerating magnetic resonance imaging via k-space data interpolation. Medical image analysis 72 (2021), 102098.
[5]
Taejoon Eo, Yohan Jun, Taeseong Kim, Jinseong Jang, Ho-Joon Lee, and Dosik Hwang. 2018. KIKI-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magnetic resonance in medicine 80, 5 (2018), 2188–2201.
[6]
Chun-Mei Feng, Huazhu Fu, Shuhao Yuan, and Yong Xu. 2021. Multi-contrast mri super-resolution via a multi-stage integration network. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VI 24. Springer, 140–149.
[7]
Pengfei Guo, Yiqun Mei, Jinyuan Zhou, Shanshan Jiang, and Vishal M Patel. 2023. ReconFormer: Accelerated MRI reconstruction using recurrent transformer. IEEE Transactions on Medical Imaging (2023).
[8]
Jiahao Huang, Yingying Fang, Yinzhe Wu, Huanjun Wu, Zhifan Gao, Yang Li, Javier Del Ser, Jun Xia, and Guang Yang. 2022. Swin transformer for fast MRI. Neurocomputing 493 (2022), 281–304.
[9]
Jiahao Huang, Xiaodan Xing, Zhifan Gao, and Guang Yang. 2022. Swin deformable attention u-net transformer (sdaut) for explainable fast mri. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 538–548.
[10]
Qiaoying Huang, Dong Yang, Pengxiang Wu, Hui Qu, Jingru Yi, and Dimitris Metaxas. 2019. MRI reconstruction via cascaded channel-wise attention network. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 1622–1626.
[11]
Jialiang Jiang, Fulang Qi, Huiyu Du, Jianan Xu, Yufu Zhou, Dayong Gao, and Bensheng Qiu. 2022. Super-Resolution Reconstruction of 3T-Like Images From 0.35 T MRI Using a Hybrid Attention Residual Network. IEEE Access 10 (2022), 32810–32821.
[12]
Dongwook Lee, Jaejun Yoo, Sungho Tak, and Jong Chul Ye. 2018. Deep residual learning for accelerated MRI using magnitude and phase networks. IEEE Transactions on Biomedical Engineering 65, 9 (2018), 1985–1995.
[13]
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. 2021. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF international conference on computer vision. 1833–1844.
[14]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012–10022.
[15]
Morteza Mardani, Enhao Gong, Joseph Y Cheng, Shreyas S Vasanawala, Greg Zaharchuk, Lei Xing, and John M Pauly. 2018. Deep generative adversarial neural networks for compressive sensing MRI. IEEE transactions on medical imaging 38, 1 (2018), 167–179.
[16]
Ilkay Oksuz, James Clough, Aurelien Bustin, Gastao Cruz, Claudia Prieto, Rene Botnar, Daniel Rueckert, Julia A Schnabel, and Andrew P King. 2018. Cardiac MR motion artefact correction from k-space using deep learning-based reconstruction. In Machine Learning for Medical Image Reconstruction: First International Workshop, MLMIR 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings 1. Springer, 21–29.
[17]
Maosong Ran, Wenjun Xia, Yongqiang Huang, Zexin Lu, Peng Bao, Yan Liu, Huaiqiang Sun, Jiliu Zhou, and Yi Zhang. 2020. Md-recon-net: A parallel dual-domain convolutional neural network for compressed sensing mri. IEEE Transactions on Radiation and Plasma Medical Sciences 5, 1 (2020), 120–135.
[18]
Abhijit Guha Roy, Nassir Navab, and Christian Wachinger. 2018. Concurrent spatial and channel ‘squeeze & excitation’in fully convolutional networks. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part I. Springer, 421–429.
[19]
Jo Schlemper, Jose Caballero, Joseph V Hajnal, Anthony N Price, and Daniel Rueckert. 2017. A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE transactions on Medical Imaging 37, 2 (2017), 491–503.
[20]
Roy Shaul, Itamar David, Ohad Shitrit, and Tammy Riklin Raviv. 2020. Subsampled brain MRI reconstruction by generative adversarial neural networks. Medical Image Analysis 65 (2020), 101747.
[21]
Ohad Shitrit and Tammy Riklin Raviv. 2017. Accelerated magnetic resonance imaging by adversarial neural network. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: Third International Workshop, DLMIA 2017, and 7th International Workshop, ML-CDS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 14, Proceedings 3. Springer, 30–38.
[22]
Roberto Souza and Richard Frayne. 2019. A hybrid frequency-domain/image-domain deep network for magnetic resonance image reconstruction. In 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, 257–264.
[23]
Roberto Souza, R Marc Lebel, and Richard Frayne. 2019. A hybrid, dual domain, cascade of convolutional neural networks for magnetic resonance image reconstruction. In International Conference on Medical Imaging with Deep Learning. PMLR, 437–446.
[24]
Roberto Souza, Oeslle Lucena, Julia Garrafa, David Gobbi, Marina Saluzzi, Simone Appenzeller, Letícia Rittner, Richard Frayne, and Roberto Lotufo. 2018. An open, multi-vendor, multi-field-strength brain MR dataset and analysis of publicly available skull stripping methods agreement. NeuroImage 170 (2018), 482–494.
[25]
Liyan Sun, Zhiwen Fan, Yue Huang, Xinghao Ding, and John Paisley. 2018. Compressed sensing MRI using a recursive dilated network. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
[26]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[27]
Guang Yang, Simiao Yu, Hao Dong, Greg Slabaugh, Pier Luigi Dragotti, Xujiong Ye, Fangde Liu, Simon Arridge, Jennifer Keegan, Yike Guo, 2017. DAGAN: Deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE transactions on medical imaging 37, 6 (2017), 1310–1321.
[28]
Bo Zhou and S Kevin Zhou. 2020. DuDoRNet: learning a dual-domain recurrent network for fast MRI reconstruction with deep T1 prior. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4273–4282.
Index Terms
- An Attention-Based, Dual-Domain Adaptive Convolutional Neural Network for Fast MRI Reconstruction
Recommendations
Sparse-view cone beam CT reconstruction using dual CNNs in projection domain and image domain
AbstractCone beam computed tomography (CBCT) is used extensively in image-guided surgery and radiotherapy, but it induces ionizing radiation to the patients. Sparse-view CBCT is a main method to lower the radiation dose; however, it introduces ...
A dual-domain neural network based on sinogram synthesis for sparse-view CT reconstruction
Highlights- To suppress or remove the streak artifacts in parse-view computed tomography (CT) images, a convolutional neural network is employed to synthesize the full-...
Abstract ObjectiveThe dual-domain deep learning-based reconstruction techniques have enjoyed many successful applications in the field of medical image reconstruction. Applying the analytical reconstruction based operator to ...
Dual-domain attention-guided convolutional neural network for low-dose cone-beam computed tomography reconstruction
AbstractExcessive ionizing radiation in cone-beam computed tomography (CBCT) causes damage to patients, whereas a low radiation dose degrades the imaging quality. To improve the quality of low-dose CBCT images, deep-learning-based methods are ...
Highlights- A novel dual-domain deep learning framework for low-dose CT reconstruction.
- A ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2024
491 pages
ISBN:9798400717055
DOI:10.1145/3669754
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCAI 2024
ICCAI 2024: 2024 10th International Conference on Computing and Artificial Intelligence
April 26 - 29, 2024
Bali Island, Indonesia
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 11Total Downloads
- Downloads (Last 12 months)11
- Downloads (Last 6 weeks)7
Reflects downloads up to 20 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format