A Multiscale Optimisation Algorithm for Shape and Material Reconstruction from a Single X-ray Image
Pages 252 - 259
Abstract
We produce thickness and bone to soft tissue ratio estimations from a single, 2D medical X-ray image. For this, we simulate the scattering of the rays through a model of the object and embed this simulation into an optimiser which iteratively adjusts the model to match the X-ray simulation to the observed X-ray image. Utilising a combination of different techniques, first, a CNN-based image segmentation serves as a regularisation term to the underlying cost function to guide the descent, while domain knowledge about physical parameter correlations is injected by additional penalty terms. Next, the optimiser is embedded into a multilevel framework which, similar to multi-grid’s philosophy, successively improves the model on varying resolutions while individual resolutions focus on particular terms of the cost function. Initial results suggest that we can obtain meaningful thickness and material estimations.
References
[1]
Jonas Adler and Ozan Öktem. 2017. Solving ill-posed inverse problems using iterative deep neural networks. Inverse Problems 33, 12 (2017), 124007.
[2]
Yatharth Ahuja, Tushar Nitharwal, Utkarsh Sundriyal, Sreedevi Indu, and Anup K Mandpura. 2022. Depth Reconstruction Based Visual SLAM Using ORB Feature Extraction. Journal of Image and Graphics 10, 4 (2022), 172–177.
[3]
Simon Arridge, Peter Maass, Ozan Öktem, and Carola-Bibiane Schönlieb. 2019. Solving inverse problems using data-driven models. Acta Numerica 28 (2019), 1–174.
[4]
Daniel Otero Baguer, Johannes Leuschner, and Maximilian Schmidt. 2020. Computed tomography reconstruction using deep image prior and learned reconstruction methods. Inverse Problems 36, 9 (2020), 094004.
[5]
Anatoly B Bakushinsky, Mihail Yu Kokurin, and Alexandra Smirnova. 2010. Iterative methods for ill-posed problems. de Gruyter.
[6]
Mohsin Bilal and Muhammad Arif. 2019. Multivariate Regression CNN for ill-posed Inverse Reconstruction of Satellite Images. Procedia Computer Science 163 (2019), 312–318. 16th Learning and Technology Conference 2019Artificial Intelligence and Machine Learning: Embedding the Intelligence.
[7]
Jasleen Birdi, Audrey Repetti, and Yves Wiaux. 2017. A regularized tri-linear approach for optical interferometric imaging. Monthly Notices of the Royal Astronomical Society 468, 1 (Feb 2017), 1142–1155.
[8]
Joseph Bullock, Carolina Cuesta-Lázaro, and Arnau Quera-Bofarull. 2019. XNet: a convolutional neural network (CNN) implementation for medical x-ray image segmentation suitable for small datasets. In Medical Imaging 2019: Biomedical Applications in Molecular, Structural, and Functional Imaging, Vol. 10953. SPIE, 453–463.
[9]
S. Delorme, Y. Petit, J. A. de Guise, H. Labelle, C. . Aubin, and J. Dansereau. 2003. Assessment of the 3-D reconstruction and high-resolution geometrical modeling of the human skeletal trunk from 2-D radiographic images. IEEE Transactions on Biomedical Engineering 50, 8 (2003), 989–998.
[10]
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2015. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence 38, 2 (2015), 295–307.
[11]
J. Allison et al.2006. Geant4 developments and applications. IEEE Transactions on Nuclear Science 53, 1 (2006), 270–278.
[12]
J. Allison et al.2016. Recent developments in Geant4. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 835 (2016), 186–225.
[13]
S. Agostinelli et al.2003. Geant4—a simulation toolkit. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 506, 3 (2003), 250–303.
[14]
Davis Gilton, Gregory Ongie, and Rebecca Willett. 2021. Model adaptation for inverse problems in imaging. IEEE Transactions on Computational Imaging 7 (2021), 661–674.
[15]
Yingying Gu, Jun Zhang, and Ping Xue. 2018. Multi-grid nonlocal techniques for x-ray scatter correction. In Medical Imaging 2018: Image Processing, Elsa D. Angelini and Bennett A. Landman (Eds.). Vol. 10574. International Society for Optics and Photonics, SPIE, 427 – 433.
[16]
Jingnan Guo, Saša Banjac, Lennart Röstel, Jan C Terasa, Konstantin Herbst, Bernd Heber, and Robert F Wimmer-Schweingruber. 2019. Implementation and validation of the GEANT4/AtRIS code to model the radiation environment at Mars. Journal of Space Weather and Space Climate 9 (2019), A2.
[17]
Normi Abdul Hadi, Arsmah Ibrahim, Fatimah Yahya, and Jamaludin Md Ali. 2013. Centroid based on branching contour matching for 3D reconstruction using Beta-spline. Journal of Image and Graphics 1, 3 (2013), 138–142.
[18]
Andreas Hauptmann, Jonas Adler, Simon Arridge, and Ozan Öktem. 2020. Multi-scale learned iterative reconstruction. IEEE transactions on computational imaging 6 (2020), 843–856.
[19]
Hideo Hirayama, Yoshihito Namito, Walter R Nelson, Alex F Bielajew, Scott J Wilderman, and U Michigan. 2005. The EGS5 code system. Technical Report. United States. Department of Energy.
[20]
Sebastien Incerti, Michael Douglass, Scott Penfold, Susanna Guatelli, and Eva Bezak. 2016. Review of Geant4-DNA applications for micro and nanoscale simulations. Physica Medica 32, 10 (2016), 1187–1200.
[21]
Prateek Jain, Purushottam Kar, 2017. Non-convex optimization for machine learning. Foundations and Trends® in Machine Learning 10, 3-4 (2017), 142–363.
[22]
Sergey I Kabanikhin. 2011. Inverse and ill-posed problems. de Gruyter.
[23]
George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. 2021. Physics-informed machine learning. Nature Reviews Physics 3, 6 (2021), 422–440.
[24]
Yoni Kasten, Daniel Doktofsky, and Ilya Kovler. 2020. End-to-end convolutional neural network for 3D reconstruction of knee bones from bi-planar X-ray images. In International Workshop on Machine Learning for Medical Image Reconstruction. Springer, 123–133.
[25]
Dari Kimanius, Gustav Zickert, Takanori Nakane, Jonas Adler, Sebastian Lunz, C-B Schönlieb, Ozan Öktem, and Sjors HW Scheres. 2021. Exploiting prior knowledge about biological macromolecules in cryo-EM structure determination. IUCrJ 8, 1 (2021), 60–75.
[26]
HN Latha and Rajiv R Sahay. 2020. Joint estimation of depth map and focus image in SFF: an optimization framework by sparsity approach. In 2020 International Conference on Communication and Signal Processing (ICCSP). IEEE, 0656–0660.
[27]
Mikhail M Lavrent’ev, Sergei M Zerkal, and Oleg E Trofimov. 2014. Computer modelling in tomography and ill-posed problems. De Gruyter.
[28]
Housen Li, Johannes Schwab, Stephan Antholzer, and Markus Haltmeier. 2020. NETT: Solving inverse problems with deep neural networks. Inverse Problems 36, 6 (2020), 065005.
[29]
Sebastian Lunz, Ozan Öktem, and Carola-Bibiane Schönlieb. 2019. Adversarial Regularizers in Inverse Problems. arxiv:1805.11572 [cs.CV]
[30]
Andreas K Maier, Christopher Syben, Bernhard Stimpel, Tobias Würfl, Mathis Hoffmann, Frank Schebesch, Weilin Fu, Leonid Mill, Lasse Kling, and Silke Christiansen. 2019. Learning with known operators reduces maximum error bounds. Nature machine intelligence 1, 8 (2019), 373–380.
[31]
Achille Mileto, Luis S Guimaraes, Cynthia H McCollough, Joel G Fletcher, and Lifeng Yu. 2019. State of the art in abdominal CT: the limits of iterative reconstruction algorithms. Radiology 293, 3 (2019), 491–503.
[32]
Francesc Salvat, José M Fernández-Varea, and Josep Sempau. 2006. PENELOPE-2006: A code system for Monte Carlo simulation of electron and photon transport. In PENELOPE workshop proceedings. OECD Publishing, 7.
[33]
Johannes Schwab, Stephan Antholzer, and Markus Haltmeier. 2019. Big in Japan: Regularizing networks for solving inverse problems. Journal of Mathematical Imaging and Vision 62, 3 (2019), 445–455.
[34]
Pedro M Silva and Joao B Florindo. 2019. Using down-sampling for multiscale analysis of texture images. Pattern Recognition Letters 125 (2019), 411–417.
[35]
Manoj Kumar Singh. 2013. Image reconstruction and edge detection based upon neural approximation characteristics. Journal of Image and Graphics 1, 1 (2013), 12–16.
[36]
Klaus Stüben. 2000. Multigrid. Academic Press, Chapter Appendix A, 413–532.
[37]
U Trottenberg. 2001. Multigrid. Academic Press, San Diego, Calif. ; London.
[38]
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2018. Deep image prior. In Proc. CVPR. IEEE, 9446–9454.
[39]
Tobias Würfl, Mathis Hoffmann, Vincent Christlein, Katharina Breininger, Yixin Huang, Mathias Unberath, and Andreas K Maier. 2018. Deep learning computed tomography: Learning projection-domain weights from image domain in limited angle problems. IEEE transactions on medical imaging 37, 6 (2018), 1454–1463.
[40]
Yibo Yang and Paris Perdikaris. 2018. Physics-informed deep generative models. arxiv:1812.03511 [stat.ML]
[41]
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. 2017. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising. IEEE Transactions on Image Processing 26, 7 (Jul 2017), 3142–3155.
Recommendations
A marker-based watershed method for X-ray image segmentation
Digital X-ray images are the most frequent modality for both screening and diagnosis in hospitals. To facilitate subsequent analysis such as quantification and computer aided diagnosis (CAD), it is desirable to exclude image background. A marker-based ...
3D-2D Registration Using X-Ray Simulation and CMA-ES
Applications of Evolutionary ComputationAbstractRadiographs of the hand are useful in diagnosing and staging diseases such as rheumatoid arthritis (RA) and other musculoskeletal diseases. Radiographs are projections of the 3D anatomy, with the useful information such as pose and pathology ...
2D/3D Shape Model Registration with X-ray Images for Patient-Specific Spine Geometry Reconstruction
Intelligent Robotics and ApplicationsAbstractThe use of preoperative CT and intraoperative fluoroscopic-guided surgical robotic assistance for spinal disease treatment has gained significant attention among surgeons. However, the intraoperative robotic systems lack guidance based on three-...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
January 2024
480 pages
ISBN:9798400716720
DOI:10.1145/3647649
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 03 May 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
ICIGP 2024
ICIGP 2024: 2024 the 7th International Conference on Image and Graphics Processing
January 19 - 21, 2024
Beijing, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 98Total Downloads
- Downloads (Last 12 months)98
- Downloads (Last 6 weeks)32
Reflects downloads up to 23 Jan 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in