Abstract
The accurate estimation of X-ray source pose in relation to pre-operative images is crucial for minimally invasive procedures. However, existing deep learning-based automatic registration methods often have one or some limitations, including heavy reliance on subsequent conventional refinement steps, requiring manual annotation for training, or ignoring the patient’s anatomical specificity. To address these limitations, we propose a patient-specific and self-supervised end-to-end framework. Our approach utilizes patient’s preoperative CT to generate simulated X-rays that include patient-specific information. We propose a self-supervised regression neural network trained on the simulated patient-specific X-rays to predict six degrees of freedom pose of the X-ray source. In our proposed network, regularized autoencoder and multi-head self-attention mechanism are employed to encourage the model to automatically capture patient-specific salient information that supports accurate pose estimation, and Incremental Learning strategy is adopted for network training to avoid over-fitting and promote network performance. Meanwhile, an novel refinement model is proposed, which provides a way to obtain gradients with respect to the pose parameters to further refine the pose predicted by the regression network. Our method achieves a mean projection distance of 3.01 mm with a success rate of \(100\%\) on simulated X-rays, and a mean projection distance of 1.55 mm on X-rays. The code is available at github.com/BaochangZhang/PSSS_registration.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bier, B., et al.: X-ray-transform invariant anatomical landmark detection for pelvic trauma surgery. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018, Part IV. LNCS, vol. 11073, pp. 55–63. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_7
Grimm, M., Esteban, J., Unberath, M., Navab, N.: Pose-dependent weights and domain randomization for fully automatic X-ray to CT registration. IEEE Trans. Med. Imaging 40(9), 2221–2232 (2021)
Grupp, R.B., et al.: Automatic annotation of hip anatomy in fluoroscopy for robust and efficient 2D/3D registration. Int. J. Comput. Assist. Radiol. Surg. 15, 759–769 (2020)
Guan, S., Meng, C., Sun, K., Wang, T.: Transfer learning for rigid 2D/3D cardiovascular images registration. In: Lin, Z., Wang, L., Yang, J., Shi, G., Tan, T., Zheng, N., Chen, X., Zhang, Y. (eds.) PRCV 2019, Part II. LNCS, vol. 11858, pp. 380–390. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-31723-2_32
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Van de Kraats, E.B., Penney, G.P., Tomazevic, D., Van Walsum, T., Niessen, W.J.: Standardized evaluation methodology for 2-D-3-D registration. IEEE Trans. Med. Imaging 24(9), 1177–1189 (2005)
Lee, B.C., et al.: Breathing-compensated neural networks for real time C-arm pose estimation in lung CT-fluoroscopy registration. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–5. IEEE (2022)
Liu, S., et al.: Unpaired stain transfer using pathology-consistent constrained generative adversarial networks. IEEE Trans. Med. Imaging 40(8), 1977–1989 (2021)
Markelj, P., Tomaževič, D., Likar, B., Pernuš, F.: A review of 3D/2D registration methods for image-guided interventions. Med. Image Anal. 16(3), 642–661 (2012)
Meng, C., Wang, Q., Guan, S., Sun, K., Liu, B.: 2D-3D registration with weighted local mutual information in vascular interventions. IEEE Access 7, 162629–162638 (2019)
Miao, S., et al.: Dilated FCN for multi-agent 2D/3D medical image registration. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Penney, G.P., Weese, J., Little, J.A., Desmedt, P., Hill, D.L., et al.: A comparison of similarity measures for use in 2-D-3-D medical image registration. IEEE Trans. Med. Imaging 17(4), 586–595 (1998)
Powell, M.J.: The BOBYQA algorithm for bound constrained optimization without derivatives, vol. 26. Cambridge NA Report NA2009/06, University of Cambridge, Cambridge (2009)
Salehi, S.S.M., Khan, S., Erdogmus, D., Gholipour, A.: Real-time deep pose estimation with geodesic loss for image-to-template rigid registration. IEEE Trans. Med. Imaging 38(2), 470–481 (2018)
Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)
Unberath, M., et al.: The impact of machine learning on 2D/3D registration for image-guided interventions: a systematic review and perspective. Front. Robot. AI 8, 716007 (2021)
Unberath, M., et al.: DeepDRR – a catalyst for machine learning in fluoroscopy-guided procedures. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018, Part IV. LNCS, vol. 11073, pp. 98–106. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_12
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Acknowledgements
The project was supported by the Bavarian State Ministry of Science and Arts within the framework of the “Digitaler Herz-OP” project under the grant number 1530/891 02 and the China Scholarship Council (File No.202004910390). We also thank BrainLab AG for their partial support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, B. et al. (2023). A Patient-Specific Self-supervised Model for Automatic X-Ray/CT Registration. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14228. Springer, Cham. https://doi.org/10.1007/978-3-031-43996-4_49
Download citation
DOI: https://doi.org/10.1007/978-3-031-43996-4_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43995-7
Online ISBN: 978-3-031-43996-4
eBook Packages: Computer ScienceComputer Science (R0)