Abstract
We propose an adaptive training scheme for unsupervised medical image registration. Existing methods rely on image reconstruction as the primary supervision signal. However, nuisance variables (e.g. noise and covisibility), violation of the Lambertian assumption in physical waves (e.g. ultrasound), and inconsistent image acquisition can all cause a loss of correspondence between medical images. As the unsupervised learning scheme relies on intensity constancy between images to establish correspondence for reconstruction, this introduces spurious error residuals that are not modeled by the typical training objective. To mitigate this, we propose an adaptive framework that re-weights the error residuals with a correspondence scoring map during training, preventing the parametric displacement estimator from drifting away due to noisy gradients, which leads to performance degradation. To illustrate the versatility and effectiveness of our method, we tested our framework on three representative registration architectures across three medical image datasets along with other baselines. Our adaptive framework consistently outperforms other methods both quantitatively and qualitatively. Paired t-tests show that our improvements are statistically significant. Code available at: https://voldemort108x.github.io/AdaCS/.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahn, S.S., et al.: Co-attention spatial transformer network for unsupervised motion tracking and cardiac strain analysis in 3D echocardiography. Med. Image Anal. 84, 102711 (2023)
Ashburner, J.: A fast diffeomorphic image registration algorithm. NeuroImage 38(1), 95–113 (2007)
Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C.: Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12(1), 26–41 (2008)
Bae, G., Budvytis, I., Cipolla, R.: Estimating and exploiting the aleatoric uncertainty in surface normal estimation. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 13117–13126. IEEE, Montreal (2021)
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019)
Bernard, O., et al.: Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans. Med. Imaging 37(11), 2514–2525 (2018)
Chen, J., et al.: TransUNet: transformers make strong encoders for medical image segmentation (2021). arXiv:2102.04306 [cs]
Chen, J., Frey, E.C., He, Y., Segars, W.P., Li, Y., Du, Y.: TransMorph: transformer for unsupervised medical image registration. Med. Image Anal. 82, 102615 (2022)
Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning for fast probabilistic diffeomorphic registration, vol. 11070, pp. 729–738 (2018). arXiv:1805.04605 [cs]
Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Med. Image Anal. 57(1), 226–236 (2019)
Davatzikos, C.: Spatial transformation and registration of brain images using elastically deformable models. Comput. Vision Image Underst. 66(2), 207–222 (1997)
Dosovitskiy, A., et al.: An image is worth 16\(\times \)16 words: transformers for image recognition at scale (2021). arXiv:2010.11929 [cs]
Hill, D.L.G., Batchelor, P.G., Holden, M., Hawkes, D.J.: Medical image registration (2001)
Hoffmann, M., Billot, B., Greve, D.N., Iglesias, J.E., Fischl, B., Dalca, A.V.: SynthMorph: learning contrast-invariant registration without acquired images. IEEE Trans. Med. Imaging 41(3), 543–558 (2022). arXiv:2004.10282 [cs, eess, q-bio]
Hong, B.W., Koo, J.K., Burger, M., Soatto, S.: Adaptive regularization of some inverse problems in image analysis (2017). arXiv:1705.03350 [cs]
Hong, B.W., Koo, J.K., Dirks, H., Burger, M.: Adaptive regularization in convex composite optimization for variational imaging problems (2017). arXiv:1609.02356 [cs]
Hoopes, A., Hoffmann, M., Fischl, B., Guttag, J., Dalca, A.V.: HyperMorph: amortized hyperparameter learning for image registration (2021). arXiv:2101.01035 [cs, eess]
Hüllermeier, E., Waegeman, W.: Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Mach. Learn. 110(3), 457–506 (2021)
Keelan, R., Shimada, K., Rabin, Y.: GPU-Based Simulation of Ultrasound Imaging Artifacts for Cryosurgery Training. Technol. Cancer Res. Treat. 16(1), 5–14 (2017)
Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? (2017). arXiv:1703.04977 [cs]
Kim, B., Han, I., Ye, J.C.: DiffuseMorph: unsupervised deformable image registration using diffusion model. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, pp. 347–364. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19821-2_20
Klein, S., Staring, M., Murphy, K., Viergever, M., Pluim, J.: elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29(1), 196–205 (2010)
Leclerc, S., et al.: Deep learning for segmentation using an open large-scale dataset in 2D echocardiography. IEEE Trans. Med. Imaging 38(9), 2198–2210 (2019)
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: hierarchical vision transformer using shifted windows, pp. 10012–10022 (2021)
Ma, T., Dai, X., Zhang, S., Wen, Y.: PIViT: large deformation image registration with pyramid-iterative vision transformer. In: Greenspan, H., et al. (eds.) MICCAI 2023. LNCS, pp. 602–612. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43999-5_57
Mok, T.C.W., Chung, A.C.S.: Conditional deformable image registration with convolutional neural network. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, pp. 35–45. Springer, Cham (2021). https://doi.org/10.1007/978-3-031-43999-5_57
Monteiro, M., et al.: Stochastic segmentation networks: modelling spatially correlated aleatoric uncertainty. In: Advances in Neural Information Processing Systems, vol. 33, pp. 12756–12767. Curran Associates, Inc. (2020)
Oliveira, F.P.M.: Medical image registration: a review (2014)
Qin, Y., Li, X.: FSDiffReg: feature-wise and score-wise diffusion-guided unsupervised deformable image registration for cardiac images (2023). arXiv:2307.12035 [cs]
Rueckert, D., Sonoda, L., Hayes, C., Hill, D., Leach, M., Hawkes, D.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 18(8), 712–721 (1999)
Seitzer, M., Tavakoli, A., Antic, D., Martius, G.: On the pitfalls of heteroscedastic uncertainty estimation with probabilistic neural networks (2022). arXiv:2203.09168 [cs, stat]
Shi, J., et al.: XMorpher: full transformer for deformable medical image registration via cross attention (2022). arXiv:2206.07349 [cs]
Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models (2022), arXiv:2010.02502 [cs]
Ta, K., et al.: Multi-task learning for motion analysis and segmentation in 3D echocardiography. IEEE Trans. Med. Imaging (2024)
Wong, A., Fei, X., Hong, B.W., Soatto, S.: An adaptive framework for learning unsupervised depth completion. IEEE Robot. Autom. Lett. 6(2), 3120–3127 (2021)
Wong, A., Soatto, S.: Bilateral cyclic constraint and adaptive regularization for unsupervised monocular depth prediction. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5637–5646. IEEE, Long Beach (2019)
Zhang, X., Dong, H., Gao, D., Zhao, X.: A comparative study for non-rigid image registration and rigid image registration. arXiv preprint arXiv:2001.03831 (2020)
Zhang, X., Noga, M., Martin, D.G., Punithakumar, K.: Fully automated left atrium segmentation from anatomical cine long-axis MRI sequences using deep convolutional neural network with unscented Kalman filter. Med. Image Anal. 68, 101916 (2021)
Zhang, X., et al.: Heteroscedastic uncertainty estimation for probabilistic unsupervised registration of noisy medical images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham (2024)
Zhang, X., You, C., Ahn, S., Zhuang, J., Staib, L., Duncan, J.: Learning Correspondences of cardiac motion from images using biomechanics-informed modeling. In: Camara, O., et al. (eds.) STACOM 2022. LNCS, vol. 13593, pp. 13–25. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-23443-9_2
Acknowledgements
This work is supported by NIH/NHLBI grant R01HL121226.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, X., Stendahl, J.C., Staib, L.H., Sinusas, A.J., Wong, A., Duncan, J.S. (2025). Adaptive Correspondence Scoring for Unsupervised Medical Image Registration. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15096. Springer, Cham. https://doi.org/10.1007/978-3-031-72920-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-72920-1_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72919-5
Online ISBN: 978-3-031-72920-1
eBook Packages: Computer ScienceComputer Science (R0)