Abstract
Accurate deep learning-based segmentation of retinal arteries and veins (A/V) enables improved diagnosis, monitoring, and management of ocular fundus diseases and systemic diseases. However, existing resized and patch-based algorithms face challenges with redundancy, overlooking thin vessels, and underperforming in low-contrast edge areas of the retinal images, due to imbalanced background-to-A/V ratios and limited contexts. Here, we have developed a novel deep learning framework for retinal A/V segmentation, named RIP-AV, which integrates a Representative Instance Pre-training (RIP) task with a context-aware network for retinal A/V segmentation for the first time. Initially, we develop a direct yet effective algorithm for vascular patch-pair selection (PPS) and then introduce a RIP task, formulated as a multi-label problem, aiming at enhancing the network's capability to learn latent arteriovenous features from diverse spatial locations across vascular patches. Subsequently, in the training phase, we introduce two novel modules: Patch Context Fusion (PCF) module and Distance Aware (DA) module. They are designed to improve the discriminability and continuity of thin vessels, especially in low-contrast edge areas, by leveraging the relationship between vascular patches and their surrounding contexts cooperatively and complementarily. The effectiveness of RIP-AV has been validated on three publicly available retinal datasets: AV-DRIVE, LES-AV, and HRF, demonstrating remarkable accuracies of 0.970, 0.967, and 0.981, respectively, thereby outperforming existing state-of-the-art methods. Notably, our method achieves a significant 1.7% improvement in accuracy on the HRF dataset, particularly enhancing the segmentation of thin edge arteries and veins.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Safi, H., Safi, S., Hafezi-Moghadam, A., Ahmadieh, H.: Early detection of diabetic retinopathy. Surv. Ophthalmol. 63(5), 601–608 (2018)
Tapp, R.J., et al.: Associations of retinal microvascular diameters and tortuosity with blood pressure and arterial stiffness: United Kingdom biobank. Hypertension 74(6), 1383–1390 (2019)
Zekavat, S.M., et al.: Deep learning of the retina enables phenome- and genome-wide analyses of the microvasculature. Circulation 145(2), 134–150 (2022)
Fogel-Levin, M., et al.: Advanced retinal imaging and applications for clinical practice: a consensus review. Surv. Ophthalmol. 67(5), 1373–1390 (2022)
Galdran, A., Meyer, M., Costa, P., Campilho, A.: Uncertainty-aware artery/vein classification on retinal images. In: ISBI, pp. 556–560 (2019)
Galdran, A., et al.: The little W-net that could: state-of-the-art retinal vessel segmentation with minimalistic models. arXiv preprint arXiv:2009.01907 (2020)
Li, L., Verma, M., Nakashima, Y., Kawasaki, R., Nagahara, H.: Joint learning of vessel segmentation and artery/vein classification with post-processing. In: Medical Imaging with Deep Learning, pp. 440–453 (2020)
Zhou, Y., et al.: Learning to address intra-segment misclassification in retinal imaging. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 482–492. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_46
Girard, F., Kavalec, C., Cheriet, F.: Joint segmentation and classification of retinal arteries/veins from fundus images. Artif. Intell. Med. 94, 96–109 (2019)
Ma, W., Yu, S., Ma, K., Wang, J., Ding, X., Zheng, Y.: Multi-task neural networks with spatial activation for retinal vessel segmentation and artery/vein classification. In: Shen, D., et al. (eds.) MICCAI 2019.LNCS, vol. 11764, pp. 769–778. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_85
Chen, W., et al.: TR-GAN: topology ranking GAN with triplet loss for retinal artery/vein classification. In: Martel, A.L., et al. (eds.) MICCAI 2020, LNCS, vol. 12265, pp. 616–625. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_59
Hu, J., et al.: Automatic artery/vein classification using a vessel-constraint network for multicenter fundus images. Front. Cell Dev. Biol. 9, 659941 (2021)
Chen, W., et al.: TW-GAN: topology and width aware GAN for retinal artery/vein classification. Med. Image Anal. 77, 102340 (2022)
Luo, S., Heng, Z., Pagnucco, M., Song, Y.: Two-stage topological refinement network for retinal artery/vein classification. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–4 (2022)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Goodfellow, I., et al.: Generative adversarial nets. In: NeurIPS, vol. 27 (2014)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR, pp. 3431–3440 (2015)
Liu, Z., Mao, H., Wu, C., Feichtenhofer, C., Darrell, T., Xie, S.: A ConvNet for the 2020s. In: CVPR, pp. 11966–11976 (2022)
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR, pp. 5967–5976 (2017)
Qu, Z., Zhuo, L., Cao, J., Li, X., Yin, H., Wang, Z.: TP-net: two-path network for retinal vessel segmentation. JBHI 27(4), 1979–1990 (2023)
Maurer J., C.R., Qi, R., Raghavan, V.: A linear time algorithm for computing exact Euclidean distance transforms of binary images in arbitrary dimensions. TPAMI 25(2), 265–270 (2003)
Zhang, S., Song, L., Gao, C., Sang, N.: GLNet: global local network for weakly supervised action localization. IEEE Trans. Mult. 22(10), 2610–2622 (2020)
Li, Q., Yang, W., Liu, W., Yu, Y., He, S.: From Contexts to Locality: Ultra-high Resolution Image Segmentation via Locality-aware Contextual Correlation. In: ICCV, pp. 7232–7241 (2021)
Vaswani, A., et al.: Attention is all you need. In: NeurIPS, pp. 5998–6008 (2017)
Hu, Q., Abràmoff, M.D., Garvin, M.K.: Automated Separation of Binary Overlapping Trees in Low-Contrast Color Retinal Images. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, LNCS, vol. 8150, pp. 436–443. Springer, Berlin, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40763-5_54
Odstrcilik, J., et al.: Retinal vessel segmentation by improved matched filtering: evaluation on a new high-resolution fundus image database. IET Image Pro. 7(4), 373–383 (2013)
Orlando, J.I., Barbosa Breda, J., van Keer, K., Blaschko, M.B., Blanco, P.J., Bulant, C.A.: Towards a glaucoma risk index based on simulated hemodynamics from fundus images. In: Frangi, A., Schnabel, J., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds) MICCAI 2018, LNCS, vol.11071, pp. 65–73. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_8
Paszke, A., et al.: Pytorch: An imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. (2008)
Morano, J., et al.: Simultaneous segmentation and classification of the retinal arteries and veins from color fundus images. Artif. Intell. Med. 118 (2021)
Karlsson, R.A., Sveinn H.H.: Artery vein classification in fundus images using serially connected U-Nets. Comput. Meth. Programs Biomed. 216 (2022)
Zhao, A.D., et al.: Optimization of retinal artery/vein classification based on vascular topology. Biomed. Signal Process. Control 88 (2024)
Wang, C., Xu, R., Xu, S., Meng, W., Zhang, X.: DA-Net: dual branch transformer and adaptive strip upsampling for retinal vessels segmentation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13432, pp. 528–538. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16434-7_51
Acknowledgments
This study was funded by the National Natural Science Foundation of China (82172882) and supported by Biomedical Big Data Intelligent Computing Center of Oujiang Lab.
Funding
We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the study.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dai, W. et al. (2024). RIP-AV: Joint Representative Instance Pre-training with Context Aware Network for Retinal Artery/Vein Segmentation. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15001. Springer, Cham. https://doi.org/10.1007/978-3-031-72378-0_71
Download citation
DOI: https://doi.org/10.1007/978-3-031-72378-0_71
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72377-3
Online ISBN: 978-3-031-72378-0
eBook Packages: Computer ScienceComputer Science (R0)