Abstract
Ultra-wide-field (UWF) fundus photography is a new imaging technique with providing a broader field of view images, and it has become a popular and effective tool for the screening and diagnosis for many eye diseases, such as diabetic retinopathy (DR). However, it is practically challenging to train a robust deep learning model for DR grading in UWF images, due to the limited scale of data and manual annotations. By contrast, we may find large-scale high-quality regular color fundus photography datasets in the research community, with either image-level or pixel-level annotation. In consequence, we propose an Unsupervised Lesion-aware TRAnsfer learning framework (ULTRA) for DR grading in UWF images, by leveraging a large amount of publicly well-annotated regular color fundus images. Inspired by the clinical identification of DR severity, i.e., the decision making process of ophthalmologists based on the type and number of associated lesions, we design an adversarial lesion map generator to provide the auxiliary lesion information for DR grading. A Lesion External Attention Module (LEAM) is introduced to integrate the lesion feature into the model, allowing a relative explainable DR grading. Extensive experimental results show the proposed method is superior to the state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yong, J.K., Kim, B.H., Bo, M.C., Sun, H.J., Choi, K.S.: Bariatric surgery is associated with less progression of diabetic retinopathy: a systematic review and meta-analysis. Surg. Obes. Relat. Dis. 13(2), 352 (2017)
Chilamkurthy, S., et al.: Development and validation of deep learning algorithms for detection of critical findings in head CT scans (2018)
Ju, L., Wang, X., Zhou, Q., Zhu, H., Ge, Z.: Bridge the domain gap between ultra-wide-field and traditional fundus images via adversarial domain adaptation (2020)
Zhao, Y., et al.: Uniqueness-driven saliency analysis for automated lesion detection with applications to retinal diseases. In: Frangi, A.F., et al. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 109–118. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_13
Ting, D., et al.: Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. Jama 318(22), 2211 (2017)
Sayres, R., et al.: Using a deep learning algorithm and integrated gradients explanation to assist grading for diabetic retinopathy. Ophthalmology (2018)
Foo, A., Hsu, W., Lee, M.L., Lim, G., Wong, T.Y.: Multi-task learning for diabetic retinopathy grading and lesion segmentation. Proc. AAAI Conf. Artif. Intell. 34(8), 13267–13272 (2020)
Sun, R., Li, Y., Zhang, T., Mao, Z., Wu, F., Zhang, Y.: Lesion-aware transformers for diabetic retinopathy grading. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10938–10947 (2021)
Nagasato, D., et al.: Deep neural network-based method for detecting central retinal vein occlusion using ultrawide-field fundus ophthalmoscopy. J. Ophthalmol. 2018, 1–6 (2018)
Daisuke, N., Hitoshi, T., Hideharu, O., Hiroki, M., Hiroki, E.: Deep-learning classifier with ultrawide-field fundus ophthalmoscopy for detecting branch retinal vein occlusion. Int. J. Ophthalmol. 12(1), 6 (2019)
Ohsugi, H., Tabuchi, H., Enno, H., Ishitobi, N.: Accuracy of deep learning, a machine-learning technology, using ultra-wide-field fundus ophthalmoscopy for detecting rhegmatogenous retinal detachment. Sci. Rep. 7, 9425 (2017)
Singh, R.P., et al.: Protecting vision in patients with diabetes with ultra-widefield imaging: a review of current literature. Ophthal. Surg. Lasers Imag. Retina 50(10), 639–648 (2019)
Ju, L., Wang, X., Zhao, X., Bonnington, P., Drummond, T., Ge, Z.: Leveraging regular fundus images for training UWF fundus diagnosis models via adversarial learning and pseudo-labeling. IEEE Trans. Med. Imaging (2021)
Nagasawa, T., et al.: Accuracy of ultrawide-field fundus ophthalmoscopy-assisted deep learning for detecting treatment-naïve proliferative diabetic retinopathy. Int. Ophthalmol. 39(10), 2153–2159 (2019). https://doi.org/10.1007/s10792-019-01074-z
Xie, H., et al.: Cross-attention multi-branch network for fundus diseases classification using SLO images. Med. Image Anal. 71, 102031 (2021)
Graham, B.: Kaggle Diabetic Retinopathy Detection Competition Report. University of Warwick (2015)
Porwal, P., et al.: Indian diabetic retinopathy image dataset (IDRID): a database for diabetic retinopathy screening research. Data 3(3), 25 (2018)
Long, M., Wang, J., Ding, G., Sun, J., Yu, P.S.: Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2013)
Wang, P., Lu, L., Li, J., Gan, W.: Transfer learning with joint distribution adaptation and maximum margin criterion (2018)
Xiao, N., Zhang, L.: Dynamic weighted learning for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15242–15251 (2021)
Abdelmaksoud, E., El-Sappagh, S., Barakat, S., Abuhmed, T., Elmogy, M.: Automatic diabetic retinopathy grading system based on detecting multiple retinal lesions. IEEE Access 9, 15939–15960 (2021)
Zhou, Y., et al.: Collaborative learning of semi-supervised segmentation and classification for medical images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2079–2088 (2019)
Grinsven, M.V., Ginneken, B.V., Hoyng, C., Theelen, T., Sanchez, C.: Fast convolutional neural network training using selective data sampling: application to hemorrhage detection in color fundus images. IEEE Transactions on Medical Imaging, pp. 1273–1284 (2016)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: International Conference on Machine Learning, PMLR, pp. 1180–1189 (2015)
Tsai, Y.H., Hung, W.C., Schulter, S., Sohn, K., Yang, M.H., Chandraker, M.: Learning to adapt structured output space for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7472–7481 (2018)
Goodfellow, I., et al.: Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27 (2014)
Kamnitsas, K., et al.: Unsupervised domain adaptation in brain lesion segmentation with adversarial networks. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 597–609. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59050-9_47
Fu, J., et al.: Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3146–3154 (2019)
Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., Erhan, D.: Domain separation networks (2016)
Tzeng, E., Hoffman, J., Saenko, K., Darrell, T.: Adversarial discriminative domain adaptation (2017)
Acknowledgment
This work was supported in part by the National Science Foundation Program of China (62103398), Zhejiang Provincial Natural Science Foundation of China (LR22F020008), in part by the Youth Innovation Promotion Association CAS (2021298), in part by the Ningbo major science and technology task project (2021Z054) and the AME Programmatic Fund (A20H4b0141).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bai, Y. et al. (2022). Unsupervised Lesion-Aware Transfer Learning for Diabetic Retinopathy Grading in Ultra-Wide-Field Fundus Photography. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13432. Springer, Cham. https://doi.org/10.1007/978-3-031-16434-7_54
Download citation
DOI: https://doi.org/10.1007/978-3-031-16434-7_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16433-0
Online ISBN: 978-3-031-16434-7
eBook Packages: Computer ScienceComputer Science (R0)