Abstract
Chest radiology images such as CT scans and X-ray images have been extensively employed in computer-assisted analysis of COVID-19, utilizing various learning-based techniques. As a trending topic, image retrieval is a practical solution by providing users with a selection of remarkably similar images from a retrospective database, thereby assisting in timely diagnosis and intervention. Many existing studies utilize deep learning algorithms for chest radiology image retrieval by extracting features from images and searching the most similar images based on the extracted features. However, these methods seldom consider the complex relationship among images (e.g., images belonging to the same category tend to share similar representations, and vice versa), which may result in sub-optimal retrieval accuracy. In this paper, we develop a triplet-constrained image retrieval (TIR) framework for chest radiology image search to aid in COVID-19 diagnosis. The TIR contains two components: (a) feature extraction and (b) image retrieval, where a triplet constraint and an image reconstruction constraint are embedded to enhance the discriminative ability of learned features. In particular, the triplet constraint is designed to minimize the distances between images belonging to the same category and maximize the distances between images from different categories. Based on the extracted features, we further perform chest X-ray (CXR) image search. Experimental results on a total of 29, 986 CXR images from a public COVIDx dataset with 16, 648 subjects demonstrate the effectiveness of the proposed method compared with several state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Apostolopoulos, I.D., Mpesiana, T.A.: Covid-19: automatic detection from X-ray images utilizing transfer learning with convolutional neural networks. Phys. Eng. Sci. Med. 43, 635–640 (2020)
Wang, L., Lin, Z.Q., Wong, A.: Covid-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images. Sci. Rep. 10(1), 1–12 (2020)
Hu, B., Vasu, B., Hoogs, A.: X-MRI: explainable medical image retrieval. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 440–450 (2022)
Qi, A., et al.: Directional mutation and crossover boosted ant colony optimization with application to covid-19 x-ray image segmentation. Comput. Biol. Med. 148, 105810 (2022)
Covid, C., et al.: Severe outcomes among patients with coronavirus disease 2019 (COVID-19)-United States, February 12–March 16 2020. Morb. Mortal. Wkly Rep. 69(12), 343 (2020)
Akgül, C.B., Rubin, D.L., Napel, S., Beaulieu, C.F., Greenspan, H., Acar, B.: Content-based image retrieval in radiology: current status and future directions. J. Digit. Imaging 24, 208–222 (2011)
Das, P., Neelima, A.: An overview of approaches for content-based medical image retrieval. Int. J. Multimedia Inf. Retrieval 6(4), 271–280 (2017). https://doi.org/10.1007/s13735-017-0135-x
Jain, R., Gupta, M., Taneja, S., Hemanth, D.J.: Deep learning based detection and analysis of COVID-19 on chest X-ray images. Appl. Intell. 51, 1690–1700 (2021)
Minaee, S., Kafieh, R., Sonka, M., Yazdani, S., Soufi, G.J.: Deep-COVID: predicting COVID-19 from chest X-ray images using deep transfer learning. Med. Image Anal. 65, 101794 (2020)
Zhong, A., et al.: Deep metric learning-based image retrieval system for chest radiograph and its clinical applications in COVID-19. Med. Image Anal. 70, 101993 (2021)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)
Jégou, H., Douze, M., Schmid, C.: Improving bag-of-features for large scale image search. Int. J. Comput. Vision 87, 316–336 (2010)
Yao, D., et al.: A mutual multi-scale triplet graph convolutional network for classification of brain disorders using functional or structural connectivity. IEEE Trans. Med. Imaging 40(4), 1279–1289 (2021)
Liu, M., Zhang, D., Shen, D.: Relationship induced multi-template learning for diagnosis of Alzheimer’s disease and mild cognitive impairment. IEEE Trans. Med. Imaging 35(6), 1463–1474 (2016)
Smeulders, A.W., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349–1380 (2000)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Rundo, L., et al.: USE-Net: incorporating squeeze-and-excitation blocks into U-Net for prostate zonal segmentation of multi-institutional MRI datasets. Neurocomputing 365, 31–43 (2019)
Kaiming, H., Shaoqing, R., Jian, S.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Liu, G., Reda, F.A., Shih, K.J., Wang, T.-C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11215, pp. 89–105. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01252-6_6
Zhu, H., Long, M., Wang, J., Cao, Y.: Deep hashing network for efficient similarity retrieval. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)
Liu, H., Wang, R., Shan, S., Chen, X.: Deep supervised hashing for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2064–2072 (2016)
Cao, Y., Long, M., Liu, B., Wang, J.: Deep cauchy hashing for hamming space retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1229–1237 (2018)
Mohite, N.B., Gonde, A.B.: Deep features based medical image retrieval. Multimedia Tools Appl. 81(8), 11379–11392 (2022). https://doi.org/10.1007/s11042-022-12085-x
Shen, Y., Li, H., Yi, S., Chen, D., Wang, X.: Person re-identification with deep similarity-guided graph neural network. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11219, pp. 508–526. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01267-0_30
Wen, Y., Zhang, K., Li, Z., Qiao, Yu.: A discriminative feature learning approach for deep face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 499–515. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_31
Liu, Y., Yue, L., Xiao, S., Yang, W., Shen, D., Liu, M.: Assessing clinical progression from subjective cognitive decline to mild cognitive impairment with incomplete multi-modal neuroimages. Med. Image Anal. 75, 102266 (2022)
Liu, M., Gao, Y., Yap, P.T., Shen, D.: Multi-hypergraph learning for incomplete multimodality data. IEEE J. Biomed. Health Inf. 22(4), 1197–1208 (2017)
Acknowledgements
L. Qiao was supported in part by National Natural Science Foundation of China (Nos. 61976110, 62176112, 11931008) and Natural Science Foundation of Shandong Province (No. ZR202102270451).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, L., Wang, Q., Wang, X., Ma, Y., Qiao, L., Liu, M. (2024). Triplet Learning for Chest X-Ray Image Search in Automated COVID-19 Analysis. In: Cao, X., Xu, X., Rekik, I., Cui, Z., Ouyang, X. (eds) Machine Learning in Medical Imaging. MLMI 2023. Lecture Notes in Computer Science, vol 14349. Springer, Cham. https://doi.org/10.1007/978-3-031-45676-3_41
Download citation
DOI: https://doi.org/10.1007/978-3-031-45676-3_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45675-6
Online ISBN: 978-3-031-45676-3
eBook Packages: Computer ScienceComputer Science (R0)