Abstract
Conventional hematoxylin-eosin (H&E) staining is limited to revealing cell morphology and distribution, whereas immunohistochemical (IHC) staining provides precise and specific visualization of protein activation at the molecular level. Virtual staining technology has emerged as a solution for highly efficient IHC examination, which directly transforms H&E-stained images to IHC-stained images. However, virtual staining is challenged by the insufficient mining of pathological semantics and the spatial misalignment of pathological semantics. To address these issues, we propose the Pathological Semantics-Preserving Learning method for Virtual Staining (PSPStain), which directly incorporates the molecular-level semantic information and enhances semantics interaction despite any spatial inconsistency. Specifically, PSPStain comprises two novel learning strategies: 1) Protein-Aware Learning Strategy (PALS) with Focal Optical Density (FOD) map maintains the coherence of protein expression level, which represents molecular-level semantic information; 2) Prototype-Consistent Learning Strategy (PCLS), which enhances cross-image semantic interaction by prototypical consistency learning. We evaluate PSPStain on two public datasets using five metrics: three clinically relevant metrics and two for image quality. Extensive experiments indicate that PSPStain outperforms current state-of-the-art H&E-to-IHC virtual staining methods and demonstrates a high pathological correlation between the staging of real and virtual stains. Code is available at https://github.com/ccitachi/PSPStain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Anglade, F., Milner Jr, D.A., Brock, J.E.: Can pathology diagnostic services for cancer be stratified and serve global health? Cancer 126(S10), 2431–2438 (2020)
Avcıbaş, I.s., Sankur, B.l., Sayood, K.: Statistical evaluation of image quality measures. Journal of Electronic Imaging 11(2), 206–223 (2002)
Di Cataldo, S., Ficarra, E., Macii, E.: Computer-aided techniques for chromogenic immunohistochemistry: status and directions. Computers in biology and medicine 42(10), 1012–1025 (2012)
Dubey, S., Kataria, T., Knudsen, B., Elhabian, S.Y.: Structural cycle gan for virtual immunohistochemistry staining of gland markers in the colon. In: Machine Learning in Medical Imaging: 14th International Workshop, MLMI 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8, 2023, Proceedings, Part II. pp. 447–456. Springer (2023)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 770–778 (2016)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1125–1134 (2017)
Li, F., Hu, Z., Chen, W., Kak, A.: Adaptive supervised patchnce loss for learning h &e-to-ihc stain translation with inconsistent groundtruth image pairs. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: 26th International Conference, Vancouver, BC, Canada, October 8–12, 2023, Proceedings, Part VI. pp. 632–641. Springer (2023)
Liu, S., Zhu, C., Xu, F., Jia, X., Shi, Z., Jin, M.: Bci: Breast cancer immunohistochemical image generation through pyramid pix2pix. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1815–1824 (2022)
Liu, S., Zhang, B., Liu, Y., Han, A., Shi, H., Guan, T., He, Y.: Unpaired stain transfer using pathology-consistent constrained generative adversarial networks. IEEE Transactions on Medical Imaging 40(8), 1977–1989 (2021)
Park, T., Efros, A.A., Zhang, R., Zhu, J.Y.: Contrastive learning for unpaired image-to-image translation. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX 16. pp. 319–345. Springer (2020)
Ruifrok, A.C., Johnston, D.A., et al.: Quantification of histochemical staining by color deconvolution. Analytical and quantitative cytology and histology 23(4), 291–299 (2001)
Varghese, F., Bukhari, A.B., Malhotra, R., De, A.: Ihc profiler: an open source plugin for the quantitative evaluation and automated scoring of immunohistochemistry images of human tissue samples. PloS one 9(5), e96801 (2014)
Wang, C.J., Zhou, Z.G., Holmqvist, A., Zhang, H., Li, Y., Adell, G., Sun, X.F.: Survivin expression quantified by image pro-plus compared with visual assessment. Applied Immunohistochemistry & Molecular Morphology 17(6), 530–535 (2009)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)
Zeng, B., Lin, Y., Wang, Y., Chen, Y., Dong, J., Li, X., Zhang, Y.: Semi-supervised pr virtual staining for breast histopathological images. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2022: 25th International Conference, Singapore, September 18-22, 2022, Proceedings, Part II. pp. 232–241. Springer (2022)
Zhang, R., Cao, Y., Li, Y., Liu, Z., Wang, J., He, J., Zhang, C., Sui, X., Zhang, P., Cui, L., et al.: Mvfstain: multiple virtual functional stain histopathology images generation based on specific domain mapping. Medical Image Analysis 80, 102520 (2022)
Zhang, Z., Ran, R., Tian, C., Zhou, H., Li, X., Yang, F., Jiao, Z.: Self-aware and cross-sample prototypical learning for semi-supervised medical image segmentation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: 26th International Conference, Vancouver, BC, Canada, October 8-12, 2023, Proceedings, Part II. pp. 192–201. Springer (2023)
Zhu, C., Liu, S., Yu, Z., Xu, F., Aggarwal, A., Corredor, G., Madabhushi, A., Qu, Q., Fan, H., Li, F., et al.: Breast cancer immunohistochemical image generation: a benchmark dataset and challenge review. arXiv preprint arXiv:2305.03546 (2023)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2223–2232 (2017)
Acknowledgments
This work was supported by the National Natural Science Foundation of China (No. 62271475), Ministry of Science and Technology’s key research and development program (2023YFF0723400), the Youth Innovation Promotion Association CAS (2022365) and Shenzhen-Hong Kong Joint Lab on Intelligence Computational Analysis for Tumor lmaging (E3G111).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors declare no competing interests in the paper.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, F., Zhang, R., Zheng, B., Sun, Y., He, J., Qin, W. (2024). Pathological Semantics-Preserving Learning for H&E-to-IHC Virtual Staining. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15004. Springer, Cham. https://doi.org/10.1007/978-3-031-72083-3_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-72083-3_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72082-6
Online ISBN: 978-3-031-72083-3
eBook Packages: Computer ScienceComputer Science (R0)