Abstract
In recent years, the Whole Slide Image (WSI) classification task has achieved great advancement due to the success of Multiple Instance Learning (MIL). However, the MIL-based studies usually consider instances within each bag as unordered, potentially resulting in the missing of local and global contextual information. To overcome this limitation, we propose a Noise Robust Memory-Augmented (Norma) framework for addressing the WSI classification task using a sequential approach. Norma serializes a WSI into a long sequence and adopts the Vision Transformer (ViT) to encode the local and global context information of the WSIs. Instead of processing long sequences at once, Norma splits the long sequence into multiple segments and sequentially trains these segments, with each segment being cached for future reuse. In addition, considering that segment-level labels are inherited from slide-level labels, which may introduce noise during training, Norma further introduces a cyclic method to reduce label noise. We achieve state-of-the-art performance on the CAMELYON-16, TCGA-BRAC and TCGA-LUNG datasets compared to recent studies. The code is available at https://github.com/weiaicunzai/Norma.
Y. Bai and B. Zhang—Contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bai, Y., et al.: A scalable graph-based framework for multi-organ histology image classification. IEEE J. Biomed. Health Inform. 26(11), 5506–5517 (2022)
Bai, Y., et al.: CoCa: a connectivity-aware cascade framework for histology gland segmentation. In: Proceedings of the 31st ACM International Conference on Multimedia, pp. 1598–1606 (2023)
Campanella, G., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25(8), 1301–1309 (2019)
Chan, T.H., Cendra, F.J., Ma, L., Yin, G., Yu, L.: Histopathology whole slide image analysis with heterogeneous graph representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15661–15670 (2023)
Chen, R.J., et al.: Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16144–16155 (2022)
Chu, X., Tian, Z., Zhang, B., Wang, X., Shen, C.: Conditional positional encodings for vision transformers. In: International Conference on Learning Representations (2023)
Dai, Z., Yang, Z., Yang, Y., Carbonell, J.G., Le, Q., Salakhutdinov, R.: Transformer-XL: attentive language models beyond a fixed-length context. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2978–2988 (2019)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. In: International Conference on Learning Representations (2020)
Guan, Y., et al.: Node-aligned graph convolutional network for whole-slide image representation and classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18813–18823 (2022)
Han, J., Luo, P., Wang, X.: Deep self-learning from noisy labels. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5138–5147 (2019)
Hashimoto, N., et al.: Multi-scale domain-adversarial multiple-instance CNN for cancer subtype classification with unannotated histopathological images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3852–3861 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Huang, Z., Zhang, J., Shan, H.: Twin contrastive learning with noisy labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11661–11670 (2023)
Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning, pp. 2127–2136 (2018)
Islam, M.A., Jia, S., Bruce, N.D.: How much position information do convolutional neural networks encode? In: International Conference on Learning Representations (2020)
Khosla, P., et al.: Supervised contrastive learning. In: Advances in Neural Information Processing Systems, vol. 33, pp. 18661–18673 (2020)
Li, B., Li, Y., Eliceiri, K.W.: Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318–14328 (2021)
Li, H., et al.: Task-specific fine-tuning via variational information bottleneck for weakly-supervised pathology whole slide image classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7454–7463 (2023)
Li, J., Xiong, C., Hoi, S.C.: Learning from noisy data with robust representation learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9485–9494 (2021)
Li, Y., Ping, W.: Cancer metastasis detection with neural conditional random field. arXiv preprint arXiv:1806.07064 (2018)
Lin, T., Yu, Z., Hu, H., Xu, Y., Chen, C.W.: Interventional bag multi-instance learning on whole-slide pathological images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19830–19839 (2023)
Lu, M.Y., Williamson, D.F., Chen, T.Y., Chen, R.J., Barbieri, M., Mahmood, F.: Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5(6), 555–570 (2021)
Ortego, D., Arazo, E., Albert, P., O’Connor, N.E., McGuinness, K.: Multi-objective interpolation training for robustness to label noise. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6606–6615 (2021)
Qin, W., Xu, R., Jiang, S., Jiang, T., Luo, L.: PathTR: context-aware memory transformer for tumor localization in gigapixel pathology images. In: Wang, L., Gall, J., Chin, T.J., Sato, I., Chellappa, R. (eds.) ACCV 2022. LNCS, vol. 13846, pp. 3603–3619. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-26351-4_8
Qu, L., Wang, M., Song, Z., et al.: Bi-directional weakly supervised knowledge distillation for whole slide image classification. In: Advances in Neural Information Processing Systems, vol. 35, pp. 15368–15381 (2022)
Qu, L., et al.: Boosting whole slide image classification from the perspectives of distribution, correlation and magnification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 21463–21473 (2023)
Rae, J.W., Potapenko, A., Jayakumar, S.M., Hillier, C., Lillicrap, T.P.: Compressive transformers for long-range sequence modelling. In: International Conference on Learning Representations (2019)
Reisenbüchler, D., Wagner, S.J., Boxberg, M., Peng, T.: Local attention graph-based transformer for multi-target genetic alteration prediction. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13432, pp. 377–386. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16434-7_37
Shao, W., Wang, T., Huang, Z., Han, Z., Zhang, J., Huang, K.: Weakly supervised deep ordinal cox model for survival prediction from whole-slide pathological images. IEEE Trans. Med. Imaging 40(12), 3739–3747 (2021)
Shao, Z., et al.: TransMIL: transformer based correlated multiple instance learning for whole slide image classification. In: Advances in Neural Information Processing Systems, vol. 34, pp. 2136–2147 (2021)
Shaw, P., Uszkoreit, J., Vaswani, A.: Self-attention with relative position representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 464–468 (2018)
Shen, Y., Ke, J.: A deformable CRF model for histopathology whole-slide image classification. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 500–508. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_48
Su, Y., Bai, Y., Zhang, B., Zhang, Z., Wang, W.: HAT-Net: a hierarchical transformer graph neural network for grading of colorectal cancer histology images. In: The British Machine Vision Conference, p. 412 (2021)
Tang, W., Huang, S., Zhang, X., Zhou, F., Zhang, Y., Liu, B.: Multiple instance learning framework with masked hard instance mining for whole slide image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4078–4087 (2023)
Tang, W., Zhou, F., Huang, S., Zhu, X., Zhang, Y., Liu, B.: Feature re-embedding: towards foundation model-level performance in computational pathology. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11343–11352 (2024)
Tellez, D., Litjens, G., Van der Laak, J., Ciompi, F.: Neural image compression for gigapixel histopathology image analysis. IEEE Trans. Pattern Anal. Mach. Intell. 43(2), 567–578 (2019)
Thandiackal, K., et al.: Differentiable zooming for multiple instance learning on whole-slide images. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, vol. 13681, pp. 699–715. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19803-8_41
Wang, X., Yan, Y., Tang, P., Bai, X., Liu, W.: Revisiting multiple instance neural networks. Pattern Recogn. 74, 15–24 (2018)
Wang, X., et al.: SCL-WC: cross-slide contrastive learning for weakly-supervised whole-slide image classification. In: Advances in Neural Information Processing Systems, vol. 35, pp. 18009–18021 (2022)
Wang, X., et al.: TransPath: transformer-based self-supervised learning for histopathological image classification. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12908, pp. 186–195. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87237-3_18
Wu, C.Y., et al.: MeMViT: memory-augmented multiscale vision transformer for efficient long-term video recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13587–13597 (2022)
Xiong, Y., et al.: Nyströmformer: a nyström-based algorithm for approximating self-attention. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35(16), pp. 14138–14148 (2021)
Xu, Y., et al.: Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinformatics 18, 1–17 (2017)
Yan, R., et al.: Breast cancer histopathological image classification using a hybrid deep neural network. Methods 173, 52–60 (2020)
Zhang, B., et al.: LSRML: a latent space regularization based meta-learning framework for MR image segmentation. Pattern Recogn. 130, 108821 (2022)
Zhang, B., et al.: Factorized omnidirectional representation based vision GNN for anisotropic 3D multimodal MR image segmentation. In: Proceedings of the 31st ACM International Conference on Multimedia, pp. 1607–1615 (2023)
Zhang, H., et al.: DTFD-MIL: double-tier feature distillation multiple instance learning for histopathology whole slide image classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18802–18812 (2022)
Zhang, M., Lucas, J., Ba, J., Hinton, G.E.: Lookahead optimizer: K steps forward, 1 step back. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Zhang, Y., et al.: AutoSight: distributed edge caching in short video network. IEEE Netw. 34(3), 194–199 (2020)
Zheng, Y., et al.: A graph-transformer for whole slide image classification. IEEE Trans. Med. Imaging 41(11), 3003–3015 (2022)
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China (No. 62072047, 61972046, 61802022, and 61802027), the Beijing Natural Science Foundation-Haidian Original Innovation Joint Fund Project (No. L182034), the Fundamental Research Funds for the Central Universities (No. 2019XD-A12), the Beijing Nova Program (NO. 20220484063), and the BUPT Excellent Ph.D. Students Foundation (No. CX2021136).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bai, Y. et al. (2025). Norma: A Noise Robust Memory-Augmented Framework for Whole Slide Image Classification. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15109. Springer, Cham. https://doi.org/10.1007/978-3-031-72983-6_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-72983-6_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72982-9
Online ISBN: 978-3-031-72983-6
eBook Packages: Computer ScienceComputer Science (R0)