Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3663976.3663993acmotherconferencesArticle/Chapter ViewAbstractPublication PagescvipprConference Proceedingsconference-collections
research-article

CVMIL: Cluster Variance Multiple Instance Learning for Whole Slide Images Survival Prediction

Published: 27 June 2024 Publication History

Abstract

Tumor survival prediction using whole slide images (WSIs) is a crucial application in pathology aimed at assisting doctors in better formulating post-surgical treatment plans. The key challenges in current WSIs survival prediction lie in the vast scale of WSIs and the scarcity of manual annotations, which hinders the extraction of effective information from WSIs. To address these issues, previous studies have mainly employed the multiple instance learning (MIL) approach. However, existing methods often fail to consider the complexity of tumors and integrate clinically relevant knowledge, leading to suboptimal outcomes in survival prediction. To capture the intricate characteristics of tumors, we propose Cluster Variance Multiple Instance Learning (CVMIL) framework capable of representing tumor heterogeneity. By leveraging the differences from cluster centers, CVMIL represents both intra-tumor and inter-tumor heterogeneity, thereby enhancing the performance of MIL methods in WSI survival prediction. Results from prognosis tasks conducted on three publicly available TCGA datasets and the in-house ARGO dataset demonstrate that our approach outperforms current state-of-the-art methods, enabling more effective prediction of patient prognosis.

References

[1]
[1] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436–444, May 2015.
[2]
[2] Jiawen Yao, Xinliang Zhu, Jitendra Jonnagaddala, Nicholas Hawkins, and Junzhou Huang. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Medical Image Analysis, 65:101789, October 2020.
[3]
[3] Zhuchen Shao, Hao Bian, Yang Chen, Yifeng Wang, Jian Zhang, Xiangyang Ji, and Yongbing Zhang. TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification. In 35th Conference on Neural Information Processing Systems, volume 34, pages 2136–2147, 2021.
[4]
[4] Hongrun Zhang, Yanda Meng, Yitian Zhao, Yihong Qiao, Xiaoyun Yang, Sarah E. Coupland, and Yalin Zheng. DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification, March 2022. arXiv:2203.12081.
[5]
[5] Noemi Andor, Trevor A Graham, Marnix Jansen, Li C Xia, C Athena Aktipis, Claudia Petritsch, Hanlee P Ji, and Carlo C Maley. Pan-cancer analysis of the extent and consequences of intratumor heterogeneity. Nature Medicine, 22(1):105–113, January 2016.
[6]
[6] Jinping Liu and Dang Hien. The significance of intertumor and intratumor heterogeneity in liver cancer. Experimental & Molecular Medicine, 50(1):e416–e416, 2018.
[7]
[7] Iain Carmichael, Andrew H. Song, Richard J. Chen, Drew F. K. Williamson, Tiffany Y. Chen, and Faisal Mahmood. Incorporating intratumoral heterogeneity into weakly-supervised deep learning models via variance pooling, November 2022. arXiv:2206.08885.
[8]
[8] Zhikang Wang, Yue Bi, Tong Pan, Xiaoyu Wang, Chris Bain, Richard Bassed, Seiya Imoto, Jianhua Yao, Roger J Daly, and Jiangning Song. Targeting tumor heterogeneity: multiplex-detection-based multiple instance learning for whole slide image classification. Bioinformatics, 39(3):btad114, March 2023.
[9]
[9] Jin-Gang Yu, Zihao Wu, Yu Ming, Shule Deng, Yuanqing Li, Caifeng Ou, Chunjiang He, Baiye Wang, Pusheng Zhang, and Yu Wang. Prototypical multiple instance learning for predicting lymph node metastasis of breast cancer from whole-slide pathological images. Medical Image Analysis, 85:102748, April 2023.
[10]
[10] Ming Y. Lu, Drew F. K. Williamson, Tiffany Y. Chen, Richard J. Chen, Matteo Barbieri, and Faisal Mahmood. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering, 5(6):555–570, March 2021.
[11]
[11] Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas J. Montine, and James Zou. A visual–language foundation model for pathology image analysis using medical Twitter. Nature Medicine, 29(9):2307–2316, September 2023.
[12]
[12] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38 th International Conference on Machine Learning, pages 8748–8763, 2021.
[13]
[13] Maximilian Ilse, Jakub M Tomczak, and Max Welling. Attention-based Deep Multiple Instance Learning. In Proceedings of the 35 th International Conference on Machine Learning, pages 2127–2136, 2018.
[14]
[14] D. R. Cox. Regression models and life-tables. Journal of the Royal Statistical Society: Series B (Methodological), 34(2):187–220, 1972.
[15]
[15] Wing Hung Wong. Theory of partial likelihood. The Annals of Statistics, 14(1):88–123, 1986.
[16]
[16] Cyriac Kandoth, Michael D. McLellan, Fabio Vandin, Kai Ye, Beifang Niu, Charles Lu, Mingchao Xie, Qunyuan Zhang, Joshua F. McMichael, Matthew A. Wyczalkowski, Mark D. M. Leiserson, Christopher A. Miller, John S. Welch, Matthew J. Walter, Michael C. Wendl, Timothy J. Ley, Richard K. Wilson, Benjamin J. Raphael, and Li Ding. Mutational landscape and significance across 12 major cancer types. Nature, 502(7471):333–339, October 2013.
[17]
[17] Wentai Hou, Yan He, Bingjian Yao, Lequan Yu, Rongshan Yu, Feng Gao, and Liansheng Wang. Multi-scope Analysis Driven Hierarchical Graph Transformer for Whole Slide Image Based Cancer Survival Prediction. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 745–754. 2023.
[18]
[18] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, June 2016.
[19]
[19] Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization, January 2017. arXiv:1412.6980.
[20]
[20] Ping Wang, Yan Li, and Chandan K. Reddy. Machine Learning for Survival Analysis: A Survey. Acm Computing Surveys, 51(6):1–36, November 2019.
[21]
[21] Ruoyu Li, Jiawen Yao, Xinliang Zhu, Yeqing Li, and Junzhou Huang. Graph CNN for Survival Analysis on Whole Slide Pathological Images. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 174–182. 2018.
[22]
[22] Bin Li, Yin Li, and Kevin W. Eliceiri. Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14318–14328. IEEE, June 2021.

Index Terms

  1. CVMIL: Cluster Variance Multiple Instance Learning for Whole Slide Images Survival Prediction

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      CVIPPR '24: Proceedings of the 2024 2nd Asia Conference on Computer Vision, Image Processing and Pattern Recognition
      April 2024
      373 pages
      ISBN:9798400716607
      DOI:10.1145/3663976
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 27 June 2024

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. computational pathology
      2. survival prediction
      3. whole slide image

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      CVIPPR 2024

      Acceptance Rates

      Overall Acceptance Rate 14 of 38 submissions, 37%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 32
        Total Downloads
      • Downloads (Last 12 months)32
      • Downloads (Last 6 weeks)9
      Reflects downloads up to 25 Nov 2024

      Other Metrics

      Citations

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media