Abstract
In the context of multiple myeloma, patient diagnosis and treatment planning involve the medical analysis of full-body Positron Emission Tomography (PET) images. There has been a growing interest in linking quantitative measurements extracted from PET images (radiomics) with statistical methods for survival analysis. Following very recent advances, we propose an end-to-end deep learning model that learns relevant features and predicts survival given the image of a lesion. We show the importance of dealing with the variable scale of the lesions, and propose to this end an attention strategy deployed both on the spatial and channels dimensions, which improves the model performance and interpretability. We show results for the progression-free survival prediction of multiple myeloma (MM) patients on a clinical dataset coming from two prospective studies. We also discuss the difficulties of adapting deep learning for survival analysis given the complexity of the task, the small lesion sizes, and PET low SNR (signal to noise ratio).
This work has been partially funded by the SIRIC ILIAD (INCa-DGOS-Inserm_12558), the LabEx IRON (ANR-11-LABX-0018-01) and by the European Regional Development Fund, the Pays de la Loire region on the Connect Talent scheme (MILCOM Project) and Nantes Métropole (Convention 2017-10470).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhu, X., Yao, J., Zhu, F., Huang, J.: WSISA: making survival prediction from whole slide histopathological images. In: CVPR, pp. 970–975 (2017)
Amyar, A., Ruan, S., Gardin, I., Chatelain, C., Decazes, P., Modzelewski, R.: 3-D RPET-NET: development of a 3-D pet imaging convolutional neural network for radiomics analysis and outcome prediction. IEEE Trans. Radiat. Plasma Med. Sci. 3(2), 225–231 (2019)
Ching, T., Zhu, X., Garmire, L.X.: Cox-nnet: an artificial neural network method for prognosis prediction of high-throughput omics data. PLoS Comput. Biol. 14(4), 1–18 (2018). https://doi.org/10.1371/journal.pcbi.1006076
Cook, G.J.R., Siddique, M., Taylor, B.P., Yip, C., Chicklore, S., Goh, V.: Radiomics in PET: principles and applications. Clin. Transl. Imaging 2(3), 269–276 (2014). https://doi.org/10.1007/s40336-014-0064-0
Faraggi, D., Simon, R.: A neural network model for survival data. Stat. Med. 14(1), 73–82 (1995). https://doi.org/10.1002/sim.4780140108
Gensheimer, M.F., Narasimhan, B.: A scalable discrete-time survival model for neural networks. PeerJ 2019(1), 1–17 (2019). https://doi.org/10.7717/peerj.6257
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 346–361. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10578-9_23
Herent, P., et al.: Detection and characterization of MRI breast lesions using deep learning. Diagn. Interv. Imaging 100(4), 219–225 (2019). https://doi.org/10.1016/j.diii.2019.02.008
Huang, Q., Yang, D., Wu, P., Qu, H., Yi, J., Metaxas, D.: MRI reconstruction via cascaded channel-wise attention network. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1622–1626 (2019). https://doi.org/10.1109/ISBI.2019.8759423
Kaji, D.A., Zech, J.R., Kim, J.S., et al.: An attention based deep learning model of clinical events in the intensive care unit. PLoS ONE 14(2), e0211057 (2019). https://doi.org/10.1371/journal.pone.0211057
Katzman, J.L., Shaham, U., Cloninger, A., Bates, J., Jiang, T., Kluger, Y.: DeepSurv: personalized treatment recommender system using a cox proportional hazards deep neural network. BMC Med. Res. Methodol. 18, 24 (2018). https://doi.org/10.1186/s12874-018-0482-1
Kleinbaum, D.G., Klein, M. (eds.): Survival Analysis: A Self-Learning Text. Springer, New York (2012). https://doi.org/10.1007/978-1-4419-6646-9
Lao, J., et al.: A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Sci. Rep. 7(1) (2017). https://doi.org/10.1038/s41598-017-10649-8
Li, H., et al.: Deep convolutional neural networks for imaging data based survival analysis of rectal cancer. In: ISBI, April, vol. 2019, pp. 846–849 (2019). https://doi.org/10.1109/ISBI.2019.8759301
Liu, Z., Sun, Q., Bai, H., Liang, C., Chen, Y., Li, Z.: 3D deep attention network for survival prediction from magnetic resonance images in glioblastoma. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1381–1384 (2019)
Mobadersany, P., et al.: Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl. Acad. Sci. 115(13), E2970–E2979 (2018). https://doi.org/10.1073/pnas.1717139115. https://www.pnas.org/content/115/13/E2970
Shboul, Z.A., Alam, M., Vidyaratne, L., Pei, L., Elbakary, M.I., Iftekharuddin, K.M.: Feature-guided deep radiomics for glioblastoma patient survival prediction. Front. Neurosci. 13, 966 (2019). https://doi.org/10.3389/fnins.2019.00966. https://www.frontiersin.org/article/10.3389/fnins.2019.00966
Tong, Q., et al.: RIANet: recurrent interleaved attention network for cardiac MRI segmentation. Comput. Biol. Med. 109, 290–302 (2019). https://doi.org/10.1016/j.compbiomed.2019.04.042
Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_1
Zhu, X., Yao, J., Huang, J.: Deep convolutional neural network for survival analysis with pathological images. In: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), vol. 1, pp. 544–547. IEEE (2016). https://doi.org/10.1109/BIBM.2016.7822579
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Morvan, L. et al. (2020). Learned Deep Radiomics for Survival Analysis with Attention. In: Rekik, I., Adeli, E., Park, S.H., Valdés Hernández, M.d.C. (eds) Predictive Intelligence in Medicine. PRIME 2020. Lecture Notes in Computer Science(), vol 12329. Springer, Cham. https://doi.org/10.1007/978-3-030-59354-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-59354-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59353-7
Online ISBN: 978-3-030-59354-4
eBook Packages: Computer ScienceComputer Science (R0)