Abstract
PURPOSE: Deployment of deep models for clinical decision making should not only provide predicted outcomes, but also insights on how decisions are made. Considering the interpretability of Transformer models, and the power of graph networks in analyzing the inherent hierarchy of biological signals, a combined approach would be the next generation solution in computer aided interventions. In this study, we propose a framework for classification and visualization of surgical mass spectrometry data using Graph Transformer model to empower the interpretability of breast surgical margin assessment. METHODS: Using the iKnife, 144 burns (103 normal, 41 cancer) were collected and converted to multi-level graph structures. A Graph Transformer model was modified to output the intermediate attention parameters of the network. Beside ablation and prospective study, we propose multiple attention visualization approaches to facilitate the interpretability. RESULTS: In a 4-fold cross validation experiment, an average classification AUC of 95.6% was achieved, outperforming baseline models. We could also distinguish and visualize clear pattern of attention difference between burns. For instance, cancerous and normal burns gather more attention in the lower and higher subbands of the spectra respectively. Looking at cancer subtype prospectively, a pattern of cancer progression was also observed in the attention features. CONCLUSION: Graph Transformers are powerful in providing high network interpretability. When paired with proper visualization, they can be deployed for computer assisted interventions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Akbarifar, F., et al.: Graph-based analysis of mass spectrometry data for tissue characterization with application in basal cell carcinoma surgery. In: SPIE Medical Imaging: Image-Guided Procedures, Robotic Interventions, and Modeling, p. 11598 (2021)
Balog, J., et al.: Intraoperative tissue identification using rapid evaporative ionization mass spectrometry. Sci. Transl. Med. 5(2), 194 (2013)
DeBerardinis, R., Lum, J., Hatzivassiliou, G., Thompson, C.: The biology of cancer: metabolic reprogramming fuels cell growth and proliferation. Cell Metab. 7, 11–20 (2008)
Dwivedi, V., Bresson, X.: A generalization of transformer networks to graphs. In: AAAI 2021 Workshop on Deep Learning on Graphs. arXiv:2012.09699 (2021)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations, ICLR (2017)
Koundouros, N., et al.: Metabolic fingerprinting links oncogenic PIK3CA with enhanced arachidonic acid-derived eicosanoids. Cell 181, 1596–1611 (2020)
Linardatos, P., Papastefanopoulos, V., Kotsiantis, S.: Explainable AI: a review of machine learning interpretability methods. Entropy (Basel, Switzerland) 23(1), 18 (2020)
Nambiar, A., Liu, S., Hopkins, M., Heflin, M., Maslov, S., Ritz, A.: Transforming the language of life: transformer neural networks for protein prediction tasks. bioRxiv (2020). https://doi.org/10.1101/2020.06.15.153643
Santilli, A.M.L., et al.: Domain adaptation and self-supervised learning for surgical margin detection. Int. J. Comput. Assist. Radiol. Surg. 16(5), 861–869 (2021). https://doi.org/10.1007/s11548-021-02381-6
Santoro, A., et al.: In situ DESI-MSI lipidomic profiles of breast cancer molecular subtypes and precursor lesions. Cancer Res. 80, 1246–1257 (2020)
St-John, E., et al.: Diagnostic accuracy of intraoperative techniques for margin assessment in breast cancer surgery. Anal. Surg. 265(2), 300–310 (2017)
Vaswani, A., et al.: Attention is all you need. In: NIPS. arXiv:1706.03762 (2017)
Wu, J., Zhong, J., Chen, E., Zhang, J., Ye, J., Yu, L.: Weakly- and semi-supervised graph CNN for identifying basal cell carcinoma on pathological images. Graph Learn. Med. Imaging 11849, 112–119 (2019)
Yao, D., et al.: Triplet graph convolutional network for multi-scale analysis of functional connectivity using functional MRI. In: Zhang, D., Zhou, L., Jie, B., Liu, M. (eds.) GLMI 2019. LNCS, vol. 11849, pp. 70–78. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35817-4_9
Ying, Z., Bourgeois, D., You, J., Zitnik, M., Leskovec, J.: GNNExplainer: generating explanations for graph neural networks. In: Advances in Neural Information Processing Systems NeurIPS 2019, vol. 32, pp. 9240–9251. Curran Associates, Inc. (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Jamzad, A. et al. (2021). Graph Transformers for Characterization and Interpretation of Surgical Margins. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12907. Springer, Cham. https://doi.org/10.1007/978-3-030-87234-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-87234-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87233-5
Online ISBN: 978-3-030-87234-2
eBook Packages: Computer ScienceComputer Science (R0)