Abstract
Surgery to treat elderly hip fracture patients may cause complications that can lead to early mortality. An early warning system for complications could provoke clinicians to monitor high-risk patients more carefully and address potential complications early, or inform the patient. In this work, we develop a multimodal deep-learning model for post-operative mortality prediction using pre-operative and per-operative data from elderly hip fracture patients. Specifically, we include static patient data, hip and chest images before surgery in pre-operative data, vital signals, and medications administered during surgery in per-operative data. We extract features from image modalities using ResNet and from vital signals using LSTM. Explainable model outcomes are essential for clinical applicability, therefore we compute Shapley values to explain the predictions of our multimodal black box model. We find that i) Shapley values can be used to estimate the relative contribution of each modality both locally and globally, and ii) a modified version of the chain rule can be used to propagate Shapley values through a sequence of models supporting interpretable local explanations. Our findings imply that a multimodal combination of black box models can be explained by propagating Shapley values through the model sequence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We used the scikit-learn implementation https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsRegressor.html.
- 2.
In the early stages of development, we observed that our models suffered from the “dying ReLu” problem, therefore we chose to employ leaky-ReLu instead of the regular ReLu activation function for all our fully connected layers [1].
- 3.
In preliminary experiments we investigated ordinal and temporal encoding for the per-operative medication data but did not find a difference in performance.
- 4.
- 5.
SHAP library https://github.com/slundberg/shap.
References
Agarap, A.F.: Deep learning using rectified linear units (ReLU). arXiv preprint arXiv:1803.08375 (2019)
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), e0130140 (2015). https://doi.org/10.1371/journal.pone.0130140
Briganti, G., Le Moine, O.: Artificial intelligence in medicine: today and tomorrow. Front. Med. 7 (2020)
Cai, L., Gao, J., Zhao, D.: A review of the application of deep learning in medical image classification and segmentation. Annal. Transl. Med. 8(11), 713–713 (2020). https://doi.org/10.21037/atm.2020.02.44
Cao, Y., et al.: Predictive values of preoperative characteristics for 30-day mortality in traumatic hip fracture patients. J. Pers. Med. 11(5), 353 (2021). https://doi.org/10.3390/jpm11050353
Chen, H., Lundberg, S.M., Lee, S.I.: Explaining a series of models by propagating Shapley values. Nat. Commun. 13(1), 4512 (2022). https://doi.org/10.1038/s41467-022-31384-3
Das, A., Rad, P.: Opportunities and challenges in explainable artificial Intelligence (XAI): a survey. arXiv preprint arXiv:2006.11371 (2020)
de Munter, L., Polinder, S., Lansink, K.W.W., Cnossen, M.C., Steyerberg, E.W., de Jongh, M.A.C.: Mortality prediction models in the general trauma population: a systematic review. Injury 48(2), 221–229 (2017). https://doi.org/10.1016/j.injury.2016.12.009
Fritz, B.A., et al.: Deep-learning model for predicting 30-day postoperative mortality. Br. J. Anaesth. 123(5), 688–695 (2019). https://doi.org/10.1016/j.bja.2019.07.025
Gowd, A.K., et al.: Construct validation of machine learning in the prediction of short-term postoperative complications following total shoulder arthroplasty. J. Shoulder Elbow Surg. 28(12), e410–e421 (2019). https://doi.org/10.1016/j.jse.2019.05.017
Gullberg, B., Johnell, O., Kanis, J.: World-wide projections for hip fracture. Osteoporos. Int. 7(5), 407–413 (1997). https://doi.org/10.1007/PL00004148
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016).https://doi.org/10.1109/CVPR.2016.90
Hu, F., Jiang, C., Shen, J., Tang, P., Wang, Y.: Preoperative predictors for mortality following hip fracture surgery: a systematic review and meta-analysis. Injury 43(6), 676–685 (2012). https://doi.org/10.1016/j.injury.2011.05.017
Jones, H.J., de Cossart, L.: Risk scoring in surgical patients. Br. J. Surg. 86(2), 149–157 (1999). https://doi.org/10.1046/j.1365-2168.1999.01006.x
Junaid, M., Ali, S., Eid, F., El-Sappagh, S., Abuhmed, T.: Explainable machine learning models based on multimodal time-series data for the early detection of Parkinson’s disease. Comput. Methods Prog. Biomed. 234, 10749 (2023). https://doi.org/10.1016/j.cmpb.2023.107495
Karim, F., Majumdar, S., Darabi, H.: Insights into LSTM fully convolutional networks for time series classification. IEEE Access 7, 67718–67725 (2019). https://doi.org/10.1109/ACCESS.2019.2916828
Karres, J., Kieviet, N., Eerenberg, J.P., Vrouenraets, B.C.: Predicting early mortality after hip fracture surgery: the hip fracture estimator of mortality Amsterdam. J. Orthop. Trauma 32(1), 27–33 (2018). https://doi.org/10.1097/BOT.0000000000001025
King, G., Zeng, L.: Logistic regression in rare events data. Politic. Anal. 9(2), 137–163 (2001).https://doi.org/10.1093/oxfordjournals.pan.a004868
Lee, C.K., Hofer, I., Gabel, E., Baldi, P., Cannesson, M.: Development and validation of a deep neural network model for prediction of postoperative in-hospital mortality. Anesthesiology 129(4), 649–662 (2018). https://doi.org/10.1097/ALN.0000000000002186
Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017)
Nagrani, A., Yang, S., Arnab, A., Jansen, A., Schmid, C., Sun, C.: Attention bottlenecks for multimodal fusion. In: Advances in Neural Information Processing Systems, vol. 34, pp. 14200–14213. Curran Associates, Inc. (2021)
Nijmeijer, W.S., Folbert, E.C., Vermeer, M., Slaets, J.P., Hegeman, J.H.: Prediction of early mortality following hip fracture surgery in frail elderly: the Almelo hip fracture score (AHFS). Injury 47(10), 2138–2143 (2016). https://doi.org/10.1016/j.injury.2016.07.022
Pawar, U., O’Shea, D., Rea, S., O’Reilly, R.: Explainable AI in Healthcare. In: 2020 International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA), pp. 1–2 (2020). https://doi.org/10.1109/CyberSA49311.2020.9139655
Perng, J.W., Kao, I.H., Kung, C.T., Hung, S.C., Lai, Y.H., Su, C.M.: Mortality prediction of septic patients in the emergency department based on machine learning. J. Clin. Med. 8(11), 1906 (2019). https://doi.org/10.3390/jcm8111906
Pham, T., Tran, T., Phung, D., Venkatesh, S.: Predicting healthcare trajectories from medical records: a deep learning approach. J. Biomed. Inform. 69, 218–229 (2017). https://doi.org/10.1016/j.jbi.2017.04.001
Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should I trust you?”: Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2016), pp. 1135–1144. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2939672.2939778
Schoenfeld, A.J., et al.: Assessing the utility of a clinical prediction score regarding 30-day morbidity and mortality following metastatic spinal surgery: the New England Spinal Metastasis Score (NESMS). The Spine J. 16(4), 482–490 (2016). https://doi.org/10.1016/j.spinee.2015.09.043
Schuster, M., Paliwal, K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45, 2673–2681 (1997). https://doi.org/10.1109/78.650093
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)
Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: International Conference on Machine Learning, pp. 3145–3153. PMLR (2017)
Tonekaboni, S., Joshi, S., McCradden, M.D., Goldenberg, A.: What clinicians want: contextualizing explainable machine learning for clinical end use. In: Proceedings of the 4th Machine Learning for Healthcare Conference, pp. 359–380. PMLR (2019)
Wang, W., Krishnan, E.: Big data and clinicians: a review on the state of the science. JMIR Med. Inf. 2(1), e1 (2014).https://doi.org/10.2196/medinform.2913
Xue, B., et al.: Use of machine learning to develop and evaluate models using preoperative and intraoperative data to identify risks of postoperative complications. JAMA Netw. Open 4(3), e212240 (2021). https://doi.org/10.1001/jamanetworkopen.2021.2240
Yenidogan, B., Pathak, S., Geerdink, J., Hegeman, J.H., van Keulen, M.: Multimodal machine learning for 30-days post-operative mortality prediction of elderly hip fracture patients. In: 2021 International Conference on Data Mining Workshops (ICDMW), pp. 508–516 (2021).https://doi.org/10.1109/ICDMW53433.2021.00068
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
Christin Seifert is a member of the XAI 2024 World Conference committee. All other authors have no competing interests to declare that are relevant to the content of this article.
Appendices
A Hyperparameters
See Table 5.
B Detailed Feature Overview
Lab results | Medication (reason/effect) | Comorbidities |
|---|---|---|
HB | Blood thinners | Chronic pulmonary disease |
HT | Vitamin D | Congestive heart failure |
CRP | Polypharmacy | Peripheral vascular disease |
LEUC | A02 (acid related disorders) | Cerebrovascular disease |
THR | A10 (diabetes) | Dementia |
BLGR | B01 (antithrombotic) | Renal disease |
IRAI | B02 (antihemmorrhagics) | Rheumatological disease |
ALKF | B03 (antianemic) | Cancer |
GGT | C01 (cardiac therapy) | Cerebrovascular event |
ASAT | C03 (diuretics) | Liver disease |
ALAT | C07 (beta blockers) | Lymphoma |
LDH1 | C08 (calcium channel blockers) | Leukemia |
UREU | C09 (renin-angiotensin system) | Peptic ulcer disease |
KREA | C10 (lipid modification) | Diabetes |
GFRM | L04 (immunosuppressants) | Prior myocardial infarction |
NA | M01 (anti-inflammatory) | |
XKA | N05 (psycholeptics) | |
GLUCGLUC | R03 (airway obstruction) |
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
van de Beld, JJ., Pathak, S., Geerdink, J., Hegeman, J.H., Seifert, C. (2024). Feature Importance to Explain Multimodal Prediction Models. a Clinical Use Case. In: Longo, L., Lapuschkin, S., Seifert, C. (eds) Explainable Artificial Intelligence. xAI 2024. Communications in Computer and Information Science, vol 2156. Springer, Cham. https://doi.org/10.1007/978-3-031-63803-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-63803-9_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-63802-2
Online ISBN: 978-3-031-63803-9
eBook Packages: Computer ScienceComputer Science (R0)