Explainable Artificial Intelligence on Biosignals for Clinical Decision Support
Authors: 
Miriam Cindy Maurer, Jacqueline Michelle Metsch, Philip Hempel, Theresa Bender, Nicolai Spicher, Anne-Christin HauschildAuthors Info & Claims
Pages 6597 - 6604
Abstract
Deep learning has proven effective in several areas, including computer vision, natural language processing, and disease prediction, which can support clinicians in making decisions along the clinical pathway. However, in order to successfully integrate these algorithms into clinical practice, it is important that their decision-making processes are transparent, explainable, and interpretable. Firstly, this tutorial will introduce targeted eXplainable Artificial Intelligence (XAI) methods to address the urgent need for explainability of deep learning in healthcare applications. In particular, it focuses on algorithms for raw biosignals without prior feature extraction that enable medical diagnoses, specifically electrocardiograms (ECG) -- stemming from the heart -- and electroencephalograms (EEG) representing the electrical activity of the brain. Secondly, participants are provided with a comprehensive workflow that includes both data processing and an introduction to relevant network architectures. Subsequently, various XAI methods are described and it is shown, how the resulting relevance attributions can be visualized on biosignals. Finally, two compelling real-world use cases are presented that demonstrate the effectiveness of XAI in analyzing ECG and EEG signals for disease prediction and sleep classification, respectively. In summary, the tutorial will provide the skills required for gaining insight into the decision process of deep neural networks processing authentic clinical biosignal data.
References
[1]
Regulation (eu) 2017/745 of the european parliament and of the council of 5 april 2017 on medical devices, amending directive 2001/83/ec, regulation (ec) no 178/2002 and regulation (ec) no 1223/2009 and repealing council directives 90/385/eec and 93/42/eec.
[2]
Erick A Perez Alday, Annie Gu, Amit J Shah, Chad Robichaux, An-Kwok Ian Wong, Chengyu Liu, Feifei Liu, Ali Bahrami Rad, Andoni Elola, Salman Seyedi, Qiao Li, Ashish Sharma, Gari D Clifford, and Matthew A Reyna. Classification of 12-lead ecgs: the physionet/computing in cardiology challenge 2020. Physiological Measurement, 41(12):124003, dec 2020.
[3]
Emina Alickovic and Abdulhamit Subasi. Ensemble svm method for automatic sleep stage classification. IEEE Transactions on Instrumentation and Measurement, 67(6):1258--1265, 2018.
[4]
American Academy of Sleep Medicine and others. The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications, volume 23 of Westchester, IL: American Academy of Sleep Medicine. 2007.
[5]
Atul Anand, Tushar Kadian, Manu Kumar Shetty, and Anubha Gupta. Explainable ai decision model for ecg data of cardiac disorders. Biomedical Signal Processing and Control, 75:103584, 2022.
[6]
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for nonlinear classifier decisions by layer-wise relevance propagation. PLOS ONE, 10(7):1-- 46, 07 2015.
[7]
Theresa Bender, Jacqueline Beinecke, Dagmar Krefting, Carolin Muller, Henning Dathe, Tim Seidler, Nicolai Spicher, and Anne-Christin Hauschild. Analysis of a deep learning model for 12-lead ecg classification reveals learned features similar to diagnostic criteria. IEEE journal of biomedical and health informatics, PP, 05 2023.
[8]
Alexander Brown, Nenad Tomasev, Jan Freyberg, Yuan Liu, Alan Karthikesalingam, and Jessica Schrouff. Detecting shortcut learning for fair medical ai using shortcut testing. Nature Communications, 14(1):4314, 2023.
[9]
Maowei Cheng,Worku J. Sori, Feng Jiang, Adil Khan, and Shaohui Liu. Recurrent neural network based classification of ecg signal features for obstruction of sleep apnea detection. In 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC), volume 2, pages 199--202, 2017.
[10]
Luay Fraiwan, Khaldon Lweesy, Natheer Khasawneh, Heinrich Wenz, and Hartmut Dickhaus. Automated sleep stage identification system based on time-- frequency analysis of a single eeg channel and random forest classifier. Computer Methods and Programs in Biomedicine, 108(1):10--19, 2012.
[11]
T.B. Garcia and N.E. Holtz. Introduction to 12-lead ECG: The Art of Interpretation. Emergency Medical Services Series. Jones and Bartlett, 2003.
[12]
A. L. Goldberger, L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. Ch. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101(23):e215--e220, 2000 (June 13). Circulation Electronic Pages: http://circ.ahajournals.org/content/101/23/e215.full 1085218;
[13]
Mark L Graber, Nancy Franklin, and Ruthanna Gordon. Diagnostic error in internal medicine. Archives of internal medicine, 165(13):1493--1499, 2005.
[14]
Antoine Guillot, Fabien Sauvet, Emmanuel H. During, and Valentin Thorey. Dreem open datasets: Multi-scored sleep datasets to compare human and automated sleep staging. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(9):1955--1965, 2020.
[15]
Antoine Guillot and Valentin Thorey. Robustsleepnet: Transfer learning for automated sleep staging at scale. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 29:1441--1451, 2021.
[16]
Salih Güne", Kemal Polat, and "ebnem Yosunkaya. Efficient sleep stage recognition system based on eeg signal using k-means clustering based feature weighting. Expert Systems with Applications, 37(12):7922--7928, 2010.
[17]
Iqram Hussain, Rafsan Jany, Richard Boyer, AKM Azad, Salem A Alyami, Se Jin Park, Md Mehedi Hasan, and Md Azam Hossain. An explainable eeg-based human activity recognition model using machine-learning approach and lime. Sensors, 23(17):7452, 2023.
[18]
Mohammed Saidul Islam, Iqram Hussain, Md Mezbaur Rahman, Se Jin Park, and Md Azam Hossain. Explainable artificial intelligence model for stroke prediction using eeg signal. Sensors, 22(24):9859, 2022.
[19]
Jacob C Jentzer, Anthony H Kashou, Francisco Lopez-Jimenez, Zachi I Attia, Suraj Kapa, Paul A Friedman, and Peter A Noseworthy. Mortality risk stratification using artificial intelligence-augmented electrocardiogram in cardiac intensive care unit patients. European Heart Journal Acute Cardiovascular Care, 10(5):532-- 541, 2021.
[20]
Xiaopeng Ji, Yan Li, and Peng Wen. Jumping knowledge based spatial-temporal graph convolutional networks for automatic sleep stage classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30:1464--1472, 2022.
[21]
Yong-Yeon Jo, Younghoon Cho, Soo Youn Lee, Joon-myoung Kwon, Kyung-Hee Kim, Ki-Hyun Jeon, Soohyun Cho, Jinsik Park, and Byung-Hee Oh. Explainable artificial intelligence to detect atrial fibrillation using electrocardiogram. International journal of cardiology, 328:104--110, 2021.
[22]
Susumu Katsushika, Satoshi Kodera, Shinnosuke Sawano, Hiroki Shinohara, Naoto Setoguchi, Kengo Tanabe, Yasutomi Higashikuni, Norifumi Takeda, Katsuhito Fujiu, Masao Daimon, et al. An explainable artificial intelligence-enabled electrocardiogram analysis model for the classification of reduced left ventricular function. European Heart Journal-Digital Health, 4(3):254--264, 2023.
[23]
Mehrin Kiani, Javier Andreu-Perez, Hani Hagras, Silvia Rigato, and Maria Laura Filippetti. Towards understanding human functional brain development with explainable artificial intelligence: Challenges and perspectives. IEEE Computational Intelligence Magazine, 17(1):16--33, 2022.
[24]
Serkan Kiranyaz, Turker Ince, and Moncef Gabbouj. Real-time patient-specific ecg classification by 1-d convolutional neural networks. IEEE Transactions on Biomedical Engineering, 63(3):664--675, 2016.
[25]
Tamás Kiss, Stephen Morairty, Michael Schwartz, Thomas Kilduff, Derek Buhl, and Dmitri Volfson. Automated sleep stage scoring using k-nearest neighbors classifier. Journal of Open Source Software, 5(53):2377, 2020.
[26]
Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz-Richardson. Captum: A unified and generic model interpretability library for pytorch, 2020.
[27]
Johann Laux, Sandra Wachter, and Brent Mittelstadt. Trustworthy artificial intelligence and the european union ai act: On the conflation of trustworthiness and acceptability of risk. Regulation & Governance, 18(1):3--32, 2024.
[28]
Feifei Liu, Chengyu Liu, Lina Zhao, Xiangyu Zhang, Xiaoling Wu, Xiaoyan Xu, Yulin Liu, Caiyun Ma, Shoushui Wei, Zhiqiang He, Jianqing Li, and Eddie Ng. An open access database for evaluating the algorithms of electrocardiogram rhythm and morphology abnormality detection. Journal of Medical Imaging and Health Informatics, 8:1368--1373, 09 2018.
[29]
Francesco Carlo Morabito, Cosimo Ieracitano, and Nadia Mammone. An explainable artificial intelligence approach to study mci to ad conversion via hd-eeg processing. Clinical EEG and Neuroscience, 54(1):51--60, 2023.
[30]
Meraj Neyazi, Jan P Bremer, Marius S Knorr, Stefan Gross, Jan Brederecke, Nils Schweingruber, Dora Csengeri, Benedikt Schrage, Martin Bahls, Nele Friedrich, et al. Deep learning-based nt-probnp prediction from the ecg for risk assessment in the community. Clinical Chemistry and Laboratory Medicine (CCLM), (0), 2023.
[31]
Hmayag Partamian, Fouad Khnaisser, Mohamad Mansour, Reem Mahmoud, and Hazem Hajjand Fadi Karameh. A deep model for eeg seizure detection with explainable ai using connectivity features. In International Conference on Biomedical Engineering and Science (BIOEN 2021) doi, volume 10, 2021.
[32]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pages 8024--8035. Curran Associates, Inc., 2019.
[33]
Antônio H Ribeiro, Manoel Horta Ribeiro, GabrielaMMPaixão, DerickMOliveira, Paulo R Gomes, Jéssica A Canazart, Milton PS Ferreira, Carl R Andersson, PeterW Macfarlane, Wagner Meira Jr, et al. Automatic diagnosis of the 12-lead ecg using a deep neural network. Nature communications, 11(1):1760, 2020.
[34]
Marco Ribeiro, Sameer Singh, and Carlos Guestrin. "why should I trust you"": Explaining the predictions of any classifier. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 97--101, San Diego, California, June 2016. Association for Computational Linguistics.
[35]
Khaled Rjoob, Raymond Bond, Dewar Finlay, Victoria McGilligan, Stephen J Leslie, Ali Rababah, Aleeha Iftikhar, Daniel Guldenring, Charles Knoery, Anne McShane, et al. Towards explainable artificial intelligence and explanation user interfaces to open the "black box'of automated ecg interpretation. In Advanced Visual Interfaces. Supporting Artificial Intelligence and Big Data Applications: AVI 2020 Workshops, AVI-BDA and ITAVIS, Ischia, Italy, June 9, 2020 and September 29, 2020, Revised Selected Papers, pages 96--108. Springer, 2021.
[36]
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 618--626, 2017.
[37]
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, page 3145--3153. JMLR.org, 2017.
[38]
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps, 2014.
[39]
Timo Speith. A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 2239--2250, Seoul Republic of Korea, June 2022. ACM.
[40]
J.T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller. Striving for simplicity: The all convolutional net. In ICLR (workshop track), 2015.
[41]
Jens B. Stephansen, Alexander N. Olesen, Mads Olsen, Aditya Ambati, Eileen B. Leary, Hyatt E. Moore, Oscar Carrillo, Ling Lin, Fang Han, Han Yan, Yun L. Sun, Yves Dauvilliers, Sabine Scholz, Lucie Barateau, Birgit Hogl, Ambra Stefani, Seung Chul Hong, Tae Won Kim, Fabio Pizza, Giuseppe Plazzi, Stefano Vandi, Elena Antelmi, Dimitri Perrin, Samuel T. Kuna, Paula K. Schweitzer, Clete Kushida, Paul E. Peppard, Helge B. D. Sorensen, Poul Jennum, and Emmanuel Mignot. Neural network analysis of sleep stages enables efficient diagnosis of narcolepsy. Nature communications, 9(1):5229, 2018.
[42]
Erik Strumbelj and Igor Kononenko. An efficient explanation of individual classifications using game theory. J. Mach. Learn. Res., 11:1--18, 2010.
[43]
Cathie Sudlow, John Gallacher, Naomi Allen, Valerie Beral, Paul Burton, John Danesh, Paul Downey, Paul Elliott, Jane Green, Martin Landray, et al. Uk biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS medicine, 12(3):e1001779, 2015.
[44]
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks, 2017.
[45]
Borys Surawicz, Rory Childers, Barbara J Deal, and Leonard S Gettes. Aha/accf/hrs recommendations for the standardization and interpretation of the electrocardiogram: part iii: intraventricular conduction disturbances: a scientific statement from the american heart association electrocardiography and arrhythmias committee, council on clinical cardiology; the american college of cardiology foundation; and the heart rhythm society: endorsed by the international society for computerized electrocardiology. Circulation, 119(10):e235--e240, 2009.
[46]
Hirohisa Taniguchi, Tomohiro Takata, Mineki Takechi, Asuka Furukawa, Jin Iwasawa, Akio Kawamura, Tadahiro Taniguchi, and Yuichi Tamura. Explainable artificial intelligence model for diagnosis of atrial fibrillation using holter electrocardiogram waveforms. International heart journal, 62(3):534--539, 2021.
[47]
Hugues Turbé, Mina Bjelogrlic, Christian Lovis, and Gianmarco Mengaldo. Dataset: Evaluation of post-hoc interpretability methods in time-series classification, January 2023.
[48]
Hugues Turbé, Mina Bjelogrlic, Christian Lovis, and Gianmarco Mengaldo. Evaluation of post-hoc interpretability methods in time-series classification. Nat. Mach. Intell., 5(3):250--260, March 2023.
[49]
Akhil Vaid, Joy Jiang, Ashwin Sawant, Stamatios Lerakis, Edgar Argulian, Yuri Ahuja, Joshua Lampert, Alexander Charney, Heather Greenspan, Jagat Narula, Benjamin Glicksberg, and Girish Nadkarni. A foundational vision transformer improves diagnostic performance for electrocardiograms. npj Digital Medicine, 6, 06 2023.
[50]
Rutger R van de Leur, Max N Bos, Karim Taha, Arjan Sammani, Ming Wai Yeung, Stefan van Duijvenboden, Pier D Lambiase, Rutger J Hassink, Pim van der Harst, Pieter A Doevendans, Deepak K Gupta, and René van Es. Improving explainability of deep neural network-based electrocardiogram interpretation using variational auto-encoders. European Heart Journal - Digital Health, 07 2022. ztac038.
[51]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, " ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H.Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
[52]
Dukyong Yoon, Jong-Hwan Jang, Byung Jin Choi, Tae Young Kim, and Chang Ho Han. Discovering hidden information in biosignals from patients using artificial intelligence. Korean journal of anesthesiology, 73(4):275--284, 2020.
[53]
Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision -- ECCV 2014, pages 818--833, Cham, 2014. Springer International Publishing.
[54]
Kamyar Zeinalipour and Marco Gori. Graph Neural Networks for Topological Feature Extraction in ECG Classification, pages 17--27. Springer Nature Singapore, Singapore, 2023.
Index Terms
- Explainable Artificial Intelligence on Biosignals for Clinical Decision Support
Index terms have been assigned to the content through auto-classification.
Recommendations
Methods and standards for research on explainable artificial intelligence: Lessons from intelligent tutoring systems
AbstractThe DARPA Explainable Artificial Intelligence (AI) (XAI) Program focused on generating explanations for AI programs that use machine learning techniques. This article highlights progress during the DARPA Program (2017‐2021) relative to research ...
Lessons learned in the work on intelligent tutoring systems that apply to system design in Explainable AI. image image
Epileptic signal classification using convolutional neural network and Shapley additive explainable artificial intelligence method
AbstractNeurological disorder epilepsy can cause life-threatening seizures which may produce abnormal patterns in electroencephalogram (EEG) signals. However, manual tracking of abnormal EEG patterns is subjective, and only artificial intelligence (AI)-...
A Multidimensional Taxonomy for Recent Trends in Explainable Artificial Intelligence
Progress in Artificial IntelligenceAbstractWith the pervasive integration of Artificial Intelligence (AI) into various facets of society, concerns regarding its transparency and interpretability have gained prominence, particularly in critical or citizen-facing applications. The field of ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
August 2024
6901 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 August 2024
Check for updates
Author Tags
Qualifiers
- Tutorial
Funding Sources
- Lower Saxony Vorab of the Volkswagen Foundation and the Ministry for Science and Culture of Lower Saxony
- Instationsausschuss beim Gemeinsamen Bundesausschuss (G-BA)
Conference
KDD '24: The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 25 - 29, 2024
Barcelona, Spain
Acceptance Rates
Overall Acceptance Rate 1,133 of 8,635 submissions, 13%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 636Total Downloads
- Downloads (Last 12 months)636
- Downloads (Last 6 weeks)96
Reflects downloads up to 16 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in