short-paper

Open access

SERF: Interpretable Sleep Staging using Embeddings, Rules, and Features

Authors:

Irfan Al-Hussaini,

Cassie S. MitchellAuthors Info & Claims

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Pages 3791 - 3795

https://doi.org/10.1145/3511808.3557700

Published: 17 October 2022 Publication History

Abstract

The accuracy of recent deep learning based clinical decision support systems is promising. However, lack of model interpretability remains an obstacle to widespread adoption of artificial intelligence in healthcare. Using sleep as a case study, we propose a generalizable method to combine clinical interpretability with high accuracy derived from black-box deep learning.

Clinician-determined sleep stages from polysomnogram (PSG) remain the gold standard for evaluating sleep quality. However, PSG manual annotation by experts is expensive and time-prohibitive. We propose SERF, interpretable Sleep staging using Embeddings, Rules, and Features to read PSG. SERF provides interpretation of classified sleep stages through meaningful features derived from the AASM Manual for the Scoring of Sleep and Associated Events.

In SERF, the embeddings obtained from a hybrid of convolutional and recurrent neural networks are transposed to the interpretable feature space. These representative interpretable features are used to train simple models like a shallow decision tree for classification. Model results are validated on two publicly available datasets. SERF surpasses the current state-of-the-art for interpretable sleep staging by 2%. Using Gradient Boosted Trees as the classifier, SERF obtains 0.766 κ and 0.870 AUC-ROC, within 2% of the current state-of-the-art black-box models.

References

[1]

Julia Adler-Milstein and Ashish K Jha. 2017. HITECH Act drove large gains in hospital electronic health record adoption. Health affairs, Vol. 36, 8 (2017), 1416--1422.

[2]

Muhammad Aurangzeb Ahmad, Carly Eckert, and Ankur Teredesai. 2018. Interpretable Machine Learning in Healthcare. In Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (Washington, DC, USA) (BCB '18). Association for Computing Machinery, New York, NY, USA, 559--560. https://doi.org/10.1145/3233547.3233667

Digital Library

[3]

Irfan Al-Hussaini, Cao Xiao, M Brandon Westover, and Jimeng Sun. 2019. SLEEPER: interpretable Sleep staging via Prototypes from Expert Rules. In Machine Learning for Healthcare Conference. PMLR, 721--739.

[4]

Richard B. Berry, Rohit Budhiraja, Daniel J. Gottlieb, David Gozal, Conrad Iber, Vishesh K. Kapur, Carole L. Marcus, Reena Mehra, Sairam Parthasarathy, Stuart F. Quan, et al. 2012. Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events. Journal of clinical sleep medicine, Vol. 8, 05 (2012), 597--619.

[5]

Siddharth Biswal, Joshua Kulas, Haoqi Sun, Balaji Goparaju, M. Brandon Westover, Matt T. Bianchi, and Jimeng Sun. 2017. SLEEPNET: automated sleep staging system via deep learning. arXiv preprint arXiv:1707.08262 (2017).

[6]

Diogo V Carvalho, Eduardo M Pereira, and Jaime S Cardoso. 2019. Machine learning interpretability: A survey on methods and metrics. Electronics, Vol. 8, 8 (2019), 832.

[7]

Stanislas Chambon, Mathieu N. Galtier, Pierrick J. Arnal, Gilles Wainrib, and Alexandre Gramfort. 2018. A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 26, 4 (2018), 758--769.

[8]

Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD '16). Association for Computing Machinery, New York, NY, USA, 785--794. https://doi.org/10.1145/2939672.2939785

Digital Library

[9]

Luigi De Gennaro and Michele Ferrara. 2003. Sleep spindles: an overview. Sleep medicine reviews, Vol. 7, 5 (2003), 423--440.

[10]

Hao Dong, Akara Supratak, Wei Pan, Chao Wu, Paul M. Matthews, and Yike Guo. 2018. Mixed neural network approach for temporal sleep stage classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 26, 2 (2018), 324--333.

[11]

Radwa Elshawi, Mouaz H Al-Mallah, and Sherif Sakr. 2019. On the interpretability of machine learning-based model for predicting hypertension. BMC medical informatics and decision making, Vol. 19, 1 (2019), 1--32.

[12]

Ary L Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mietus, George B Moody, Chung-Kang Peng, and H Eugene Stanley. 2000. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. circulation, Vol. 101, 23 (2000), e215--e220.

[13]

Antonio Guglietta. 2015. Drug treatment of sleep disorders. Springer.

[14]

Antoine Guillot, Fabien Sauvet, Emmanuel H During, and Valentin Thorey. 2020. Dreem open datasets: Multi-scored sleep datasets to compare human and automated sleep staging. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 28, 9 (2020), 1955--1965.

[15]

Andreas Holzinger, Georg Langs, Helmut Denk, Kurt Zatloukal, and Heimo Müller. 2019. Causability and explainability of artificial intelligence in medicine. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, Vol. 9, 4 (2019), e1312.

[16]

Peter B Jensen, Lars J Jensen, and Søren Brunak. 2012. Mining electronic health records: towards better research applications and clinical care. Nature Reviews Genetics, Vol. 13, 6 (2012), 395--405.

[17]

Ziyu Jia, Youfang Lin, Jing Wang, Ronghao Zhou, Xiaojun Ning, Yuanlai He, and Yaoshuai Zhao. 2020. GraphSleepNet: Adaptive Spatial-Temporal Graph Convolutional Networks for Sleep Stage Classification. In IJCAI. 1324--1330.

[18]

Sharon Keenan and Max Hirshkowitz. 2011. Monitoring and staging human sleep. Principles and practice of sleep medicine, Vol. 5 (2011), 1602--1609.

[19]

Bob Kemp, Aeilko H Zwinderman, Bert Tuk, Hilbert AC Kamphuisen, and Josefien JL Oberye. 2000. Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG. IEEE Transactions on Biomedical Engineering, Vol. 47, 9 (2000), 1185--1194.

[20]

Sirvan Khalighi, Teresa Sousa, José Moutinho Santos, and Urbano Nunes. 2016. ISRUC-Sleep: a comprehensive public dataset for sleep researchers. Computer methods and programs in biomedicine, Vol. 124 (2016), 180--192.

Digital Library

[21]

Menglei Li, Hongbo Chen, and Zixue Cheng. 2022. An Attention-Guided Spatiotemporal Graph Convolutional Network for Sleep Stage Classification. Life, Vol. 12, 5 (2022), 622.

[22]

Zachary Chase Lipton. 2016. The Mythos of Model Interpretability. CoRR, Vol. abs/1606.03490 (2016). http://arxiv.org/abs/1606.03490

[23]

Alex John London. 2019. Artificial intelligence and black-box medical decisions: accuracy versus explainability. Hastings Center Report, Vol. 49, 1 (2019), 15--21.

[24]

Martin Längkvist, Lars Karlsson, and Amy Loutfi. 2012. Sleep Stage Classification Using Unsupervised Feature Learning. Adv. Artif. Neu. Sys., Vol. 2012 (2012), 5:5--5:5. http://dx.doi.org/10.1155/2012/107046

[25]

Erik Naylor, Daniel V Aillon, Seth Gabbert, Hans Harmon, David A Johnson, George S Wilson, and Peter A Petillo. 2011. Simultaneous real-time measurement of EEG/EMG and L-glutamate in mice: a biosensor study of neuronal activity during sleep. Journal of Electroanalytical Chemistry, Vol. 656, 1--2 (2011), 106--113.

[26]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. dtextquotesingle Alché-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf

Digital Library

[27]

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, Vol. 12 (2011), 2825--2830.

Digital Library

[28]

Mathias Perslev, Michael Jensen, Sune Darkner, Poul Jø rgen Jennum, and Christian Igel. 2019. U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 4415--4426. http://papers.nips.cc/paper/8692-u-time-a-fully-convolutional-network-for-time-series-segmentation-applied-to-sleep-staging.pdf

[29]

Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y Chén, and Maarten De Vos. 2019. SeqSleepNet: end-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 27, 3 (2019), 400--410.

[30]

Huy Phan, Kaare B Mikkelsen, Oliver Chen, Philipp Koch, Alfred Mertins, and Maarten De Vos. 2022. SleepTransformer: Automatic sleep staging with interpretability and uncertainty quantification. IEEE Transactions on Biomedical Engineering (2022).

[31]

Wei Qu, Zhiyong Wang, Hong Hong, Zheru Chi, David Dagan Feng, Ron Grunstein, and Christopher Gordon. 2020. A residual based attention model for eeg based sleep staging. IEEE journal of biomedical and health informatics, Vol. 24, 10 (2020), 2833--2843.

[32]

Allan Rechtschaffen. 1968. A manual for standardized terminology, techniques and scoring system for sleep stages in human subjects. Brain information service (1968).

[33]

Arnaud Sors, Stéphane Bonnet, Sébastien Mirek, Laurent Vercueil, and Jean-François Payen. 2018. A convolutional neural network for sleep stage scoring from raw single-channel EEG. Biomedical Signal Processing and Control, Vol. 42 (2018), 107--114.

[34]

Jens B. Stephansen, Alexander N. Olesen, Mads Olsen, Aditya Ambati, Eileen B. Leary, Hyatt E. Moore, Oscar Carrillo, Ling Lin, Fang Han, Han Yan, et al. 2018. Neural network analysis of sleep stages enables efficient diagnosis of narcolepsy. Nature Communications, Vol. 9, 1 (2018), 5229--5229.

[35]

Gregor Stiglic, Primoz Kocbek, Nino Fijacko, Marinka Zitnik, Katrien Verbert, and Leona Cilar. 2020. Interpretability of machine learning-based prediction models in healthcare. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, Vol. 10, 5 (2020), e1379.

[36]

A. Supratak, H. Dong, C. Wu, and Y. Guo. 2017. DeepSleepNet: A Model for Automatic Sleep Stage Scoring Based on Raw Single-Channel EEG. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 25, 11 (2017), 1998--2008. https://doi.org/10.1109/TNSRE.2017.2721116

[37]

Orestis Tsinalis, Paul M. Matthews, and Yike Guo. 2016. Automatic Sleep Stage Scoring Using Time-Frequency Analysis and Stacked Sparse Autoencoders. Annals of Biomedical Engineering, Vol. 44, 5 (2016), 1587--1597. https://doi.org/10.1007/s10439-015--1444-y

[38]

B Van Sweden, B Kemp, HAC Kamphuisen, and EA Van der Velde. 1990. Alternative electrode placement in (automatic) sleep scoring (f pz-cz/p z-oz versus c4-at). Sleep, Vol. 13, 3 (1990), 279--283.

[39]

Nathan W Whitmore, Adrianna M Bassard, and Ken A Paller. 2022. Targeted memory reactivation of face-name learning depends on ample and undisturbed slow-wave sleep. npj Science of Learning, Vol. 7, 1 (2022), 1--6.

[40]

Jenna Wiens, Suchi Saria, Mark Sendak, Marzyeh Ghassemi, Vincent X Liu, Finale Doshi-Velez, Kenneth Jung, Katherine Heller, David Kale, Mohammed Saeed, et al. 2019. Do no harm: a roadmap for responsible machine learning for health care. Nature medicine, Vol. 25, 9 (2019), 1337--1340.

[41]

Bufang Yang, Xilin Zhu, Yitian Liu, and Hongxing Liu. 2021. A single-channel EEG based automatic sleep stage classification method leveraging deep one-dimensional convolutional neural network and hidden Markov model. Biomedical Signal Processing and Control, Vol. 68 (2021), 102581.

[42]

Hanrui Zhang, Xueqing Wang, Hongyang Li, Soham Mehendale, and Yuanfang Guan. 2022. Auto-annotating sleep stages based on polysomnographic data. Patterns, Vol. 3, 1 (2022), 100371. https://doi.org/10.1016/j.patter.2021.100371

Cited By

Yu LZhang XQian SSun H(2025)DistillSleep: Leverage Self-distillation to Improve Performance After Representation Learning for Sleep StagingMultiMedia Modeling10.1007/978-981-96-2054-8_22(291-303)Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1007/978-981-96-2054-8_22
Liu HValderrama CZhang XLiu Q(2024)MPSleepNet: Matrix Profile-Guided Transformer for Multi-Channel Sleep ClassificationProceedings of the 2024 7th International Conference on Signal Processing and Machine Learning10.1145/3686490.3686519(195-201)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3686490.3686519
Soleimani RBarahona JChen YBozkurt ADaniele MPozdin VLobaton E(2023)Advances in Modeling and Interpretability of Deep Neural Sleep Staging: A Systematic ReviewPhysiologia10.3390/physiologia40100014:1(1-42)Online publication date: 20-Dec-2023
https://doi.org/10.3390/physiologia4010001
Show More Cited By

Recommendations

Interpretability of BERT Latent Space through Knowledge Graphs
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

The advent of pretrained language have renovated the ways of handling natural languages, improving the quality of systems that rely on them. BERT played a crucial role in revolutionizing the Natural Language Processing (NLP) area. However, the deep ...
Probing the Robustness of Pre-trained Language Models for Entity Matching
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

The paradigm of fine-tuning Pre-trained Language Models (PLMs) has been successful in Entity Matching (EM). Despite their remarkable performance, PLMs exhibit tendency to learn spurious correlations from training data. In this work, we aim at ...
TripJudge: A Relevance Judgement Test Collection for TripClick Health Retrieval
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Robust test collections are crucial for Information Retrieval research. Recently there is a growing interest in evaluating retrieval systems for domain-specific retrieval tasks, however these tasks often lack a reliable test collection with human-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

October 2022

5274 pages

ISBN:9781450392365

DOI:10.1145/3511808

General Chairs:
Mohammad Al Hasan
Indiana University Purdue University, Indianapolis, USA
,
Li Xiong
Emory University, Atlanta, USA

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Conference

CIKM '22

Sponsor:

CIKM '22: The 31st ACM International Conference on Information and Knowledge Management

October 17 - 21, 2022

GA, Atlanta, USA

Acceptance Rates

CIKM '22 Paper Acceptance Rate 621 of 2,257 submissions, 28%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
638
Total Downloads

Downloads (Last 12 months)211
Downloads (Last 6 weeks)24

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yu LZhang XQian SSun H(2025)DistillSleep: Leverage Self-distillation to Improve Performance After Representation Learning for Sleep StagingMultiMedia Modeling10.1007/978-981-96-2054-8_22(291-303)Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1007/978-981-96-2054-8_22
Liu HValderrama CZhang XLiu Q(2024)MPSleepNet: Matrix Profile-Guided Transformer for Multi-Channel Sleep ClassificationProceedings of the 2024 7th International Conference on Signal Processing and Machine Learning10.1145/3686490.3686519(195-201)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3686490.3686519
Soleimani RBarahona JChen YBozkurt ADaniele MPozdin VLobaton E(2023)Advances in Modeling and Interpretability of Deep Neural Sleep Staging: A Systematic ReviewPhysiologia10.3390/physiologia40100014:1(1-42)Online publication date: 20-Dec-2023
https://doi.org/10.3390/physiologia4010001
Al-Hussaini IMitchell C(2023)SeizFt: Interpretable Machine Learning for Seizure Detection Using WearablesBioengineering10.3390/bioengineering1008091810:8(918)Online publication date: 2-Aug-2023
https://doi.org/10.3390/bioengineering10080918
Al-Hussaini IMitchell C(2023)Towards Interpretable Seizure Detection Using WearablesICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10097091(1-2)Online publication date: 4-Jun-2023
https://doi.org/10.1109/ICASSP49357.2023.10097091

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten