Nothing Special   »   [go: up one dir, main page]

Skip to main content
Log in

Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

In sleep medicine, assessing the evolution of a subject’s sleep often involves the costly manual scoring of electroencephalographic (EEG) signals. In recent years, a number of Deep Learning approaches have been proposed to automate this process, mainly by extracting features from said signals. However, despite some promising developments in related problems, such as Brain–Computer Interfaces, analyses of the covariances between brain regions remain underutilized in sleep stage scoring. Expanding upon our previous work, we investigate the capabilities of SPDTransNet, a Transformer-derived network designed to classify sleep stages from EEG data through timeseries of covariance matrices. Furthermore, we present a novel way of integrating learned signal-wise features into said matrices without sacrificing their Symmetric Definite Positive (SPD) nature. Through comparison with other State-of-the-Art models within a methodology optimized for class-wise performance, we achieve a level of performance at or beyond various State-of-the-Art models, both in single-dataset and - particularly - multi-dataset experiments. In this article, we prove the capabilities of our SPDTransNet model, particularly its adaptability to multi-dataset tasks, within the context of EEG sleep stage scoring - though it could easily be adapted to any classification task involving timeseries of covariance matrices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data Availibility Statement

Our code is publicly available in the following repository: github.com/MathieuSeraphim/SPDTransNet_plus. The Montreal Archive of Sleep Studies (MASS) dataset is available upon request, pending approval from the Ethics Review Board (ERB) of the MASS host [66]. The Dreem DOD-H dataset is freely available online, and downloading instructions can be found in the authors’ GitHub repository [27].

Notes

  1. Electrical signals acquired from electrodes located around the brain, often non-invasively.

  2. As implemented in [61].

  3. As can be seen in the equivalent matrices computed for each recording and channel, available in our GitHub repository.

  4. Following the common convention of referring to the vector inputs of Transformer-based architectures as “tokens".

  5. Given a token size d and a number of attention heads h for a Transformer encoder, we found that imposing \(\frac{d}{h} \ge 32\) yielded better results.

  6. Here, the logarithm is defined from SPD(m) to Sym(m), and the exponential from Sym(p) to SPD(p).

  7. The largest possible number of epochs ignored by DeepSleepNet [14] at the end of a test set recording.

  8. All MASS-SS3 subjects use a linked-ear reference (LER), as do some MASS-SS1 subjects, the others using a computed linked-ear (CLE) reference.

  9. Located behind the ear in the opposite hemisphere.

References

  1. Sun R, Ma T, Liu S, Sathye M. Improved covariance matrix estimation for portfolio risk measurement: A review. Journal of Risk and Financial Management 2019;12(1) https://doi.org/10.3390/jrfm12010048

  2. Nguyen XS, Brun L, Lézoray O, Bougleux S A neural network based on spd manifold learning for skeleton-based hand gesture recognition. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019;12028–12037

  3. Bhatia, R. Positive Definite Matrices. Princeton University Press, (2007). http://www.jstor.org/stable/j.ctt7rxv2 Accessed 2024-01-12

  4. Pennec X. Manifold-valued image processing with spd matrices. In: Riemannian Geometric Statistics in Medical Image Analysis, pp. 75–134. Elsevier, 2020. https://doi.org/10.1016/B978-0-12-814725-2.00010-8

  5. Eickhoff SB, Müller VI. Functional connectivity. In: Toga AW, editor. Brain Mapping. Waltham: Academic Press; 2015. p. 187–201.

    Chapter  Google Scholar 

  6. You K, Park .J. Re-visiting riemannian geometry of symmetric positive definite matrices for the analysis of functional connectivity. NeuroImage 2021;225, 117464 https://doi.org/10.1016/j.neuroimage.2020.117464

  7. Yger F, Berar M, Lotte F. Riemannian approaches in brain-computer interfaces: A review. IEEE Trans Neural Syst Rehabil Eng. 2017;25(10):1753–62.

    Article  Google Scholar 

  8. Bouchard M, Lina J-M, Gaudreault P-O, Dubé J., Gosselin N, Carrier J. EEG connectivity across sleep cycles and age. Sleep 2019;43(3)

  9. Seraphim M, Dequidt P, Lechervy A, Yger F, Brun L, Etard O. Temporal sequences of eeg covariance matrices for automated sleep stage scoring with attention mechanisms. In: Tsapatsoulis N, Lanitis A, Pattichis M, Pattichis C, Kyrkou C, Kyriacou E, Theodosiou Z, Panayides A, editors. Computer Analysis of Images and Patterns. Cham: Springer; 2023. p. 67–76.

    Chapter  Google Scholar 

  10. Seraphim M, Lechervy A, Yger F, Brun L, Etard O Structure-preserving transformers for sequences of spd matrices. In: 2024 32nd European Signal Processing Conference (EUSIPCO) (2024). In press, preprint available at arxiv.org/pdf/2309.07579

  11. Berry RB, Brooks R, Gamaldo C, Harding SM, Lloyd RM, Quan SF, Troester MT, Vaughn BV. Aasm scoring manual updates for 2017 (version 2.4). J Clin Sleep Med. 2017;13(05):665–6. https://doi.org/10.5664/jcsm.6576.

    Article  Google Scholar 

  12. Chambon S, Galtier M, Arnal PJ, Wainrib G, Gramfort A. A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. IEEE Trans Neural Syst Rehabil Eng. 2018;26(4):17683810.

    Article  Google Scholar 

  13. Seo H, Back S, Lee S, Park D, Kim T, Lee K. Intra- and inter-epoch temporal context network (iitnet) using sub-epoch features for automatic sleep scoring on raw single-channel eeg. Biomed Signal Process Control. 2020;61: 102037.

    Article  Google Scholar 

  14. Supratak A, Dong H, Wu C, Guo Y. Deepsleepnet: a model for automatic sleep stage scoring based on raw single-channel eeg. IEEE Trans Neural Syst Rehabil Eng. 2017;25(11):1998–2008.

    Article  Google Scholar 

  15. Phan H, Andreotti F, Cooray N, Chén OY, De Vos M. Seqsleepnet: End-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging. IEEE Trans Neural Syst Rehabil Eng. 2019;27(3):400–10.

    Article  Google Scholar 

  16. Phan H, Mikkelsen K, Chén OY, Koch P, Mertins A, De Vos M. Sleeptransformer: Automatic sleep staging with interpretability and uncertainty quantification. IEEE Trans Biomed Eng. 2022;69(8):2456–67.

    Article  Google Scholar 

  17. Phan H, Chén OY, Tran MC, Koch P, Mertins A, De Vos M. Xsleepnet: Multi-view sequential model for automatic sleep staging. IEEE Trans Pattern Anal Mach Intell. 2022;44(9):5903–15.

    Google Scholar 

  18. Phan H, Lorenzen KP, Heremans E, Chén OY, Tran MC, Koch P, Mertins A, Baumert M, Mikkelsen KB, De Vos M. L-seqsleepnet: whole-cycle long sequence modelling for automatic sleep staging. IEEE J Biomed Health Inf. 2023. https://doi.org/10.1109/JBHI.2023.3303197.

    Article  Google Scholar 

  19. Phan H, Mikkelsen K. Automatic sleep staging of eeg signals: recent development, challenges, and future directions. Physiol Meas. 2022;43(4):04–01. https://doi.org/10.1088/1361-6579/ac6049.

    Article  Google Scholar 

  20. Phan H, Andreotti F, Cooray N, Chén O, Vos M. Joint classification and prediction cnn framework for automatic sleep stage classification. IEEE Trans Biomed Eng. 2019;66:1285–96.

    Article  Google Scholar 

  21. Perslev, M., Jensen, M., Darkner, S., Jennum, P.J.r., Igel, C.: U-time: A fully convolutional network for time series segmentation applied to sleep staging. In: Wallach, H., Larochelle, H., Beygelzimer, A., Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc., (2019). https://proceedings.neurips.cc/paper_files/paper/2019/file/57bafb2c2dfeefba931bb03a835b1fa9-Paper.pdf

  22. Perslev M, Darkner S, Kempfner L, Nikolic M, Jennum PJ, Igel C U-sleep: resilient high-frequency sleep staging. npj Digital Medicine 2021;4(1), 72 https://doi.org/10.1038/s41746-021-00440-5

  23. Jia Z, Lin Y, Wang J, Wang X, Xie P, Zhang Y Salientsleepnet: Multimodal salient wave detection network for sleep staging, pp. 2021;2614–2620 . https://doi.org/10.24963/ijcai.2021/360

  24. Ronneberger, O., P.Fischer, Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). LNCS, vol. 9351, pp. 234–241. Springer, (2015). (available on arXiv:1505.04597 [cs.CV]). http://lmb.informatik.uni-freiburg.de/Publications/2015/RFB15a

  25. Vilamala A, Madsen KH, Hansen LK (2017) Deep convolutional neural networks for interpretable analysis of eeg sleep stage scoring. In: 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6 . IEEE

  26. Dequidt P, Seraphim M, Lechervy A, Gaez II, Brun L, Etard O. Automatic sleep stage classification on eeg signals using time-frequency representation. In: Juarez JM, Marcos M, Stiglic G, Tucker A, editors. Artificial Intelligence in Medicine. Cham: Springer; 2023. p. 250–9.

    Chapter  Google Scholar 

  27. Guillot A, Sauvet F, During EH, Thorey V. Dreem open datasets: Multi-scored sleep datasets to compare human and automated sleep staging. IEEE Trans Neural Syst Rehabil Eng. 2020;28(9):1955–65. https://doi.org/10.1109/TNSRE.2020.3011181.

    Article  Google Scholar 

  28. Guillot A. Thorey, V Robustsleepnet: Transfer learning for automated sleep staging at scale. IEEE Trans Neural Syst Rehabil Eng. 2021;29:1441–51. https://doi.org/10.1109/TNSRE.2021.3098968.

    Article  Google Scholar 

  29. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I.: Attention is all you need. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc., (2017)

  30. Qu W, Wang Z, Hong H, Chi Z, Feng DD, Grunstein R, Gordon C. A residual based attention model for eeg based sleep staging. IEEE J Biomed Health Inform. 2020;24(10):2833–43.

    Article  Google Scholar 

  31. Zhu, T., Luo, W., Yu, F. Convolution-and Attention-Based Neural Network for Automated Sleep Stage Classification. Int J Environ Res Public Health 2020;17(11)

  32. Eldele, E., Chen, Z., Liu, C., Wu, M., Kwoh, C.-K., Li, X., Guan, C.: An attention-based deep learning approach for sleep stage classification with single-channel eeg. IEEE Transactions on Neural Systems and Rehabilitation Engineering 29, 809–818 (2021) https://doi.org/10.1109/TNSRE.2021.3076234

  33. Pradeepkumar, J., Anandakumar, M., Kugathasan, V., Suntharalingham, D., Kappel, S.L., Silva, A.C.D., Edussooriya, C.U.S.: Towards interpretable sleep stage classification using cross-modal transformers. ArXiv abs/2208.06991 (2022)

  34. Dai Y, Li X, Liang S, Wang L, Duan Q, Yang H, Zhang C, Chen X, Li L, Li X, Liao X. Multichannelsleepnet: A transformer-based model for automatic sleep stage classification with psg. IEEE J Biomed Health Inform. 2023;27(9):4204–15. https://doi.org/10.1109/JBHI.2023.3284160.

    Article  Google Scholar 

  35. Kemp B, Zwinderman AH, Tuk B, Kamphuisen HAC, Oberye JJL. Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the eeg. IEEE Trans Biomed Eng. 2000;47(9):1185–94. https://doi.org/10.1109/10.867928.

    Article  Google Scholar 

  36. Zhang G-Q, Cui L, Mueller R, Tao S, Kim M, Rueschman M, Mariani S, Mobley D, Redline S. The national sleep research resource: towards a sleep data commons. J Am Med Inform Assoc. 2018;25(10):1351–8.

    Article  Google Scholar 

  37. Quan SF, Howard BV, Iber C, Kiley JP, Nieto FJ, O’Connor GT, Rapoport DM, Redline S, Robbins J, Samet JM, Wahl PW. The sleep heart health study: design, rationale, and methods. Sleep. 1997;20(12):1077–85.

    Google Scholar 

  38. Jia, Z., Lin, Y., Wang, J., Zhou, R., Ning, X., He, Y., Zhao, Y.: Graphsleepnet: Adaptive spatial-temporal graph convolutional networks for sleep stage classification. In: IJCAI, pp. 1324–1330 (2020)

  39. Jia, Z., Lin, Y., Wang, J., Ning, X., He, Y., Zhou, R., Zhou, Y., Lehman, L.-w.H.: Multi-view spatial-temporal graph convolutional networks with domain generalization for sleep stage classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering 29, 1977–1986 (2021)

  40. Einizade, A., Nasiri, S., Sardouie, S.H., Clifford, G.D.: Productgraphsleepnet: Sleep staging using product spatio-temporal graph learning with attentive temporal aggregation. Neural Networks 164, 667–680 (2023) https://doi.org/10.1016/j.neunet.2023.05.016

  41. Arsigny V, Fillard P, Pennec X, Ayache N. Log-euclidean metrics for fast and simple calculus on diffusion tensors. Magn Reson Med. 2006;56(2):411–21.

    Article  Google Scholar 

  42. Pennec X, Fillard P, Ayache N. A riemannian framework for tensor computing. Int J Comput Vision. 2006;66(1):41–66.

    Article  Google Scholar 

  43. Barachant, A., Bonnet, S., Congedo, M., Jutten, C.: Classification of covariance matrices using a riemannian-based kernel for bci applications. Neurocomputing 112, 172–178 (2013) https://doi.org/10.1016/j.neucom.2012.12.039 . Advances in artificial neural networks, machine learning, and computational intelligence

  44. Yger, F., Sugiyama, M.: Supervised LogEuclidean Metric Learning for Symmetric Positive Definite Matrices. arXiv (2015)

  45. Huang, Z., Van Gool, L.: A riemannian network for spd matrix learning. In: Association for the Advancement of Artificial Intelligence (AAAI) (2017)

  46. Chakraborty R, Bouza J, Manton JH, Vemuri BC. Manifoldnet: A deep neural network for manifold-valued data with applications. IEEE Trans Pattern Anal Mach Intell. 2022;44(2):799–810. https://doi.org/10.1109/TPAMI.2020.3003846.

    Article  Google Scholar 

  47. Lotte F, Bougrain L, Cichocki A, Clerc M, Congedo M, Rakotomamonjy A, Yger F. A review of classification algorithms for eeg-based brain-computer interfaces: a 10 year update. J Neural Eng. 2018;15(3): 031005. https://doi.org/10.1088/1741-2552/aab2f2.

    Article  Google Scholar 

  48. Huang, Z., Wang, R., Shan, S., Li, X., Chen, X.: Log-euclidean metric learning on symmetric positive definite manifold with application to image set classification. In: Bach, F., Blei, D. (eds.) Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research, 2015;37:720–729. PMLR, Lille, France https://proceedings.mlr.press/v37/huanga15.html

  49. Tang Y, Chen D, Wu J, Tu W, Monaghan JJM, Sowman P, Mcalpine D. Functional connectivity learning via siamese-based spd matrix representation of brain imaging data. Neural Netw. 2023;163:272–85. https://doi.org/10.1016/j.neunet.2023.04.004.

    Article  Google Scholar 

  50. Peng Z, Li H, Zhao D, Pan C. Reducing the dimensionality of spd matrices with neural networks in bci. Mathematics 2023;11(7) https://doi.org/10.3390/math11071570

  51. Lu J, Tian Y, Zhang Y, Ge J, Sheng QZ, Zheng X LGL-BCI: A Lightweight Geometric Learning Framework for Motor Imagery-Based Brain-Computer Interfaces 2023

  52. Ouahidi YE, Gripo, V, Pasdeloup B, Bouallegue G, Farrugia N, Lioi G A Strong and Simple Deep Learning Baseline for BCI MI Decoding (2024)

  53. Abibullaev, B., Keutayeva, A., Zollanvari, A.: Deep learning in eeg-based bcis: A comprehensive review of transformer models, advantages, challenges, and applications. IEEE Access 11, 127271–127301 (2023) https://doi.org/10.1109/ACCESS.2023.3329678

  54. Konstantinidis D, Papastratis I, Dimitropoulos K, Daras P. Multi-manifold attention for vision transformers. IEEE Access. 2023;11:123433–44. https://doi.org/10.1109/ACCESS.2023.3329952.

    Article  Google Scholar 

  55. Dong Z, Wang Q, Zhu P. Multi-head second-order pooling for graph transformer networks. Pattern Recogn Lett. 2023;167:53–9. https://doi.org/10.1016/j.patrec.2023.01.017.

    Article  Google Scholar 

  56. He L, Dong Y, Wang Y, Tao D, Lin Z Gauge equivariant transformer. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P.S., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems, vol. 34, pp. 27331–27343. Curran Associates, Inc., 2021. https://proceedings.neurips.cc/paper_files/paper/2021/file/e57c6b956a6521b28495f2886ca0977a-Paper.pdf

  57. Li Z, TANG X, Xu Z, Wang X, Yu H, Chen M, Wei X. Geodesic self-attention for 3d point clouds. In: Oh, A.H., Agarwal, A., Belgrave, D., Cho, K. (eds.) Advances in Neural Information Processing Systems 2022. https://openreview.net/forum?id=2ndfW2bw4mi

  58. Kratsios, A., Zamanlooy, B., Liu, T., Dokmanić, I.: Universal approximation under constraints is possible with transformers. In: International Conference on Learning Representations (2022). https://openreview.net/forum?id=JGO8CvG5S9

  59. Pan, Y-T, Chou J-L, Wei C-S. MAtt: A manifold attention network for EEG decoding. In: Oh, A.H., Agarwal, A., Belgrave, D., Cho, K. (eds.) Advances in Neural Information Processing Systems (2022)

  60. Qin, R., Song, Z., Ren, H., Pei, Z., Zhu, L., Shi, X., Guo, Y., Liu, H., Zhang, M., Zhang, Z.: Bnmtrans: A brain network sequence-driven manifold-based transformer for cognitive impairment detection using eeg. In: ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2016–2020 (2024). https://doi.org/10.1109/ICASSP48485.2024.10447106

  61. Barachant, A., Barthélemy, Q., King, J.-R., Gramfort, A., Chevallier, S., Rodrigues, P.L.C., Olivetti, E., Goncharenko, V., Berg, G.W., Reguig, G., Lebeurrier, A., Bjäreholt, E., Yamamoto, M.S., Clisson, P., Corsi, M.-C.: pyRiemann/pyRiemann: V0.5. https://doi.org/10.5281/zenodo.8059038

  62. Song, Y., Sebe, N., Wang, W.: Why approximate matrix square root outperforms accurate svd in global covariance pooling? In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1095–1103 (2021). https://doi.org/10.1109/ICCV48922.2021.00115

  63. Wang, G., Lu, Y., Cui, L., Lv, T., Florencio, D., Zhang, C.: A simple yet effective learnable positional encoding method for improving document transformer model. In: He, Y., Ji, H., Li, S., Liu, Y., Chang, C.-H. (eds.) Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, pp. 453–463. Association for Computational Linguistics, Online only (2022). https://aclanthology.org/2022.findings-aacl.42

  64. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. 2016. https://doi.org/10.1109/CVPR.2016.308.

    Article  Google Scholar 

  65. Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: A next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2623–2631 (2019)

  66. O’reilly C, Gosselin N, Carrier J, Nielsen T. Montreal archive of sleep studies: an open-access resource for instrument benchmarking and exploratory research. J Sleep Res. 2014;23(6):628–35.

    Article  Google Scholar 

Download references

Acknowledgements

This work was granted access to the HPC resources of IDRIS (Institut du Développement et des Ressources en Informatique Scientifique) under the allocation 2022-AD010613618 made by GENCI (Grand Ëquipement National de Calcul Intensif), and to the computing resources of CRIANN (Centre Régional Informatique et d’Applications Numériques de Normandie, Normandy, France).

Funding

This work was granted access to computing resources of IDRIS and CRIANN (see Acknoledgements). Mathieu Seraphim is supported by the French National Research Agency (ANR) and Region Normandie under grant HAISCoDe. This work was partially carried out during a CNRS leave at GREYC of Florian Yger.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the study conception and design. Experiments were performed by Mathieu Seraphim, under the supervision of Alexis Lechervy, Florian Yger, Luc Brun and Olivier Etard. The first draft of the manuscript was written by Mathieu Seraphim and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mathieu Seraphim.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Research Involving Human

Both utilized datasets are composed of anonymized recordings of human biological signals. The MASS recordings were anonymized and distributed in accordance with guidelines from the MASS research team’s ethics board [66]. The Dreem DOD-H dataset was compiled and made publicly available with approval from the Committees of Protection of Persons (CPP), and declared and carried out in accordance with French law [27]. No further biological data was utilized in the production of this paper.

Informed Consent

Not applicable (see above).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 368 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Seraphim, M., Lechervy, A., Yger, F. et al. Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks. SN COMPUT. SCI. 5, 953 (2024). https://doi.org/10.1007/s42979-024-03310-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-024-03310-5

Keywords

Navigation