Abstract
In sleep medicine, assessing the evolution of a subject’s sleep often involves the costly manual scoring of electroencephalographic (EEG) signals. In recent years, a number of Deep Learning approaches have been proposed to automate this process, mainly by extracting features from said signals. However, despite some promising developments in related problems, such as Brain–Computer Interfaces, analyses of the covariances between brain regions remain underutilized in sleep stage scoring. Expanding upon our previous work, we investigate the capabilities of SPDTransNet, a Transformer-derived network designed to classify sleep stages from EEG data through timeseries of covariance matrices. Furthermore, we present a novel way of integrating learned signal-wise features into said matrices without sacrificing their Symmetric Definite Positive (SPD) nature. Through comparison with other State-of-the-Art models within a methodology optimized for class-wise performance, we achieve a level of performance at or beyond various State-of-the-Art models, both in single-dataset and - particularly - multi-dataset experiments. In this article, we prove the capabilities of our SPDTransNet model, particularly its adaptability to multi-dataset tasks, within the context of EEG sleep stage scoring - though it could easily be adapted to any classification task involving timeseries of covariance matrices.
Similar content being viewed by others
Data Availibility Statement
Our code is publicly available in the following repository: github.com/MathieuSeraphim/SPDTransNet_plus. The Montreal Archive of Sleep Studies (MASS) dataset is available upon request, pending approval from the Ethics Review Board (ERB) of the MASS host [66]. The Dreem DOD-H dataset is freely available online, and downloading instructions can be found in the authors’ GitHub repository [27].
Notes
Electrical signals acquired from electrodes located around the brain, often non-invasively.
As implemented in [61].
As can be seen in the equivalent matrices computed for each recording and channel, available in our GitHub repository.
Following the common convention of referring to the vector inputs of Transformer-based architectures as “tokens".
Given a token size d and a number of attention heads h for a Transformer encoder, we found that imposing \(\frac{d}{h} \ge 32\) yielded better results.
Here, the logarithm is defined from SPD(m) to Sym(m), and the exponential from Sym(p) to SPD(p).
The largest possible number of epochs ignored by DeepSleepNet [14] at the end of a test set recording.
All MASS-SS3 subjects use a linked-ear reference (LER), as do some MASS-SS1 subjects, the others using a computed linked-ear (CLE) reference.
Located behind the ear in the opposite hemisphere.
References
Sun R, Ma T, Liu S, Sathye M. Improved covariance matrix estimation for portfolio risk measurement: A review. Journal of Risk and Financial Management 2019;12(1) https://doi.org/10.3390/jrfm12010048
Nguyen XS, Brun L, Lézoray O, Bougleux S A neural network based on spd manifold learning for skeleton-based hand gesture recognition. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019;12028–12037
Bhatia, R. Positive Definite Matrices. Princeton University Press, (2007). http://www.jstor.org/stable/j.ctt7rxv2 Accessed 2024-01-12
Pennec X. Manifold-valued image processing with spd matrices. In: Riemannian Geometric Statistics in Medical Image Analysis, pp. 75–134. Elsevier, 2020. https://doi.org/10.1016/B978-0-12-814725-2.00010-8
Eickhoff SB, Müller VI. Functional connectivity. In: Toga AW, editor. Brain Mapping. Waltham: Academic Press; 2015. p. 187–201.
You K, Park .J. Re-visiting riemannian geometry of symmetric positive definite matrices for the analysis of functional connectivity. NeuroImage 2021;225, 117464 https://doi.org/10.1016/j.neuroimage.2020.117464
Yger F, Berar M, Lotte F. Riemannian approaches in brain-computer interfaces: A review. IEEE Trans Neural Syst Rehabil Eng. 2017;25(10):1753–62.
Bouchard M, Lina J-M, Gaudreault P-O, Dubé J., Gosselin N, Carrier J. EEG connectivity across sleep cycles and age. Sleep 2019;43(3)
Seraphim M, Dequidt P, Lechervy A, Yger F, Brun L, Etard O. Temporal sequences of eeg covariance matrices for automated sleep stage scoring with attention mechanisms. In: Tsapatsoulis N, Lanitis A, Pattichis M, Pattichis C, Kyrkou C, Kyriacou E, Theodosiou Z, Panayides A, editors. Computer Analysis of Images and Patterns. Cham: Springer; 2023. p. 67–76.
Seraphim M, Lechervy A, Yger F, Brun L, Etard O Structure-preserving transformers for sequences of spd matrices. In: 2024 32nd European Signal Processing Conference (EUSIPCO) (2024). In press, preprint available at arxiv.org/pdf/2309.07579
Berry RB, Brooks R, Gamaldo C, Harding SM, Lloyd RM, Quan SF, Troester MT, Vaughn BV. Aasm scoring manual updates for 2017 (version 2.4). J Clin Sleep Med. 2017;13(05):665–6. https://doi.org/10.5664/jcsm.6576.
Chambon S, Galtier M, Arnal PJ, Wainrib G, Gramfort A. A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. IEEE Trans Neural Syst Rehabil Eng. 2018;26(4):17683810.
Seo H, Back S, Lee S, Park D, Kim T, Lee K. Intra- and inter-epoch temporal context network (iitnet) using sub-epoch features for automatic sleep scoring on raw single-channel eeg. Biomed Signal Process Control. 2020;61: 102037.
Supratak A, Dong H, Wu C, Guo Y. Deepsleepnet: a model for automatic sleep stage scoring based on raw single-channel eeg. IEEE Trans Neural Syst Rehabil Eng. 2017;25(11):1998–2008.
Phan H, Andreotti F, Cooray N, Chén OY, De Vos M. Seqsleepnet: End-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging. IEEE Trans Neural Syst Rehabil Eng. 2019;27(3):400–10.
Phan H, Mikkelsen K, Chén OY, Koch P, Mertins A, De Vos M. Sleeptransformer: Automatic sleep staging with interpretability and uncertainty quantification. IEEE Trans Biomed Eng. 2022;69(8):2456–67.
Phan H, Chén OY, Tran MC, Koch P, Mertins A, De Vos M. Xsleepnet: Multi-view sequential model for automatic sleep staging. IEEE Trans Pattern Anal Mach Intell. 2022;44(9):5903–15.
Phan H, Lorenzen KP, Heremans E, Chén OY, Tran MC, Koch P, Mertins A, Baumert M, Mikkelsen KB, De Vos M. L-seqsleepnet: whole-cycle long sequence modelling for automatic sleep staging. IEEE J Biomed Health Inf. 2023. https://doi.org/10.1109/JBHI.2023.3303197.
Phan H, Mikkelsen K. Automatic sleep staging of eeg signals: recent development, challenges, and future directions. Physiol Meas. 2022;43(4):04–01. https://doi.org/10.1088/1361-6579/ac6049.
Phan H, Andreotti F, Cooray N, Chén O, Vos M. Joint classification and prediction cnn framework for automatic sleep stage classification. IEEE Trans Biomed Eng. 2019;66:1285–96.
Perslev, M., Jensen, M., Darkner, S., Jennum, P.J.r., Igel, C.: U-time: A fully convolutional network for time series segmentation applied to sleep staging. In: Wallach, H., Larochelle, H., Beygelzimer, A., Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc., (2019). https://proceedings.neurips.cc/paper_files/paper/2019/file/57bafb2c2dfeefba931bb03a835b1fa9-Paper.pdf
Perslev M, Darkner S, Kempfner L, Nikolic M, Jennum PJ, Igel C U-sleep: resilient high-frequency sleep staging. npj Digital Medicine 2021;4(1), 72 https://doi.org/10.1038/s41746-021-00440-5
Jia Z, Lin Y, Wang J, Wang X, Xie P, Zhang Y Salientsleepnet: Multimodal salient wave detection network for sleep staging, pp. 2021;2614–2620 . https://doi.org/10.24963/ijcai.2021/360
Ronneberger, O., P.Fischer, Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). LNCS, vol. 9351, pp. 234–241. Springer, (2015). (available on arXiv:1505.04597 [cs.CV]). http://lmb.informatik.uni-freiburg.de/Publications/2015/RFB15a
Vilamala A, Madsen KH, Hansen LK (2017) Deep convolutional neural networks for interpretable analysis of eeg sleep stage scoring. In: 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6 . IEEE
Dequidt P, Seraphim M, Lechervy A, Gaez II, Brun L, Etard O. Automatic sleep stage classification on eeg signals using time-frequency representation. In: Juarez JM, Marcos M, Stiglic G, Tucker A, editors. Artificial Intelligence in Medicine. Cham: Springer; 2023. p. 250–9.
Guillot A, Sauvet F, During EH, Thorey V. Dreem open datasets: Multi-scored sleep datasets to compare human and automated sleep staging. IEEE Trans Neural Syst Rehabil Eng. 2020;28(9):1955–65. https://doi.org/10.1109/TNSRE.2020.3011181.
Guillot A. Thorey, V Robustsleepnet: Transfer learning for automated sleep staging at scale. IEEE Trans Neural Syst Rehabil Eng. 2021;29:1441–51. https://doi.org/10.1109/TNSRE.2021.3098968.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I.: Attention is all you need. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc., (2017)
Qu W, Wang Z, Hong H, Chi Z, Feng DD, Grunstein R, Gordon C. A residual based attention model for eeg based sleep staging. IEEE J Biomed Health Inform. 2020;24(10):2833–43.
Zhu, T., Luo, W., Yu, F. Convolution-and Attention-Based Neural Network for Automated Sleep Stage Classification. Int J Environ Res Public Health 2020;17(11)
Eldele, E., Chen, Z., Liu, C., Wu, M., Kwoh, C.-K., Li, X., Guan, C.: An attention-based deep learning approach for sleep stage classification with single-channel eeg. IEEE Transactions on Neural Systems and Rehabilitation Engineering 29, 809–818 (2021) https://doi.org/10.1109/TNSRE.2021.3076234
Pradeepkumar, J., Anandakumar, M., Kugathasan, V., Suntharalingham, D., Kappel, S.L., Silva, A.C.D., Edussooriya, C.U.S.: Towards interpretable sleep stage classification using cross-modal transformers. ArXiv abs/2208.06991 (2022)
Dai Y, Li X, Liang S, Wang L, Duan Q, Yang H, Zhang C, Chen X, Li L, Li X, Liao X. Multichannelsleepnet: A transformer-based model for automatic sleep stage classification with psg. IEEE J Biomed Health Inform. 2023;27(9):4204–15. https://doi.org/10.1109/JBHI.2023.3284160.
Kemp B, Zwinderman AH, Tuk B, Kamphuisen HAC, Oberye JJL. Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the eeg. IEEE Trans Biomed Eng. 2000;47(9):1185–94. https://doi.org/10.1109/10.867928.
Zhang G-Q, Cui L, Mueller R, Tao S, Kim M, Rueschman M, Mariani S, Mobley D, Redline S. The national sleep research resource: towards a sleep data commons. J Am Med Inform Assoc. 2018;25(10):1351–8.
Quan SF, Howard BV, Iber C, Kiley JP, Nieto FJ, O’Connor GT, Rapoport DM, Redline S, Robbins J, Samet JM, Wahl PW. The sleep heart health study: design, rationale, and methods. Sleep. 1997;20(12):1077–85.
Jia, Z., Lin, Y., Wang, J., Zhou, R., Ning, X., He, Y., Zhao, Y.: Graphsleepnet: Adaptive spatial-temporal graph convolutional networks for sleep stage classification. In: IJCAI, pp. 1324–1330 (2020)
Jia, Z., Lin, Y., Wang, J., Ning, X., He, Y., Zhou, R., Zhou, Y., Lehman, L.-w.H.: Multi-view spatial-temporal graph convolutional networks with domain generalization for sleep stage classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering 29, 1977–1986 (2021)
Einizade, A., Nasiri, S., Sardouie, S.H., Clifford, G.D.: Productgraphsleepnet: Sleep staging using product spatio-temporal graph learning with attentive temporal aggregation. Neural Networks 164, 667–680 (2023) https://doi.org/10.1016/j.neunet.2023.05.016
Arsigny V, Fillard P, Pennec X, Ayache N. Log-euclidean metrics for fast and simple calculus on diffusion tensors. Magn Reson Med. 2006;56(2):411–21.
Pennec X, Fillard P, Ayache N. A riemannian framework for tensor computing. Int J Comput Vision. 2006;66(1):41–66.
Barachant, A., Bonnet, S., Congedo, M., Jutten, C.: Classification of covariance matrices using a riemannian-based kernel for bci applications. Neurocomputing 112, 172–178 (2013) https://doi.org/10.1016/j.neucom.2012.12.039 . Advances in artificial neural networks, machine learning, and computational intelligence
Yger, F., Sugiyama, M.: Supervised LogEuclidean Metric Learning for Symmetric Positive Definite Matrices. arXiv (2015)
Huang, Z., Van Gool, L.: A riemannian network for spd matrix learning. In: Association for the Advancement of Artificial Intelligence (AAAI) (2017)
Chakraborty R, Bouza J, Manton JH, Vemuri BC. Manifoldnet: A deep neural network for manifold-valued data with applications. IEEE Trans Pattern Anal Mach Intell. 2022;44(2):799–810. https://doi.org/10.1109/TPAMI.2020.3003846.
Lotte F, Bougrain L, Cichocki A, Clerc M, Congedo M, Rakotomamonjy A, Yger F. A review of classification algorithms for eeg-based brain-computer interfaces: a 10 year update. J Neural Eng. 2018;15(3): 031005. https://doi.org/10.1088/1741-2552/aab2f2.
Huang, Z., Wang, R., Shan, S., Li, X., Chen, X.: Log-euclidean metric learning on symmetric positive definite manifold with application to image set classification. In: Bach, F., Blei, D. (eds.) Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research, 2015;37:720–729. PMLR, Lille, France https://proceedings.mlr.press/v37/huanga15.html
Tang Y, Chen D, Wu J, Tu W, Monaghan JJM, Sowman P, Mcalpine D. Functional connectivity learning via siamese-based spd matrix representation of brain imaging data. Neural Netw. 2023;163:272–85. https://doi.org/10.1016/j.neunet.2023.04.004.
Peng Z, Li H, Zhao D, Pan C. Reducing the dimensionality of spd matrices with neural networks in bci. Mathematics 2023;11(7) https://doi.org/10.3390/math11071570
Lu J, Tian Y, Zhang Y, Ge J, Sheng QZ, Zheng X LGL-BCI: A Lightweight Geometric Learning Framework for Motor Imagery-Based Brain-Computer Interfaces 2023
Ouahidi YE, Gripo, V, Pasdeloup B, Bouallegue G, Farrugia N, Lioi G A Strong and Simple Deep Learning Baseline for BCI MI Decoding (2024)
Abibullaev, B., Keutayeva, A., Zollanvari, A.: Deep learning in eeg-based bcis: A comprehensive review of transformer models, advantages, challenges, and applications. IEEE Access 11, 127271–127301 (2023) https://doi.org/10.1109/ACCESS.2023.3329678
Konstantinidis D, Papastratis I, Dimitropoulos K, Daras P. Multi-manifold attention for vision transformers. IEEE Access. 2023;11:123433–44. https://doi.org/10.1109/ACCESS.2023.3329952.
Dong Z, Wang Q, Zhu P. Multi-head second-order pooling for graph transformer networks. Pattern Recogn Lett. 2023;167:53–9. https://doi.org/10.1016/j.patrec.2023.01.017.
He L, Dong Y, Wang Y, Tao D, Lin Z Gauge equivariant transformer. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P.S., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems, vol. 34, pp. 27331–27343. Curran Associates, Inc., 2021. https://proceedings.neurips.cc/paper_files/paper/2021/file/e57c6b956a6521b28495f2886ca0977a-Paper.pdf
Li Z, TANG X, Xu Z, Wang X, Yu H, Chen M, Wei X. Geodesic self-attention for 3d point clouds. In: Oh, A.H., Agarwal, A., Belgrave, D., Cho, K. (eds.) Advances in Neural Information Processing Systems 2022. https://openreview.net/forum?id=2ndfW2bw4mi
Kratsios, A., Zamanlooy, B., Liu, T., Dokmanić, I.: Universal approximation under constraints is possible with transformers. In: International Conference on Learning Representations (2022). https://openreview.net/forum?id=JGO8CvG5S9
Pan, Y-T, Chou J-L, Wei C-S. MAtt: A manifold attention network for EEG decoding. In: Oh, A.H., Agarwal, A., Belgrave, D., Cho, K. (eds.) Advances in Neural Information Processing Systems (2022)
Qin, R., Song, Z., Ren, H., Pei, Z., Zhu, L., Shi, X., Guo, Y., Liu, H., Zhang, M., Zhang, Z.: Bnmtrans: A brain network sequence-driven manifold-based transformer for cognitive impairment detection using eeg. In: ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2016–2020 (2024). https://doi.org/10.1109/ICASSP48485.2024.10447106
Barachant, A., Barthélemy, Q., King, J.-R., Gramfort, A., Chevallier, S., Rodrigues, P.L.C., Olivetti, E., Goncharenko, V., Berg, G.W., Reguig, G., Lebeurrier, A., Bjäreholt, E., Yamamoto, M.S., Clisson, P., Corsi, M.-C.: pyRiemann/pyRiemann: V0.5. https://doi.org/10.5281/zenodo.8059038
Song, Y., Sebe, N., Wang, W.: Why approximate matrix square root outperforms accurate svd in global covariance pooling? In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1095–1103 (2021). https://doi.org/10.1109/ICCV48922.2021.00115
Wang, G., Lu, Y., Cui, L., Lv, T., Florencio, D., Zhang, C.: A simple yet effective learnable positional encoding method for improving document transformer model. In: He, Y., Ji, H., Li, S., Liu, Y., Chang, C.-H. (eds.) Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, pp. 453–463. Association for Computational Linguistics, Online only (2022). https://aclanthology.org/2022.findings-aacl.42
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. 2016. https://doi.org/10.1109/CVPR.2016.308.
Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: A next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2623–2631 (2019)
O’reilly C, Gosselin N, Carrier J, Nielsen T. Montreal archive of sleep studies: an open-access resource for instrument benchmarking and exploratory research. J Sleep Res. 2014;23(6):628–35.
Acknowledgements
This work was granted access to the HPC resources of IDRIS (Institut du Développement et des Ressources en Informatique Scientifique) under the allocation 2022-AD010613618 made by GENCI (Grand Ëquipement National de Calcul Intensif), and to the computing resources of CRIANN (Centre Régional Informatique et d’Applications Numériques de Normandie, Normandy, France).
Funding
This work was granted access to computing resources of IDRIS and CRIANN (see Acknoledgements). Mathieu Seraphim is supported by the French National Research Agency (ANR) and Region Normandie under grant HAISCoDe. This work was partially carried out during a CNRS leave at GREYC of Florian Yger.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Experiments were performed by Mathieu Seraphim, under the supervision of Alexis Lechervy, Florian Yger, Luc Brun and Olivier Etard. The first draft of the manuscript was written by Mathieu Seraphim and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Research Involving Human
Both utilized datasets are composed of anonymized recordings of human biological signals. The MASS recordings were anonymized and distributed in accordance with guidelines from the MASS research team’s ethics board [66]. The Dreem DOD-H dataset was compiled and made publicly available with approval from the Committees of Protection of Persons (CPP), and declared and carried out in accordance with French law [27]. No further biological data was utilized in the production of this paper.
Informed Consent
Not applicable (see above).
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Seraphim, M., Lechervy, A., Yger, F. et al. Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks. SN COMPUT. SCI. 5, 953 (2024). https://doi.org/10.1007/s42979-024-03310-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s42979-024-03310-5