Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Drowsiness detection system using deep learning based data fusion approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Drowsy driving is a major cause of road accidents worldwide, and detecting drowsiness in drivers is critical to improving road safety. In this work, we developed a system for detecting drowsiness using Electroencephalogram and Electrocardiogram signals and a combination of 2D convolutional neural networks and a fuzzy neural network. The Electroencephalogram and Electrocardiogram signals were processed using convolutional neural networks to extract features, and a fuzzy neural network was used to classify the features and detect drowsiness. Our results show that our system can detect drowsiness with a high degree of accuracy, making it a reliable and efficient tool for improving road safety. The proposed system helps to reduce the number of accidents caused by drowsy driving and improve road safety for all drivers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The EEG and ECG Data collected from the subjects is meant for study purposes only and hence they cannot be shared as public dataset without the consent from all the subjects. Moreover, the collected will be benchmarked at a later stage.

References

  1. Idogawa K (1991) On the brain wave activity of professional drivers during monotonous work. Behaviormetrika 18(30):23–34. https://doi.org/10.2333/bhmk.18.30_23

    Article  Google Scholar 

  2. Lal SKL, Craig A (2001) A critical review of the psychophysiology of driver fatigue. Biol Psychol 55(3):173–194. https://doi.org/10.1016/s0301-0511(00)00085-5

    Article  Google Scholar 

  3. Lal SKL, Craig A (2001) Electroencephalography activity associated with driver fatigue: implications for a fatigue countermeasure device. J Psychophysiol 15(3):183–189. https://doi.org/10.1027/0269-8803.15.3.183

    Article  Google Scholar 

  4. Lin FC, Ko LW, Chuang CH, Su TP, Lin CT (2012) Generalized EEG-Based drowsiness prediction system by using a Self-Organizing neural fuzzy system. IEEE Trans Circuits Syst I Regul Pap 59(9):2044–2055. https://doi.org/10.1109/tcsi.2012.2185290

    Article  MathSciNet  Google Scholar 

  5. Lin C-T, Li-Wei K (2013) EEG-based learning system for online motion sickness level estimation in a dynamic vehicle environment. IEEE Trans Neural Netw Learn Syst 24(10):1689–1700. https://doi.org/10.1109/tnnls.2013.2275003

    Article  Google Scholar 

  6. Chuang CH, Huang CS, Ko LW, Lin CT (2015) An EEG-based perceptual function integration network for application to drowsy driving. Knowl Based Syst 80:143–152. https://doi.org/10.1016/j.knosys.2015.01.007

    Article  Google Scholar 

  7. Liu YT, Lin YY, Wu SL, Chuang CH, Lin CT (2016) Brain dynamics in predicting driving fatigue using a recurrent self-evolving fuzzy neural network. IEEE Trans Neural Netw Learn Syst 27(2):347–360. https://doi.org/10.1109/tnnls.2015.2496330

    Article  Google Scholar 

  8. Zhou M, Yu Y, Qu X (2019) Development of an efficient driving strategy for connected and automated vehicles at signalized intersections: a reinforcement learning approach. IEEE Trans Intell Transp Syst 1–11. https://doi.org/10.1109/tits.2019.2942014

  9. Zhang D, Yao L, Chen K, Wang S, Chang X, Liu Y (2019) Making sense of spatio-temporal preserving representations for EEG-based human intention recognition. IEEE Trans Cybern 1–12. https://doi.org/10.1109/tcyb.2019.2905157

  10. Kwak NS, Lee SW (2020) Error correction regression framework for enhancing the decoding accuracies of Ear-EEG brain–computer interfaces. IEEE Trans Cybern 50(8):3654–3667. https://doi.org/10.1109/TCYB.2019.2924237

    Article  Google Scholar 

  11. Dai C, Wu J, Pi D, Becker SI, Cui L, Zhang Q, Johnson B (2020) Brain EEG time-series clustering using maximum-weight clique. IEEE Trans Cybern 1–15. https://doi.org/10.1109/tcyb.2020.2974776

  12. Castiblanco Jimenez IA, Gomez Acevedo JS, Olivetti EC, Marcolin F, Ulrich L, Moos S, Vezzetti E (2022) User engagement comparison between advergames and traditional advertising using EEG: does the user’s engagement influence purchase intention? Electronics 12(1). https://doi.org/10.3390/electronics12010122

  13. You SD (2021) Classification of relaxation and concentration mental states with EEG. Information 12(5):187. https://doi.org/10.3390/info12050187

    Article  Google Scholar 

  14. Kobayashi H, Ishibashi K, Noguchi H (1999) Heart rate variability; an index for monitoring and analyzing human autonomic activities. Appl Hum Sci J Physiol Anthropol 18(2):53–59. https://doi.org/10.2114/jpa.18.53

    Article  Google Scholar 

  15. Patel M, Lal SKL, Kavanagh D, Rossiter P (2011) Applying neural network analysis on heart rate variability data to assess driver fatigue. Expert Syst Appl 38(6):7235–7242. https://doi.org/10.1016/j.eswa.2010.12.028

    Article  Google Scholar 

  16. Jung SJ, Shin HS, Chung WY (2014) Driver fatigue and drowsiness monitoring system with embedded electrocardiogram sensor on steering wheel. IET Intel Transport Syst 8(1):43–50. https://doi.org/10.1049/iet-its.2012.0032

    Article  Google Scholar 

  17. Delliaux S, Delaforge A, Deharo J-C, Chaumet G (2019) Mental workload alters heart rate variability, lowering non-linear dynamics. Front Physiol 10:565. https://doi.org/10.3389/fphys.2019.00565

    Article  Google Scholar 

  18. Ladino Nocua AC, Cruz Gonzalez JP, Castiblanco Jimenez IA, Gomez Acevedo JS, Marcolin F, Vezzetti E (2021) Assessment of cognitive student engagement using heart rate data in distance learning during COVID-19. Educ Sci 11:540. https://doi.org/10.3390/educsci11090540

    Article  Google Scholar 

  19. Jung J, Lim S, Kim B, Lee S (2021) CNN-based driver monitoring using millimeter-wave radar sensor. IEEE Sensors Lett 5(3):3500404

    Article  Google Scholar 

  20. Liu YT, Lin YY, Wu SL, Chuang CH, Prasad M, Lin C-T (2014) EEG-based driving fatigue prediction system using functional-link based fuzzy neural network. In: Proc. Int. Joint Conf. Neural Netw, pp 4109–4113. https://doi.org/10.1109/IJCNN.2014.6889736

  21. Jafarifarmand A, Badamchizadeh MA, Khanmohammadi S, Nazari MA, Tazehkand BM (2018) A new self-regulated neuro-fuzzy framework for classification of EEG signals in motor imagery BCI. IEEE Trans Fuzzy Syst 26(3):1485–1497. https://doi.org/10.1109/TFUZZ.2017.2728521

    Article  Google Scholar 

  22. Du G, Long S, Li C, Wang Z, Liu PX (2023) A product fuzzy convolutional network for detecting driving fatigue. IEEE Trans Cybern 53(7):4175–4188. https://doi.org/10.1109/TCYB.2021.3123842

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Mepco Schlenk Engineering College, Sivakasi, 626005, Tamil Nadu, India

    G. Yogarajan, R. Nitin Singh, S. Avudai Nandhu & R. Mohana Rudhran

Authors
  1. G. Yogarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Nitin Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Avudai Nandhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Mohana Rudhran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. Yogarajan.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yogarajan, G., Singh, R.N., Nandhu, S.A. et al. Drowsiness detection system using deep learning based data fusion approach. Multimed Tools Appl 83, 36081–36095 (2024). https://doi.org/10.1007/s11042-023-17096-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17096-w

Keywords

Search

Navigation