Abstract
The eye are a very important organ in the human body. The eye area and eyes contain lots of useful information about human interaction with the environment. Many studies have relied on eye region analyzes to build the medical care, surveillance, interaction, security, and warning systems. This paper focuses on extracting eye region features to detect eye state using the light-weight convolutional neural networks with two stages: eye detection and classification. This method can apply on simple drowsiness warning system and perform well on Intel Core I7-4770 CPU @ 3.40 GHz (Personal Computer - PC) and on quad-core ARM Cortex-A57 CPU (Jetson Nano device) with 19.04 FPS and 17.20 FPS (frames per second), respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
The bioid face database. https://www.bioid.com/facedb. Accessed 23 Oct 2020
Gi4e - gaze interaction for everybody. http://www.unavarra.es/gi4e/databases?languageId=1. Accessed 23 Oct 2020
Road traffic injuries. https://www.who.int/news-room/fact-sheets/detail/road-traffic-injuries. Accessed 22 Oct 2020
Bulling, A., Ward, J., Gellersen, H., Tröster, G.: Eye movement analysis for activity recognition using electrooculography. IEEE Trans. Pattern Anal. Mach. Intell. 33, 741–753 (2011)
Champaty, B., Pal, K., Dash, A.: Functional electrical stimulation using voluntary eyeblink for foot drop correction. In: 2013 Annual International Conference on Emerging Research Areas and 2013 International Conference on Microelectronics, Communications and Renewable Energy, pp. 1–4 (2013). https://doi.org/10.1109/AICERA-ICMiCR.2013.6575966
Chang, W., Lim, J., Im, C.: An unsupervised eye blink artifact detection method for real-time electroencephalogram processing. Physiol. Meas. 37(3), 401–17 (2016)
Colombo, C., Comanducci, D., Bimbo, A.D.: Robust tracking and remapping of eye appearance with passive computer vision. ACM Trans. Multimedia Comput. Commun. Appl. 3, 2:1–2:20 (2007)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015). http://arxiv.org/abs/1512.03385
Hsieh, C.S., Tai, C.C.: An improved and portable eye-blink duration detection system to warn of driver fatigue. Instrum. Sci. Technol. 41(5), 429–444 (2013). https://doi.org/10.1080/10739149.2013.796560
Jo, J., Lee, S., Jung, H., Park, K., Kim, J.: Vision-based method for detecting driver drowsiness and distraction in driver monitoring system. Opt. Eng. 50, 7202 (2011). https://doi.org/10.1117/1.3657506
Kim, K.W., Lee, W.O., Kim, Y.G., Hong, H.G., Lee, E.C., Park, K.R.: Segmentation method of eye region based on fuzzy logic system for classifying open and closed eyes. Opt. Eng. 54(3), 1–19 (2015). https://doi.org/10.1117/1.OE.54.3.033103
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017). https://doi.org/10.1145/3065386
Królak, A., Strumiłło, P.: Eye-blink detection system for human–computer interaction. Univers. Access Inf. Soc. 11(4), 409–419 (2012). https://doi.org/10.1007/s10209-011-0256-6
Kégl, B.: The return of adaboost.mh: multi-class hamming trees (2013)
Lalonde, M., Byrns, D., Gagnon, L., Teasdale, N., Laurendeau, D.: Real-time eye blink detection with GPU-based sift tracking. In: Fourth Canadian Conference on Computer and Robot Vision (CRV 2007), pp. 481–487 (2007). https://doi.org/10.1109/CRV.2007.54
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791
Lee, W., Lee, E.C., Park, K.: Blink detection robust to various facial poses. J. Neurosci. Methods 193, 356–72 (2010). https://doi.org/10.1016/j.jneumeth.2010.08.034
Mita, T., Kaneko, T., Hori, O.: Joint Haar-like features for face detection. In: Tenth IEEE International Conference on Computer Vision (ICCV 2005) Volume 1, vol. 2, pp. 1619–1626 (2005). https://doi.org/10.1109/ICCV.2005.129
Mohanakrishnan, J., Nakashima, S., Odagiri, J., Shanshan, Yu.: A novel blink detection system for user monitoring. In: 2013 1st IEEE Workshop on User-Centered Computer Vision (UCCV), pp. 37–42 (2013). https://doi.org/10.1109/UCCV.2013.6530806
Nguyen, D.L., Putro, M.D., Jo, K.H.: Eyes status detector based on light-weight convolutional neural networks supporting for drowsiness detection system. In: IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society, pp. 477–482 (2020). https://doi.org/10.1109/IECON43393.2020.9254858
Ramzan, M., Khan, H.U., Awan, S.M., Ismail, A., Ilyas, M., Mahmood, A.: A survey on state-of-the-art drowsiness detection techniques. IEEE Access 7, 61904–61919 (2019). https://doi.org/10.1109/ACCESS.2019.2914373
Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. CoRR abs/1506.01497 (2015). http://arxiv.org/abs/1506.01497
Shang, W., Sohn, K., Almeida, D., Lee, H.: Understanding and improving convolutional neural networks via concatenated rectified linear units. CoRR abs/1603.05201 (2016). http://arxiv.org/abs/1603.05201
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2015)
Song, F., Tan, X., Liu, X., Chen, S.: Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients. Pattern Recogn. 47(9), 2825–2838 (2014). https://doi.org/10.1016/j.patcog.2014.03.024
Szegedy, C., et al.: Going deeper with convolutions. CoRR abs/1409.4842 (2014). http://arxiv.org/abs/1409.4842
Trutoiu, L.C., Carter, E.J., Matthews, I., Hodgins, J.K.: Modeling and animating eye blinks. ACM Trans. Appl. Percept. 8(3) (2011). https://doi.org/10.1145/2010325.2010327
Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vis. 57, 137–154 (2004). https://doi.org/10.1023/B:VISI.0000013087.49260.fb
Wu, J., Trivedi, M.M.: An eye localization, tracking and blink pattern recognition system: algorithm and evaluation. TOMCCAP 6 (2010). https://doi.org/10.1145/1671962.1671964
Acknowledgement
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the government (MSIT).(No.2020R1A2C2008972).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Nguyen, DL., Putro, M.D., Jo, KH. (2021). Eye State Recognizer Using Light-Weight Architecture for Drowsiness Warning. In: Nguyen, N.T., Chittayasothorn, S., Niyato, D., Trawiński, B. (eds) Intelligent Information and Database Systems. ACIIDS 2021. Lecture Notes in Computer Science(), vol 12672. Springer, Cham. https://doi.org/10.1007/978-3-030-73280-6_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-73280-6_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73279-0
Online ISBN: 978-3-030-73280-6
eBook Packages: Computer ScienceComputer Science (R0)