Multimedia Tools and Applications

, Volume 78, Issue 14, pp 19543–19563 | Cite as

Dynamic set point model for driver alert state using digital image processing

  • Cesar IsazaEmail author
  • Karina Anaya
  • Carlos Fuentes-Silva
  • Jonny Paul Zavala de Paz
  • Amilcar Rizzo
  • Angel-Ivan Garcia-Moreno


The driver fatigue and lose of attention while driving are the most important causes of traffic accidents. Each year more than one million of deaths occur due to these facts. Thus, this problem has been converted into a serious social issue with high impact not only in economic terms, but also in the public health sector all around the world. Several approaches based on computer vision systems have been proposed to deal with this severe situation, but none of them have fully considered the non-fatigue state as a primary knowledge to detect an unusual event of a person while driving. In fact, typical approaches to deal with the problem of fatigue detection, are based on the analysis of behavioral features extracted with digital image processing such as frequency of blinking, yawning, among others. However, the huge limitation is the short interval of time between each analysis, that generally is few frames per second. Furthermore, all available methods are focus in modeling the fatigue, instead of representing the set point alert state of the driver, which is the main core of the proposed strategy. Hence, in this paper a dynamic set point model for alert state while driving using digital image processing and machine learning techniques is presented. The approach uses an embedded system build with a Raspberry prototyping board and a USB HD camera. Raspbian operative system controls OPEN CV libraries written in Python to detect face parts with an algorithm running Harr descriptors. The features extracted were the position and orientation of the head throw several minutes. Then, a mixture of Gaussians model with its learning and updating stages is used to represent the behaviour of features. Also, a dataset was built considering professional and non-professional drivers under two main scenarios: real and simulated conditions. Experimental results show the viability of the method for posterior analysis of unusual events while driving like fatigue detection, cellphone call or chat detection, or any other distraction not related to the driving process.


Face detection Driver fatigue detection Learning model 



Authors would like to acknowledge the financial support of this work by grants from Consejo Nacional de Ciencia y Tecnologia (CONACYT), Mexico, under Sistema Nacional de Investigadores (SNI) program.

The paper was made in memory ofPhD. Miguel Angel Flores.


  1. 1.
    Abdulin E, Komogortsev O (2015) User eye fatigue detection via eye movement behavior. In: Proceedings of the 33rd annual ACM conference extended abstracts on human factors in computing systems, pp 1265–1270Google Scholar
  2. 2.
    Abulkhair M, Alsahli AH, Taleb KM, Bahran AM, Alzahrani FM, Alzahrani HA, Ibrahim LF (2015) Mobile platform detect and alerts system for driver fatigue. Procedia Comput Sci 62:555--564CrossRefGoogle Scholar
  3. 3.
    Bradski G, Kaehler A (2008) Learning OpenCV: Computer vision with the OpenCV library. “O’Reilly Media, Inc.”Google Scholar
  4. 4.
    Chai R, Naik GR, Tran Y, Ling SH, Craig A, Nguyen HT (2015) Classification of driver fatigue in an electroencephalography-based countermeasure system with source separation module. In: 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 514–517Google Scholar
  5. 5.
    Chan AB, Mahadevan V, Vasconcelos N (2011) Generalized Stauffer–Grimson background subtraction for dynamic scenes. Mach Vis Appl 22(5):751–766CrossRefGoogle Scholar
  6. 6.
    Craye C, Rashwan A, Kamel MS, Karray F (2016) A multi-modal driver fatigue and distraction assessment system. Int J Intell Transp Syst Res 14(3):173–194Google Scholar
  7. 7.
    Consejo Nacional para la Prevención de Accidentes (2013) Tercer informe sobre la situación de la seguridad vial, Mé,xico 2013, Accessed 1 December 2015Google Scholar
  8. 8.
    Dahiphale VE, Sathyanarayana R, Mukhedkar MM (2015) Computer vision system for driver fatigue detection. Computer, IJARECE 4(9):2331–2334Google Scholar
  9. 9.
    Eskandarian A, Sayed R, Delaigue P, Mortazavi A, Blum J (2007) Advanced driver fatigue research. U.S. Department of Transportation, Accessed 1 December 2015Google Scholar
  10. 10.
    Freund Y, Schapire RE (1996) Experiments with a new boosting algorithm. Int Conf Mach Learn 96:148–156Google Scholar
  11. 11.
    Gao XY, Zhang YF, Zheng WL, Lu BL (2015) Evaluating driving fatigue detection algorithms using eye tracking glasses. In: 2015 7th international IEEE/EMBS conference on neural engineering (NER), pp 767–770Google Scholar
  12. 12.
    Goyal R (2015) Fatigue Detection Using Artificial Intelligence and Computer Vision Algorithms. 2015 AAAS Annual Meeting (12–16 February 2015)Google Scholar
  13. 13.
    Ji Q, Yang X (2002) Real-time eye, gaze, and face pose tracking for monitoring driver vigilance. Real-Time Imaging 8(5):357–377CrossRefzbMATHGoogle Scholar
  14. 14.
    Kong W, Zhou L, Wang Y, Zhang J, Liu J, Gao S (2015) A system of driving fatigue detection based on machine vision and its application on smart device. J Sens 2015:1–11CrossRefGoogle Scholar
  15. 15.
    Li Z, Sun G, Zhang F, Jia L, Zheng K, Zhao D (2015) Smartphone-based fatigue detection system using progressive locating method. IET Intelligent Transport SystemsGoogle Scholar
  16. 16.
    Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115CrossRefGoogle Scholar
  17. 17.
    Liu YE, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action2Activity: Recognizing complex activities from sensor data. IJCAI, pp 1617–1623Google Scholar
  18. 18.
    McLachlan GJ, Basford KE (1988) Mixture models marcel Dekker. New YorkGoogle Scholar
  19. 19.
    Moon TK (1996) The expectation-maximization algorithm. IEEE Signal Process Mag 13(6):47–60CrossRefGoogle Scholar
  20. 20.
    Mu Z (2015) Fatigue Detection Method based on Common Spatial Subspace Decomposition. In: 2015 international conference on education, management, information and medicine, pp 1031--1034Google Scholar
  21. 21.
    Papadelis C, Kourtidou-Papadeli C, Bamidis PD, Chouvarda I, Koufogiannis D, Bekiaris E, Maglaveras N (2006) Indicators of sleepiness in an ambulatory EEG study of night driving. In: 28th annual international conference of the IEEE engineering in medicine and biology society, 2006. EMBS’06, pp 6201–6204Google Scholar
  22. 22.
    Patel SP, Patel BP, Sharma M, Shukla N, Patel HM (2015) Detection of Drowsiness and Fatigue level of Driver. Int J Innovative Res Sci Technol 1(11):133–138Google Scholar
  23. 23.
    Reynolds DA, Quatieri TF, Dunn RB (2000) Speaker verification using adapted Gaussian mixture models. Digital Signal Process 10(1–3):19–41CrossRefGoogle Scholar
  24. 24.
    Shen HM, Xu MH (2015) Design and implementation of embedded driver fatigue monitor system. In: 2015 international conference on artificial intelligence and industrial engineering AIIE, vol 2015, pp 103–106Google Scholar
  25. 25.
    Sigari MH, Pourshahabi MR, Soryani M, Fathy M (2014) A review on driver face monitoring systems for fatigue and distraction detection. Int J Adv Sci Technol 64:73–100CrossRefGoogle Scholar
  26. 26.
    Stauffer C, Grimson W (1999) Adaptive background mixture models for real-time tracking. In: CVPR, pp 246–52Google Scholar
  27. 27.
    Sun KJ, Yang XX, Yang L, Cui JX, Zhang S, Lu Y (2015) Research on the progress of physiological signal-based method for fatigue driving risks detection. In: Proceedings of the 2014 international conference on multimedia, communication and computing application MCCA, vol 2014, pp 269--271Google Scholar
  28. 28.
    Tanaka M Face detection parts, Accessed 1 December 2015. html
  29. 29.
    Tansakul W, Tangamchit P (2016) Fatigue driver detection system using a combination of blinking rate and driving inactivity. J Auto Control Eng 4(1):33–39Google Scholar
  30. 30.
    Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154CrossRefGoogle Scholar
  31. 31.
    Wang P, Mu Z (2015) Studies of brain fatigue detection method based on second-order blind identification. In: 2015 international conference on education, managementm information and medicine, pp 538–541Google Scholar
  32. 32.
    Wang Q, Tang L, Yang K (2015) Driver fatigue detection algorithm research based on the characteristics of eyes. Appl Mech Mater 701:30–35CrossRefGoogle Scholar
  33. 33.
    Wang YQ (2014) An analysis of the Viola-Jones face detection algorithm. Image Process Line 4:128–148CrossRefGoogle Scholar
  34. 34.
    Wilson PI, Fernandez J (2006) Facial feature detection using Haar classifiers. J Comput Sci Coll 21(4):127–133Google Scholar
  35. 35.
    World Health Organization (2013) Global status report on road safety 2013. supporting a decade of action, Accessed 1 December 2015Google Scholar
  36. 36.
    Zhang L, Liu F, Tang J (2015) Real-time system for driver fatigue detection by RGB-D camera. ACM Trans Intell Syst Technol (TIST) 6(2):22:1–22:17Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Universidad Politécnica de QuerétaroEl MarquésMéxico

Personalised recommendations