Skip to main content
SpringerLink
Log in

Monitoring drivers’ sleepy status at night based on machine vision

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Driver fatigue is a chief cause of traffic accidents. For this reason, it is essential to develop a monitoring system for drivers’ level of fatigue. In recent years, driver fatigue monitoring technology based on machine vision has become a research hotspot, but most research focuses on driver fatigue detection during the day. This paper presents a night monitoring system for real-time fatigue driving detection, which makes up for the deficiencies of fatigue driving detection technology at night. First, we use infrared imaging to capture a driver’s image at night, and then we design an algorithm to detect the driver’s face. Second, we propose a new eye-detection algorithm that combines a Gabor filter with template matching to locate the position of the corners of the eye, and add an eye-validation process to increase the accuracy of the detection rate. Third, we use a spline function to fit the eyelid curve. After extracting eye fatigue features, we use eye blinking parameters to evaluate fatigue. Our system has been tested on the IMM Face Database, which contains more than 200 faces, as well as in a real-time test. The experimental results show that the system has good accuracy and robustness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Author SJJ, Lee SJ, Kim J, Jung HG, Kang RP (2011) Vision-based method for detecting driver drowsiness and distraction in driver monitoring system. Opt Eng 50:127202-127202-24

  2. Bai X, Zhou F, Xue B (2011) Infrared image enhancement through contrast enhancement by using multiscale new top-hat transform. Infrared Phys Technol 54:61–69

    Article  Google Scholar 

  3. Bansode NK, Sinha PK (2013) Efficient eyes detection using fast normalised cross-correlation. Signal Image Process 3

  4. Branchitta F, Diani M, Corsini G, Porta A (2008) Dynamic-range compression and contrast enhancement in infrared imaging systems. Opt Eng 47:076401

    Article  Google Scholar 

  5. Brill JC, Hancock PA, Gilson RD (2003) DRIVER FATIGUE: IS SOMETHING MISSING? driving assessment the second international driving symposium on human factors in driver assessment training & vehicle design 16

  6. Dinges DF, Grace R (1998) Perclos: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance, Tech Brief

  7. Dong Y, Hu Z, Uchimura K, Murayama N (2011) Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans Intell Transp Syst 12:596–614

    Article  Google Scholar 

  8. Fan X, Sun Y, Yin B, Guo X (2010) Gabor-based dynamic representation for human fatigue monitoring in facial image sequences. Pattern Recogn Lett 31:234–243

    Article  Google Scholar 

  9. Hu C, Xu Z, Liu Y, Mei L (2015) Video structural description technology for the new generation video surveillance systems. Front Comp Sci 9(6):980–989

    Article  Google Scholar 

  10. Ilk HG, Jane O, Ilk O (2011) The effect of Laplacian filter in adaptive unsharp masking for infrared image enhancement. Infrared Phys Technol 54:427–438

    Article  Google Scholar 

  11. Jin L, Niu Q, Jiang Y, Xian H, Qin Y, Xu M (2013) Driver sleepiness detection system based on eye movements variables. Adv Mech Eng 2013:1–7

    Article  Google Scholar 

  12. Jo J, Lee SJ, Kang RP, Kim IJ, Kim J (2014) Detecting driver drowsiness using feature-level fusion and user-specific classification. Expert Syst Appl 41:1139–1152

    Article  Google Scholar 

  13. Khan R, Talha M, Khattak AS, Qasim M (2013) Realization of balanced contrast limited adaptive histogram equalization (B-CLAHE) for adaptive dynamic range compression of real time medical images. p 117–121

  14. Lai R, Yang YT, Wang BJ, Zhou HX (2010) A quantitative measure based infrared image enhancement algorithm using plateau histogram. Opt Commun 283:4283–4288

    Article  Google Scholar 

  15. Liang K, Ma Y, Xie Y, Zhou B, Wang R (2012) A new adaptive contrast enhancement algorithm for infrared images based on double plateaus histogram equalization. Infrared Phys Technol 55:309–315

    Article  Google Scholar 

  16. Lienhart R, Maydt J (2002) An extended set of Haar-like features for rapid object detection[C], International Conference on Image Processing. Proceedings. IEEE, 2002:I-900-I-903 vol. 1

  17. Lin CL (2011) An approach to adaptive infrared image enhancement for long-range surveillance. Infrared Phys Technol 54:84–91

    Article  Google Scholar 

  18. M. Perdersoli, J. Gonzàlez, B. Chakraborty and J.J. Villanueva, Boosting histograms of oriented gradients for human detection, Recercat Home (2012).

  19. Ranney TA, Garrott WR, Goodman MJ (2001) Nhtsa driver distraction research: past, present, and future. Proceedings: International Technical Conference on the Enhanced Safety of Vehicles. 9 p.-9 p

  20. Schleicher R, Galley N, Briest S, Galley L (2008) Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired? Ergonomics 51:982–1010

    Article  Google Scholar 

  21. Shen Y (2011) Efficient normalized cross correlation calculation method for stereo vision based robot navigation. Front Comp Sci China 5:227–235

    Article  MathSciNet  MATH  Google Scholar 

  22. Sigari MH, Fathy M, Soryani M (2013) A driver face monitoring system for fatigue and distraction detection. Int J Veh Technol 2013:73–100

    Google Scholar 

  23. Sun D, Watada J (2015) Detecting pedestrians and vehicles in traffic scene based on boosted HOG features and SVM. p 1–4.

  24. Suzuki M, Yamamoto N, Yamamoto O, Nakano T (2006) Measurement of Driver’s Consciousness by Image Processing -A Method for Presuming Driver’s Drowsiness by Eye-Blinks coping with Individual Differences. 4:2891–2896

  25. Vickers VE (1996) Plateau equalization algorithm for real-time display of high-quality infrared imagery. Opt Eng 35:1921–1926

    Article  Google Scholar 

  26. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. Proc CVPR 1:511

    Google Scholar 

  27. Wang BJ, Liu SQ, Li Q, Zhou HX (2006) A real-time contrast enhancement algorithm for infrared images based on plateau histogram. Infrared Phys Technol 48:77–82

    Article  Google Scholar 

  28. Wang YZ, Zhou LX, Kong WZ (2013) Eye blinking detecting in driving fatigue based on adaboost. J Hangzhou Dianzi Univ

  29. Watanabe T, Ito S, Yokoi K (2010) Co-occurrence histograms of oriented gradients for human detection. IPSJ Trans Comput Vis Appl 2:39–47

    Article  Google Scholar 

  30. Wei SD, Lai SH (2008) Fast template matching based on normalized cross correlation with adaptive multilevel winner update. IEEE Trans Image Process 17:2227–2235

    Article  MathSciNet  Google Scholar 

  31. Xu Z, Chen H (2015) The semantic analysis of knowledge map for the traffic violations from the surveillance video big data. Comput Syst Sci Eng 30(5)

  32. Xu Z, Hu C, Mei L (2016) Video structured description technology based intelligence analysis of surveillance videos for public security applications. Multimedia Tools Appl 75(19):12155–12172

    Article  Google Scholar 

  33. Xu Z, Mei L, Hu C, Liu Y (2016) The big data analytics and applications of the surveillance system using video structured description technology. Clust Comput 19(3):1283–1292

    Article  Google Scholar 

  34. Xu Z, Mei L, Liu Y, Hu C, Chen L (2016) Semantic enhanced cloud environment for surveillance data management using video structural description. Computing 98(1–2):35–54

    Article  MathSciNet  MATH  Google Scholar 

  35. Xu X, Quan C, Ren F (2015) Facial expression recognition based on Gabor Wavelet transform and Histogram of Oriented Gradients

  36. Yadav RP, Kutty K, Ugale SP (2015) Implementation of robust HOG-SVM based pedestrian classification. Int J Comput Appl 114:10–16

    Google Scholar 

  37. Yoo JC, Han TH (2009) Fast normalized cross-correlation. Circuits Syst Signal Process 28:819–843

    Article  MATH  Google Scholar 

  38. Zhao J, Chen Y, Feng H, Xu Z, Li Q (2014) Infrared image enhancement through saliency feature analysis based on multi-scale decomposition. Infrared Phys Technol 62:86–93

    Article  Google Scholar 

  39. Zhao W, Xu Z, Zhao J, Zhao F, Han X (2014) Variational infrared image enhancement based on adaptive dual-threshold gradient field equalization. Infrared Phys Technol 66:152–159

    Article  Google Scholar 

  40. Zheng Z, Yang J, Yang L (2005) A robust method for eye features extraction on color image. Pattern Recogn Lett 26:2252–2261

    Article  Google Scholar 

Download references

Acknowledgments

This work was partially supported by the National Natural Science Foundation of China (Grant Nos. 51408237, 51108192 and 51208500), the Chinese Postdoctoral Science Foundation (Grant Nos. 2012 M521824 and 2013 T60904), the Public Welfare Research and Capacity Building Project of Guangdong Province (Grant No. B2161520), the 2016 Students’ Research Program (SRP) of the South China University of Technology, and China new energy automobile products testing conditions research and development—Guangzhou traffic condition data collection.

Author information

Authors and Affiliations

  1. School of Civil Engineering and Transportation, South China University of Technology, Guangzhou, Guangdong, 510640, People’s Republic of China

    Feng You, Yao-hua Li, Ling Huang, Kang Chen & Jian-min Xu

  2. Research Center of Intelligent Transportation System, School of Engineering, Sun Yat-sen University, Guangzhou, Guangdong, 510275, People’s Republic of China

    Rong-hui Zhang

Authors
  1. Feng You
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yao-hua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ling Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rong-hui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jian-min Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rong-hui Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

You, F., Li, Yh., Huang, L. et al. Monitoring drivers’ sleepy status at night based on machine vision. Multimed Tools Appl 76, 14869–14886 (2017). https://doi.org/10.1007/s11042-016-4103-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4103-x

Keywords

Search

Navigation