Improved Fatigue Detection Using Eye State Recognition with HOG-LBP

  • Bin HuangEmail author
  • Renwen Chen
  • Wang Xu
  • Qinbang Zhou
  • Xu Wang
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1143)


Fatigue driving is one of the main factors in traffic accidents. Many driver fatigue alert systems have been developed to prevent fatal car accidents. Existing fatigue detection methods using image processing usually fail in face of illumination and occlusion variations. In this paper, we propose a novel fatigue detection method using eye state recognition with HOG-LBP fused features. Our method first proposes HOG-LBP fusion schemes to combine the advantages of HOG and LBP features. Taking concatenated or additive fused feature as input, we design deep neural networks and train the model on CEW eye dataset for eye state recognition. We give an in-depth analysis to select the optimal fused coefficient for the best model. Based on eye state prediction results, we extract two eye-related features and design a decision criterion for fatigue detection. Experimental results prove the feasibility of the proposed method in detecting drowsiness level at different driving conditions.


Fatigue detection HOG-LBP fused feature Additive fusion Eye state recognition 



This work was funded by a project that partially funded by National Science Foundation of China (51675265) and the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD). The authors gratefully acknowledge this support.


  1. 1.
    Bergasa, L., Nuevo, J., Sotelo, M., et al.: Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Transp. Syst. 7, 63–77 (2006)CrossRefGoogle Scholar
  2. 2.
    Wheaton, G., Shults, R.: Drowsy driving and risk behaviors 10 states and Puerto Rico. MMWR Morb. Mortal. Wkly Rep. 63(26), 557–562 (2014)Google Scholar
  3. 3.
    Qing, W., Bingxi, S., Bin, X. et al.: A perclos-based driver fatigue recognition application for smart vehicle space. In: Information Processing (ISIP), 2010 Third International Symposium on, pp. 437–441 (2010)Google Scholar
  4. 4.
    Friedrichs, F., Yang, B.: Drowsiness monitoring by steering and lane data based features under real driving conditions. In: European Signal Processing Conference, pp. 209–213 (2010)Google Scholar
  5. 5.
    Rogado, E., Garcia, L., Barea, R. et al.: Driver fatigue detection system. In: Robotics and Biomimetics, 2008. ROBIO 2008. IEEE International Conference on, pp. 1105–1110 (2009)Google Scholar
  6. 6.
    Jap, B.T., Lal, S., Fischer, P., et al.: Using EEG spectral components to assess algorithms for detecting fatigue. Expert Syst. Appl. 36(2), 2352–2359 (2009)CrossRefGoogle Scholar
  7. 7.
    Patel, M., Lal, S.K.L., Kavanagh, D., et al.: Applying neural network analysis on heart rate variability data to assess driver fatigue. Expert Syst. Appl. 38(6), 7235–7242 (2011)CrossRefGoogle Scholar
  8. 8.
    Wierwille, W.W., Wregget, S., Kirn, C. et al.: Research on vehicle-based driver status/performance monitoring: Development validation and refinement of algorithms for detection of driver drowsiness. In: National Highway Traffice Safety Administration Final Report (1994)Google Scholar
  9. 9.
    Eriksson, M., Nikolaos, P.P.: Eye-tracking for detection of driver fatigue. In: IEEE Conference on Intelligent Transportation System, pp. 314–319 (1997)Google Scholar
  10. 10.
    Singh, S., Nikolaos, P.P.: Monitoring driver fatigue using facial analysis techniques. In: Proceedings 199 IEEE/IEEJ/JSAI International Conference on Intelligent Transportation Systems (Cat. No. 99TH8383), pp. 314–318 (1999)Google Scholar
  11. 11.
    Smith, P., Shah, M., Lobo, N.D.: Monitoring head/eye motion for driver alertness with one camera. In: Proceedings 15th International Conference on Pattern Recognition (2000)Google Scholar
  12. 12.
    Ojala, T., Pietikainen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29(1), 51–59 (1996)CrossRefGoogle Scholar
  13. 13.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 886–893 (2005)Google Scholar
  14. 14.
    Srivastava, N., Hinton, G., Krizhevsky, A., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Dong, Y., Hu, Z., Uchimura, K., et al.: Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans. Intell. Transp. Syst. 12(2), 596–614 (2011)CrossRefGoogle Scholar
  16. 16.
    Lenskiy, A.A., Lee, J.: Driver’s eye blinking detection using novel color and texture segmentation algorithms. Int. J. Control Autom. Syst. 10(2), 317–327 (2012)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2021

Authors and Affiliations

  • Bin Huang
    • 1
    Email author
  • Renwen Chen
    • 1
  • Wang Xu
    • 1
  • Qinbang Zhou
    • 1
  • Xu Wang
    • 1
  1. 1.Nanjing University of Aeronautics and AstronauticsNanjingChina

Personalised recommendations