Distracted driver classification using deep learning

  • Munif AlotaibiEmail author
  • Bandar Alotaibi
Original Paper


One of the most challenging topics in the field of intelligent transportation systems is the automatic interpretation of the driver’s behavior. This research investigates distracted driver posture recognition as a part of the human action recognition framework. Numerous car accidents have been reported that were caused by distracted drivers. Our aim was to improve the performance of detecting drivers’ distracted actions. The developed system involves a dashboard camera capable of detecting distracted drivers through 2D camera images. We use a combination of three of the most advanced techniques in deep learning, namely the inception module with a residual block and a hierarchical recurrent neural network to enhance the performance of detecting the distracted behaviors of drivers. The proposed method yields very good results. The distracted driver behaviors include texting, talking on the phone, operating the radio, drinking, reaching behind, fixing hair and makeup, and talking to the passenger.


Distracted drivers Deep learning Convolutional neural network Inception 



  1. 1.
    World Health Organization: World Health Organization. Management of Substance Abuse Unit. Global Status Report on Alcohol and Health, 2014. World Health Organization, Geneva (2014)Google Scholar
  2. 2.
    Abouelnaga, Y., Eraqi, H.M., Moustafa, M.N.: Real-time distracted driver posture classification. arXiv preprint arXiv:1706.09498 (2018)
  3. 3.
    Peden, M.: World Report on Road Traffic Injury Prevention. World Health Organization, Geneva (2004) Google Scholar
  4. 4.
    Yan, C., Coenen, F., Zhang, B.: Driving posture recognition by convolutional neural networks. IET Comput. Vis. 10(2), 103–114 (2016)CrossRefGoogle Scholar
  5. 5.
    National Highway Traffic Safety Administration.: 2015 motor vehicle crashes: overview. In: Traffic Safety Facts Research Note, pp. 1–9 (2016)Google Scholar
  6. 6.
    Resalat, S.N., Saba, V.: A practical method for driver sleepiness detection by processing the EEG signals stimulated with external flickering light. Signal Image Video Process. 9, 1751–1757 (2015)CrossRefGoogle Scholar
  7. 7.
    Craye, C., Karray, F.: Driver distraction detection and recognition using RGB-D sensor. arXiv preprint arXiv:1502.00250 (2015)
  8. 8.
    Fernández, A., Usamentiaga, R., Carús, J.L., Casado, R.: Driver distraction using visual-based sensors and algorithms. Sensors 16(11), 1805 (2016) CrossRefGoogle Scholar
  9. 9.
    Watta, P., Lakshmanan, S., Hou, Y.: Nonparametric approaches for estimating driver pose. IEEE Trans. Veh. Technol. 56(4), 2028–2041 (2007)CrossRefGoogle Scholar
  10. 10.
    Murphy-Chutorian, E., Trivedi, M.M.: Head pose estimation and augmented reality tracking: an integrated system and evaluation for monitoring driver awareness. IEEE Trans. Intell. Transp. Syst. 11(2), 300–311 (2010)CrossRefGoogle Scholar
  11. 11.
    Teyeb, I., Jemai, O., Zaied, M., Amar, C.B.: A drowsy driver detection system based on a new method of head posture estimation. In: International Conference on Intelligent Data Engineering and Automated Learning, pp. 362–369, September 2014. Springer, Cham (2014)CrossRefGoogle Scholar
  12. 12.
    Doshi, A., Trivedi, M.M.: On the roles of eye gaze and head dynamics in predicting driver’s intent to change lanes. IEEE Trans. Intell. Transp. Syst. 10(3), 453–462 (2009)CrossRefGoogle Scholar
  13. 13.
    Teyeb, I., Jemai, O., Zaied, M., Amar, C.B.: A novel approach for drowsy driver detection using head posture estimation and eyes recognition system based on wavelet network. In: The 5th International Conference on Information, Intelligence, Systems and Applications, IISA 2014, pp. 379–384, July 2014. IEEE (2014)Google Scholar
  14. 14.
    Bergasa, L.M., Nuevo, J., Sotelo, M.A., Barea, R., Lopez, M.E.: Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Transp. Syst. 7(1), 63–77 (2006)CrossRefGoogle Scholar
  15. 15.
    Jemai, O., Teyeb, I., Bouchrika, T.: A novel approach for drowsy driver detection using eyes recognition system based on wavelet network. Int. J. Recent Contrib. Eng. Sci. IT (iJES) 1(1), 46–52 (2013)CrossRefGoogle Scholar
  16. 16.
    Lei, J., Han, Q., Chen, L., Lai, Z., Zeng, L., Liu, X.: A novel side face contour extraction algorithm for driving fatigue statue recognition. IEEE Access 5, 5723–5730 (2017)CrossRefGoogle Scholar
  17. 17.
    Cheng, S.Y., Park, S., Trivedi, M.M.: Multi-spectral and multi-perspective video arrays for driver body tracking and activity analysis. Comput. Vis. Image Underst. 106(2–3), 245–257 (2007)CrossRefGoogle Scholar
  18. 18.
    Tran, C., Doshi, A., Trivedi, M.M.: Modeling and prediction of driver behavior by foot gesture analysis. Comput. Vis. Image Underst. 116(3), 435–445 (2012)CrossRefGoogle Scholar
  19. 19.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  20. 20.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  21. 21.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  22. 22.
    Soon, F.C., Khaw, H.Y., Chuah, J.H., Kanesan, J.: Vehicle logo recognition using whitening transformation and deep learning. Signal Image Video Process. 13, 111–119 (2019)CrossRefGoogle Scholar
  23. 23.
    Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. In: Advances in Neural Information Processing Systems, pp. 2042–2050 (2014)Google Scholar
  24. 24.
    Abdel-Hamid, O., Mohamed, A.R., Jiang, H., Penn, G.: Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4277–4280, March 2012. IEEE (2012)Google Scholar
  25. 25.
    Abdel-Hamid, O., Mohamed, A.R., Jiang, H., Deng, L., Penn, G., Yu, D.: Convolutional neural networks for speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process. 22(10), 1533–1545 (2014)CrossRefGoogle Scholar
  26. 26.
    Ngiam, J., Chen, Z., Bhaskar, S.A., Koh, P.W., Ng, A.Y.: Sparse filtering. In: Advances in Neural Information Processing Systems, pp. 1125–1133 (2011)Google Scholar
  27. 27.
    Zhao, C.H., Zhang, B.L., He, J., Lian, J.: Recognition of driving postures by contourlet transform and random forests. IET Intell. Transp. Syst. 6(2), 161–168 (2012)CrossRefGoogle Scholar
  28. 28.
    Eraqi, H.M., Abouelnaga, Y., Saad, M.H., Moustafa, M.N.: Driver distraction identification with an ensemble of convolutional neural networks. J. Adv. Transp. (2019). CrossRefGoogle Scholar
  29. 29.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  30. 30.
    Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Garcia-Rodriguez, J.: A review on deep learning techniques applied to semantic segmentation. arXiv:1704.06857 (2017)
  31. 31.
    Chung, J., Ahn, S., Bengio, Y.: Hierarchical multiscale recurrent neural networks. arXiv preprint arXiv:1609.01704 (2016)
  32. 32.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  33. 33.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  34. 34.
    Chollet, F., et al.: Keras. (2015). Accessed 8 Aug 2018

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Computing and Information TechnologyShagra UniversityShagraSaudi Arabia
  2. 2.College of Computer Science and Information TechnologyUniversity of TabukTabukSaudi Arabia

Personalised recommendations