Skip to main content
SpringerLink
Log in

Drowsy Driver Posture, Facial, and Eye Monitoring Methods

  • Reference work entry
Handbook of Intelligent Vehicles

Abstract

This chapter presents a real-time computer vision system for monitoring drowsy driver. It uses one remotely located charge coupled device (CCD) camera to acquire video of the driver’s face. From the video, various computer vision algorithms are employed to simultaneously, nonintrusively, and in real time recognize the facial behaviors that closely relate to the driver’s level of vigilance. The facial behaviors include rigid head movement (characterized by 3D face pose), nonrigid facial muscular movement (characterized by facial expressions), and eye gaze movement. The system was tested in a simulating environment with different subjects and it was found robust, reliable, and accurate in characterizing facial behaviors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 849.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Anon (1998) Proximity array sensing system: head position monitor/metric. Advanced Safety Concepts, Sante Fe

    Google Scholar 

  • Anon (1999) Perclos and eyetracking: challenge and opportunity. Technical Report, Applied Science Laboratories, Bedford

    Google Scholar 

  • Bartlett M, Littlewort G, Frank M, Lainscsek C, Fasel I, Movellan J (2005) Recognizing facial expression: machine learning and application to spontaneous behavior. In: Proceedings of IEEE Computer society conference computer vision and pattern recognition, San Diego

    Google Scholar 

  • Bartlett M, Littlewort G, Frank M, Lainscsek C, Fasel I, Movellan J (2006) Automatic recognition of facial actions in spontaneous expressions. J Multimedia 1:22–35

    Google Scholar 

  • Bartlett M, Littlewort G, Frank M, Lainscsek C, Fasel I, Movellan J (2007) http://mplab.ucsd.edu/grants/project1/research/fully-auto-facs-coding.html

  • Bazzo J, Lamar M (2004) Recognizing facial actions using gabor wavelets with neutral face average difference. In: Proceedings of the sixth IEEE international conference on automatic face and gesture recognition, Seoul

    Google Scholar 

  • Bourel F, Chibelushi CC, Low AA (2000) Robust facial feature tracking. In: Proceedings of the British Machine Vision Conference, Bristol

    Google Scholar 

  • Boverie S, Lcqellec J, Hirl A (1998) Intelligent systems for video monitoring of vehicle cockpit. International congress and exposition ITS: advanced controls and vehicle navigation systems, pp 1–5

    Google Scholar 

  • Buenaposada JM, Munoz E, Baumela L (2008) Recognising facial expressions in video sequences. Pattern Anal Appl 11(1):101–116

    Article  MathSciNet  Google Scholar 

  • Cohen I, Sebe N, Ashutosh G, Chen LS, Huang TS (2003) Facial expression recognition from video sequences: temporal and static modeling. Comput Vis Image Underst 91(1–2):160–187

    Article  Google Scholar 

  • Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models – their training and application. Comput Vis Image Underst 61(1):38–59

    Article  Google Scholar 

  • Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance model. IEEE Trans Pattern Anal Mach Intell 23(6):681–685

    Article  Google Scholar 

  • Daugman J (1988) Complete discrete 2-d gabor transforms by neural networks for image analysis and compression. IEEE Trans ASSP 36(7):1169–1179

    Article  MATH  Google Scholar 

  • Dinges D, Mallis M, Maislin G, Powell J (1998) Evaluation of techniques for ocular measurement as an index of fatigue and the basis for alertness management. Department of Transportation Highway Safety Publication 808 762

    Google Scholar 

  • Donato G, Bartlett MS, Hager JC, Ekman P, Sejnowski TJ (1999) Classifying facial actions. IEEE Trans Pattern Anal Mach Intell 21:974–989

    Article  Google Scholar 

  • Dornaika F, Davoine F (2008) Simultaneous facial action tracking and expression recognition in the presence of head motion. Int J Comput Vis (IJCV) 76:251–281

    Google Scholar 

  • Dryden IL, Mardia KV (1998) Shape analysis. Wiley, Chichester

    MATH  Google Scholar 

  • Ekman P, Friesen WV (1978) Facial action coding system (FACS): manual. Consulting Psychologists Press, Palo Alto

    Google Scholar 

  • Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  • Fleet DJ, Jepson AD (1990) Computation of component image velocity from local phase information. Int J Comput Vis 5(1):77–104

    Article  Google Scholar 

  • Guestrin ED, Eizenman M (2006) General theory of remote gaze estimation using the pupil center and corneal reflections. IEEE Trans Biomed Eng 53:1124–1133

    Article  Google Scholar 

  • Ishii T, Hirose M, Iwata H (1987) Automatic recognition of drivers facial expression by image analysis. J Soc Automot Eng Japan 41:1398–1403

    Google Scholar 

  • Ishikawa T, Baker S, Matthews I, Kanade T (2004) Passive driver gaze tracking with active appearance models. In: Proceedings of the 11th World congress on intelligent transportation systems, Berkeley

    Google Scholar 

  • Ji Q, Zhu Z, Lan P (2004) Real-time nonintrusive monitoring and prediction of driver fatigue. IEEE Trans Veh Technol 53(4):1052–1068

    Article  Google Scholar 

  • Ji Q, Lan P, Looney C (2006) A probabilistic framework for modeling and real-time monitoring human fatigue. IEEE Trans Syst Man Cybernetics Part A Syst Humans 36(5):862–875

    Article  Google Scholar 

  • Jiao F, Li SZ, Shum HY, Schuurmans D (2003) Face alignment using statistical models and wavelet features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR03), vol 1. Madison, pp 321–327

    Google Scholar 

  • Kanade T, Cohn J, Tian YL (2000) Comprehensive database for facial expression analysis. In: Proceedings of the 4th IEEE international conference on automatic face and gesture recognition (FG’00), Grenoble

    Google Scholar 

  • Lc Technologies (2005) http://www.eyegaze.com

  • Lien JJJ, Kanade T, Cohn J, Li C (1999) Detection, tracking, and classification of action units in facial expression. J Robot Auton Syst 31:131–146

    Article  Google Scholar 

  • McKenna SJ, Gong S, Würtz RP, Tanner J, Banin D (1997) Tracking facial feature points with gabor wavelets and shape models. In: Proceedings of International Conference on Audio- and Video-Based Biometric Person Authentication, Hilton Rye Town, pp 35–42

    Google Scholar 

  • Morimoto CH, Mimica MR (2005) Eye gaze tracking techniques for interactive applications. Comput Vis Image Underst 98:4–24 (Special issue on eye detection and tracking)

    Article  Google Scholar 

  • Murphy K (1998) Inference and learning in hybrid bayesian networks. Report No. UCBCSD-98-990, Computer Science Department, U.C. Berkeley

    Google Scholar 

  • Murphy K (2002) Dynamic Bayesian networks (draft)

    Google Scholar 

  • Or SH, Luk WS, Wong KH, King I (1998) An efficient iterative pose estimation algorithm. Image Vision Comput 16(5):353–362

    Article  Google Scholar 

  • Oyster CW (1999) The human eye: structure and function. Sinauer Associate, Sunderland

    Google Scholar 

  • Saito S (1992) Does fatigue exist in a quantitative of eye movement? Ergonomics 35:607–615

    Article  Google Scholar 

  • Saito H, Ishiwaka T, Sakata M, Okabayashi S (1994) Applications of drivers's line of sight to automobiles – what can driver’s eye tell. In: Proceedings of 1994 vehicle navigation and information systems conference, Yokohama, pp 21–26

    Google Scholar 

  • Shan C, Gong S, MacOwan PW (2006) Dynamic facial expression recognition using a bayesian temporal manifold model. In: Proceedings of the British machine vision conference, Edinburgh

    Google Scholar 

  • Smith P, Shah M, da Vitoria Lobo N (2000) Monitoring head/eye motion for driver alertness with one camera. In: Proceedings of the 15th international conference on pattern recognition, vol 4. Barcelona, pp 636–642

    Google Scholar 

  • Theimer WM (1994) Phase-based binocular vergence control and depth reconstruction using active vision. CVGIP: Image Underst 60(3):343–358

    Article  Google Scholar 

  • Tian YI, Kanade T, Cohn J (2001) Recognizing action units for facial expression analysis. IEEE Trans Pattern Anal Mach Intell 23:33–80

    Article  Google Scholar 

  • Tomasi C, Kanade T (1991) Detection and tracking of point features. Carnegie Mellon University Technical Report CMU-CS-91-132

    Google Scholar 

  • Tong Y, Liao W, Ji Q (2007a) Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Trans Pattern Anal Mach Intell 29:1683–1699

    Article  Google Scholar 

  • Tong Y, Liao W, Xue Z, Ji Q (2007) A unified probabilistic framework for spontaneous facial activity modeling and understanding. In: Proceedings of the 2007 IEEE Conference on computer vision and pattern recognition (CVPR), Minneapolis

    Google Scholar 

  • Tong Y, Wang Y, Zhu Z, Ji Q (2007c) Robust facial feature tracking under varying face pose and facial expression. Pattern Recogn 40:3195–3208

    Article  MATH  Google Scholar 

  • Tong Y, Chen J, Ji Q (2010) A unified probabilistic framework for spontaneous facial action modeling and understanding. IEEE Trans Pattern Anal Mach Intell 32(2):258–273

    Article  Google Scholar 

  • Trucco E, Verri A (1998) Introductory techniques for 3-D computer vision. Prentice-Hall, Upper Saddle River

    Google Scholar 

  • Wiskott L, Fellous JM, Krger N, von der Malsburg C (1997) Active appearance model. IEEE Trans Pattern Anal Mach Intell 19:775–779

    Article  Google Scholar 

  • Yammamoto K, Higuchi S (1992) Development of a drowsiness warning system. J Soc Automot Engin Japan 46:127–133

    Google Scholar 

  • Zhang Y, Ji Q (2005) Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans Pattern Anal Mach Intell (PAMI) 27(5):699–714

    Article  MathSciNet  Google Scholar 

  • Zhou S, Krueger V, Chellappa R (2003) Probabilistic recognition of human faces from video. Comput Vis Image Underst 91(1–2):214–245

    Article  Google Scholar 

  • Zhu Z, Ji Q (2007) Novel eye gaze tracking techniques under natural head movement. IEEE Trans Biomed Eng 54(12):2246–2260

    Article  MathSciNet  Google Scholar 

  • http://www.a-s-l.com (2006)

Download references

Author information

Authors and Affiliations

  1. Visualization and Computer Vision Lab, GE Global Research Center, One Research Circle, KW-C410, 12308, Niskayuna, NY, USA

    Jixu Chen

  2. Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute, JEC 7004, 12180-3590, Troy, NY, USA

    Qiang Ji

Authors
  1. Jixu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiang Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jixu Chen .

Editor information

Editors and Affiliations

  1. Center for Intelligent Systems Research, The George Washington University, 801 22nd Street, NW, Phillips Hall 630, Washington, DC, 20052, USA

    Azim Eskandarian

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag London Ltd.

About this entry

Cite this entry

Chen, J., Ji, Q. (2012). Drowsy Driver Posture, Facial, and Eye Monitoring Methods. In: Eskandarian, A. (eds) Handbook of Intelligent Vehicles. Springer, London. https://doi.org/10.1007/978-0-85729-085-4_35

Download citation

Publish with us

Policies and ethics

Search

Navigation