A Review of Driver State Monitoring Systems in the Context of Automated Driving

  • Tobias HechtEmail author
  • Anna Feldhütter
  • Jonas Radlmayr
  • Yasuhiko Nakano
  • Yoshikuni Miki
  • Corbinian Henle
  • Klaus Bengler
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 823)


Conditionally automated driving (CAD) will lead to a paradigm shift in the field of driver state monitoring systems. High underload and the possibility of engaging in non-driving related activities will greatly influence the driver state. Level 3 also requires drivers to act as a fallback level in a take-over situation. Drivers have to get back in the loop and regain control with possible challenges due to their state. Therefore, driver state assessment will gain importance in order to ensure a safe and comfortable hand-over. The purpose of this paper is to provide an overview of driver state models and monitoring systems in the context of automated driving. Based on three driver state models, we focus on the commonly used driver state constructs fatigue, attention and workload. As part of this review, different definitions are summarized and possible metrics to operationalize these constructs were identified and critically reviewed. When reviewing the literature, it became apparent that driver state and the different constructs lack a common definition. Overall, eye-tracking is the technology with the most potential, but it needs further development to increase reliability. EEG lacks practicability and subjective measures are prone to misjudgement and may counteract extreme levels of fatigue.


Driver state assessment Fatigue Drowsiness EEG Eye-tracking 


  1. 1.
    SAE International (2016) Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems. SAE Standard J 3016_201609Google Scholar
  2. 2.
    Gold CG (2016) Modeling of Take-Over Performance in Highly Automated Vehicle Guidance. Dissertation, Technische Universität München.
  3. 3.
    Bhatt PP, Trivedi JA (2017) Various methods for driver drowsiness detection: an overview. Int J Comput Sci Eng (IJCSE) 9(03):70–74Google Scholar
  4. 4.
    Heikoop DD, de Winter JCF, van Arem B, Stanton NA (2015) Psychological constructs in driving automation. A consensus model and critical comment on construct proliferation. Theor Issues Ergon Sci. Scholar
  5. 5.
    Stanton N, Young M (2000) A proposed psychological model of driving automation. Theor Issues Ergon Sci. Scholar
  6. 6.
    Rauch N, Kaussner A, Boverie S, Giralt A (2009) HAVEit Deliverable D32.1 Report on driver assessment methodology. HAVEit - Highly automated vehicles for intelligent transport, RegensburgGoogle Scholar
  7. 7.
    Marberger C, Mielenz H, Naujoks F, Radlmayr J, Bengler K, Wandtner B (2018) Understanding and applying the concept of “Driver Availability” in automated driving. In: Stanton NA (ed) Advances in human aspects of transportation: proceedings of the AHFE 2017 international conference on human factors in transportation. Springer International Publishing, Cham, pp 595–605Google Scholar
  8. 8.
    Croo HD, Bandmann M, Mackay GM, Rumar K, Vollenhoven P (2001) The role of driver fatigue in commercial road transport crashes. European Transport Safety Council, Brussels, BelgiumGoogle Scholar
  9. 9.
    Karrer-Gauß K (2012) Prospektive Bewertung von Systemen zur Müdigkeitserkennung - Ableitung von Gestaltungsempfehlungen zur Vermeidung von Risikokompensation aus empirischen Untersuchungen. Technische Universität Berlin (2012)Google Scholar
  10. 10.
    May JF, Baldwin CL (2009) Driver fatigue. The importance of identifying causal factors of fatigue when considering detection and countermeasure technologies. Transp Res Part F Traffic Psychol Behav. Scholar
  11. 11.
    Åkerstedt T, Gillberg M (2009) Subjective and objective sleepiness in the active individual. Int J Neurosci. Scholar
  12. 12.
    Hoddes E, Zarcone V, Smythe H, Phillips R, Dement WC (1973) Quantification of sleepiness. A new approach. Psychophysiology. Scholar
  13. 13.
    Goncalves J, Happee R, Bengler K (2016) Drowsiness in conditional automation. Proneness, diagnosis and driving performance effects. In: Proceedings of the 2016 IEEE 19th international conference on intelligent transportation systems (ITSC), Rio de Janeiro, Brazil, pp 873–878Google Scholar
  14. 14.
    Jarosch O, Kuhnt M, Paradies S, Bengler K (2017) It’s out of our hands now! effects of non-driving related tasks during highly automated driving on drivers’ fatigue. In: Proceedings of the 9th international driving symposium on human factors in driver assessment, training, and vehicle design: driving assessment 2017. Driving assessment conference, Manchester Village, Vermont, USA. University of Iowa, Iowa City, Iowa, 26–29 June 2017, pp 319–325.
  15. 15.
    Knipling RR, Wierwille WW (1994) Vehicle-based drowsy driver detection. Current status and future prospects. In: IVHS America fourth annual meeting, AtlantaGoogle Scholar
  16. 16.
    Feldhütter A, Feierle A, Kalb L, Bengler K (2018) A new approach for a real-time non-invasive fatigue assessment system for automated driving. In: Proceedings of the human factors and ergonomics society (HFES) (in Press)Google Scholar
  17. 17.
    Dong Y, Hu Z, Uchimura K, Murayama N (2011) Driver inattention monitoring system for intelligent vehicles. A review. IEEE Trans Intell Transp Syst.
  18. 18.
    Coetzer RC, Hancke GP (2009) Driver fatigue detection. A survey. In: Proceedings of the AFRICON, AFRICON 2009, Nairobi, Kenya. IEEE, pp 1–6.
  19. 19.
    Vicente J, Laguna P, Bartra A, Bailón R (2016) Drowsiness detection using heart rate variability. Med Biol Eng Comput. Scholar
  20. 20.
    Hargutt V (2000) Eyelid movements and their predictive value of fatigue stages. In: 3rd international conference of psychophysiology in ergonomics, San Diego, California, 30.07.2000 (2000)Google Scholar
  21. 21.
    Wierwille WW, Wreggit SS, Kirn CL, La Ellsworth, Fairbanks, R.J (1994) Research on vehicle-based driver status/performance monitoring; development, validation, and refinement of algorithms for detection of driver drowsiness. Final report. U.S. Department of Transportation. Springfield, VirginiaGoogle Scholar
  22. 22.
    Murata A, Koriyama T, Hayami T (2012) A basic study on the prevention of drowsy driving using the change of neck bending and the sitting pressure distribution. In: Proceedings of SICE (Society of Instrument and Control Engineers) Annual Conference 2012, Akita, pp 274–279Google Scholar
  23. 23.
    Schmidt J. Braunagel C, Stolzmann W, Karrer-Gauss K (2016) Driver drowsiness and behavior detection in prolonged conditionally automated drives. In: 2016 IEEE intelligent vehicles symposium (IV), Gotenburg, Sweden. IEEE, Piscataway, NJ, 19–22 June 2016, pp 400–405.
  24. 24.
    Regan MA, Hallett C, Gordon CP (2011) Driver distraction and driver inattention. Definition, relationship and taxonomy. Accident; analysis and prevention.
  25. 25.
    Schooler JW, Smallwood J, Christoff K, Handy TC, Reichle ED, Sayette MA (2011) Meta-awareness, perceptual decoupling and the wandering mind. Trends Cogn Sci. Scholar
  26. 26.
    Oken BS, Salinsky MC, Elsas SM (2006) Vigilance, alertness, or sustained attention. physiological basis and measurement. Clin Neurophysiol. Scholar
  27. 27.
    Schmidt EA, Schrauf M, Simon M, Fritzsche M, Buchner A, Kincses WE (2009) Drivers’ mis-judgement of vigilance state during prolonged monotonous daytime driving. Accid Anal Prev. Scholar
  28. 28.
    Sathyanarayana A, Nageswaren S, Ghasemzadeh H, Jafari R, Hansen JHL (2008) Body sensor networks for driver distraction identification. In: Proceedings of the 2008 IEEE international conference on vehicular electronics and safety (ICVES 2008), Columbus, OH, 22.09.2008–24.09.2008, Columbus, USA, pp 120–125.
  29. 29.
    Azman A, Meng Q, Edirisinghe E (2010) Non intrusive physiological measurement for driver cognitive distraction detection. Eye and Mouth Movements. In: 2010 3rd international conference on advanced computer theory and engineering (ICACTE). IEEE, pp 595–599 (2010)Google Scholar
  30. 30.
    Körber M, Cingel A, Zimmermann M, Bengler K (2015) Vigilance decrement and passive fatigue caused by monotony in automated driving. Procedia Manuf. Scholar
  31. 31.
    Young K, Regan M (2007) Driver distraction. A review of the literature. In: Faulks IJ, Regan M, Stevenson M, Brown J, Porter A, Irwin JD (ed) Distracted driving, NSW, Sydney, pp 379–405 (2007)Google Scholar
  32. 32.
    Selye H (1980) Selye’s guide to stress research. Van Nostrand Reinhold, New YorkGoogle Scholar
  33. 33.
    Conti-Kufner A-S (2017) Measuring cognitive task load: an evaluation of the Detection Response Task and its implications for driver distraction assessment. Dissertation, Technische Universität München.
  34. 34.
    Wickens CD (2008) multiple resources and mental workload. Hum Factors. Scholar
  35. 35.
    Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index). Results of empirical and theoretical research. In: Meshkati N, Hancock PA (eds) Human Mental Workload. Advances in Psychology, vol 52, 1st edn. Elsevier textbooks, s.l., pp 139–183 (1988)Google Scholar
  36. 36.
    Pauzié A (2008) A method to assess the driver mental workload. The driving activity load index (DALI). IET Intell Transp Syst. Scholar
  37. 37.
    Kahneman D, Tursky B, Shapiro D, Crider A (1969) Pupillary, heart rate, and skin resistance changes during a mental task. J Exp Psychol. Scholar
  38. 38.
    Itoh M (2009) Individual differences in effects of secondary cognitive activity during driving on temperature at the nose tip. In: Proceedings of the 2009 international conference on mechatronics and automation (ICMA), Changchun, China, 09.08.2009–12.08.2009, IEEE, Changchun, China, pp 7–11.
  39. 39.
    Marquart G, Cabrall C, de Winter J (2015) Review of eye-related measures of drivers’ mental workload. Procedia Manuf. Scholar
  40. 40.
    Wang Y, Reimer B, Dobres J, Mehler B (2014) The sensitivity of different methodologies for characterizing drivers’ gaze concentration under increased cognitive demand. Transp Res Part F Traffic Psychol Behav. Scholar
  41. 41.
    Healey JA, Picard RW (2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans Intell Transp Syst. Scholar
  42. 42.
    Yamakoshi T, Yamakoshi K, Tanaka S, Nogawa M, Shibata M, Sawada Y, Rolfe P, Hirose Y (2007) A preliminary study on driver’s stress index using a new method based on differential skin temperature measurement. In: Proceedings of the 29th annual international conference of the IEEE EMBS, vol. 2007, Lyon, France, pp 722–725Google Scholar
  43. 43.
    Lee HB, Choi JM, Kim JS, Kim YS, Baek HJ, Ryu MS, Sohn RH, Park KS (2007) Nonintrusive biosignal measurement system in a vehicle. In: Proceedings of the 29th annual international conference of the IEEE EMBS, Lyon, France, pp 2303–2306Google Scholar
  44. 44.
    Jeong IC, Jun Sh, Lee DH, Yoon HR (2007) Development of bio signal measurement system for vehicles. In: Proceedings of the 2007 international conference on convergence, pp 1091–1096Google Scholar
  45. 45.
    Schmidt EA, Schrauf M, Simon M, Buchner A, Kincses WE (2011) The short-term effect of verbally assessing drivers’ state on vigilance indices during monotonous daytime driving. Transp Res Part F Traffic Psychol Behav. Scholar
  46. 46.
    Lenné MG, Jacobs EE (2016) Predicting drowsiness-related driving events. a review of recent research methods and future opportunities. Theor Issues Ergon Sci. Scholar
  47. 47.
    Heuer S, Chamadiya B, Gharbi A, Kunze C, Wagner M (2010) Unobtrusive in-vehicle biosignal instrumentation for advanced driver assistance and active safety. In: Proceedings of the IEEE EMBS conference on biomedical engineering and sciences (IECBES), Kuala Lumpur, Malaysia. IEEE, Piscataway, pp 252–256.
  48. 48.
    Bergasa LM, Nuevo J, Sotelo MA, Barea R, Lopez ME (2006) Real-Time system for monitoring driver vigilance. IEEE Trans Intell Transp Syst. Scholar
  49. 49.
    Danisman T, Bilasco IM, Djeraba C, Ihaddadene N (2010) Drowsy driver detection system using eye blink patterns. In: 2010 international conference on machine and web intelligence (ICMWI), Algiers, 03.10.2010–05.10.2010. IEEE, pp 230–233.
  50. 50.
    Friedrichs, F., Yang, B.: Camera-based drowsiness reference for driver state classification under real driving conditions. In: Proceedings of the 2010 IEEE intelligent vehicles symposium (IV), La Jolla, CA, USA, 21.06.2010–24.06.2010. IEEE, La Jolla, USA, pp 101–106.
  51. 51.
    Fors C, Ahlström C, Sörner P, Kovaceva J, Hasselberg E, Krantz M, Grönvall J-F, Kircher K, Anund A (2011) Camera-based sleepiness detection. Final report of the project SleepEYE. ViP publication: ViP - Virtual Prototyping and Assessment by Simulation. Statens vägoch transport-forskningsinstitut, LinköpingGoogle Scholar
  52. 52.
    ISO/TS 15007-2:2014 (2014) Road vehicles - Measurement of driver visual behaviour with respect to transport information and control systems: Part 2: Equipment and procedures. International Organization for Standardization, SwitzerlandGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Tobias Hecht
    • 1
    Email author
  • Anna Feldhütter
    • 1
  • Jonas Radlmayr
    • 1
  • Yasuhiko Nakano
    • 2
  • Yoshikuni Miki
    • 2
  • Corbinian Henle
    • 1
  • Klaus Bengler
    • 1
  1. 1.Chair of ErgonomicsTechnical University of MunichGarchingGermany
  2. 2.Denso Ten Europe GmbHDuesseldorfGermany

Personalised recommendations