Driver-Environment Understanding

  • Mahdi Rezaei
  • Reinhard Klette
Chapter
Part of the Computational Imaging and Vision book series (CIVI, volume 45)

Abstract

This book focuses in particular on driver-environment understanding as briefly outlined at the end of the previous chapter. This chapter provides a more detailed introduction, motivations, and a review of the state-of-the-art in this area of vision-based driver-assistance systems. The chapter also discusses existing challenges and outlines the structure of the book.

Bibliography

  1. 1.
    6D Vision (2014), www.6d-vision.com
  2. 3.
    A. Ali, S. Afghani, Shadow based on-road vehicle detection and verification using Haar wavelet packet transform, in Proceedings of the IEEE Conference on Information Communication Technologies (2005), pp. 346–350Google Scholar
  3. 5.
    J.M. Alvarez, A.M. Lopez, T. Gevers, F. Lumbreras, Combining priors, appearance and context for road detection. IEEE Trans. Intell. Transp. Syst. 15, 1168–1178 (2014)CrossRefGoogle Scholar
  4. 7.
    H. Badino, U. Franke, D. Pfeiffer, The stixel world – a compact medium level representation of the 3D-world, in Proceedings of the DAGM – Pattern Recognition (2009), pp. 51–60Google Scholar
  5. 9.
    A. Bar Hillel, R. Lerner, D. Levi, G. Raz, Recent progress in road and lane detection: a survey. Mach. Vis. Appl. 25, 727–747 (2014)CrossRefGoogle Scholar
  6. 11.
    N. Barnes, A. Zelinsky, Real-time radial symmetry for speed sign detection, in Proceedings of the IEEE Intelligent Vehicles Symposium (2004), pp. 566–571Google Scholar
  7. 12.
    A. Barth, Vehicle tracking and motion estimation based on stereo vision sequences. PhD thesis, Bonn University, 2010Google Scholar
  8. 13.
    R. Basri, D.W. Jacobs, Lambertian reflectance and linear subspaces. IEEE IEEE Trans. Pattern Anal. Mach. Intell. 25, 218–233 (2003)CrossRefGoogle Scholar
  9. 14.
    J. Batista, A drowsiness and point of attention monitoring system for driver vigilance, in Proceedings of the IEEE Conference on Intelligent Transportation Systems (2007), pp. 702–708Google Scholar
  10. 16.
    L.M. Bergasa, J. Nuevo, M.A. Sotelo, R. Barea, M.E. Lopez, Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Transp. Syst. 7, 63–77 (2006)CrossRefGoogle Scholar
  11. 20.
    A. Borkar, M. Hayes, M.T. Smith, An efficient method to generate ground truth for evaluating lane detection systems, in Proceedings of the IEEE International Conference on Acoustics Speech Signal Processing (2010), pp. 1090–1093Google Scholar
  12. 24.
    L. Breiman, Random forests. Mach. Learn. 45, 5–32 (2001)CrossRefMATHGoogle Scholar
  13. 29.
    S.G. Charlton, P.H. Baas, Fatigue, work-rest cycles, and psychomotor performance of New Zealand truck drivers. N. Z. J. Psychol. 30, 32–39 (2006)Google Scholar
  14. 38.
    J. Crisman, C. Thorpe, Unscarf: a color vision system for the detection of unstructured roads, in Proceedings of the IEEE Conference on Robotics Automation, vol. 3 (1991) pp. 2496–2501Google Scholar
  15. 41.
    N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, in Proceedings of the IEEE Computer Vision Pattern Recognition (2005), pp. 886–893Google Scholar
  16. 45.
    D. Dementhon, L. Davis, Model-based object pose in 25 lines of code. Int. J. Comput. Vis. 15, 123–141 (1995)CrossRefGoogle Scholar
  17. 49.
    P. Dollar, C. Wojek, B. Schiele, P. Perona, Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 34, 743–761 (2012)CrossRefGoogle Scholar
  18. 50.
    T. D’Orazio, M. Leo, C. Guaragnella, A. Distante, A visual approach for driver inattention detection. Pattern Recognit. 40, 2341–2355 (2007)CrossRefMATHGoogle Scholar
  19. 52.
    A. Doshi, M.M. Trivedi, Head and gaze dynamics in visual attention and context learning, in Proceedings of the IEEE Computer Vision Pattern Recognition Workshops (2009), pp. 77–84Google Scholar
  20. 53.
    DPM Virtual-World Pedestrian Dataset (CVC-07), Computer Vision Center, Universitat Autoǹoma de Barcelona (2014), www.cvc.uab.es/adas/site/?q=node/7
  21. 58.
    S. Escalera, X. Barò, O. Pujol, J. Vitrià, P. Radeva, Traffic-Sign Recognition Systems (Springer, London, 2011)CrossRefGoogle Scholar
  22. 62.
    P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan, Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1627–1645 (2010)CrossRefGoogle Scholar
  23. 66.
    L. Fletcher, A. Zelinsky, Driver inattention detection based on eye gaze—road event correlation. Int. J. Robot. Res. 28, 774–801 (2009)CrossRefGoogle Scholar
  24. 70.
    Y. Freund, R.E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, in Proceedings of the European Conference on Computational Learning Theory (1995), pp. 23–37Google Scholar
  25. 71.
    G.D. Furman, A. Baharav, C. Cahan, S. Akselrod, Early detection of falling asleep at the wheel: a heart rate variability approach, in Proceedings of the Computers in Cardiology (2008), pp. 1109–1112Google Scholar
  26. 73.
    T. Gandhi, M.M. Trivedi, Pedestrian protection systems: issues, survey, and challenges. IEEE Trans. Intell. Transp. Syst. 8, 413–430 (2007)CrossRefGoogle Scholar
  27. 75.
    I. Garcia, S. Bronte, L.M. Bergasa, N. Hernandez, B. Delgado, M. Sevillano, Vision-based drowsiness detector for a realistic driving simulator, in Proceedings of the IEEE Conference on Intelligent Transportation Systems (2010), pp. 887–894Google Scholar
  28. 78.
    D. Geronimo, A.M. Lopez, Vision-Based Pedestrian Protection Systems for Intelligent Vehicles. Springer Briefs in Computer Science (Springer, New York, 2013)Google Scholar
  29. 86.
    A. Haselhoff, A. Kummert, G. Schneider, Radar-vision fusion for vehicle detection by means of improved Haar-like feature and AdaBoost approach, in Proceedings of the European Association Signal Processing (2007), pp. 2070–2074Google Scholar
  30. 101.
    O. Jesorsky, K.J. Kirchberg, R.W. Frischholz, Robust face detection using the Hausdorff distance, in Proceedings of the International Conference on Audio-and Video-Based Biometric Person Authentication (Springer, Berlin/Heidelberg, 2001), pp. 90–95CrossRefGoogle Scholar
  31. 105.
    X. Jie, H. Chen, W. Ding, C. Zhao, J. Morris, Robust optical flow for driver assistance, in Proceedings of the Image and Vision Computing New Zealand (2010), pp. 1–7Google Scholar
  32. 106.
    H. Jing, S.R. Kumar, M. Mitra, Z. Wei-Jing, R. Zabih, Image indexing using color correlograms, in Proceedings of the IEEE Computer Vision Pattern Recognition (1997), pp. 762–768Google Scholar
  33. 107.
    B. Jun, D. Kim, Robust face detection using local gradient patterns and evidence accumulation. Pattern Recognit. 45, 3304–3316 (2012)CrossRefGoogle Scholar
  34. 111.
    A. Kasinski, A. Schmidt, The architecture and performance of the face and eyes detection system based on the Haar cascade classifiers. Pattern Anal. Appl. 13, 197–211 (2010)MathSciNetCrossRefGoogle Scholar
  35. 115.
    Z. Kim, Robust lane detection and tracking in challenging scenarios. IEEE Trans. Intell. Transp. Syst. 9, 16–26 (2008)CrossRefGoogle Scholar
  36. 118.
    The KITTI Vision Benchmark Suite (2013), www.cvlibs.net/datasets/kitti/
  37. 119.
    R. Klette, Concise Computer Vision: An Introduction into Theory and Algorithms (Springer, London, 2014)CrossRefMATHGoogle Scholar
  38. 120.
    R. Klette, N. Krüger, T. Vaudrey, K. Pauwels, M. van Hulle, S. Morales, F. Kandil, R. Haeusler, N. Pugeault, C. Rabe, M. Lappe, Performance of correspondence algorithms in vision-based driver assistance using an online image sequence database. IEEE Trans. Veh. Technol. 60, 2012–2026 (2011)CrossRefGoogle Scholar
  39. 123.
    L. Kneip, M. Chli, R. Siegwart, Robust real-time visual odometry with a single camera and an IMU, in Proceedings of British Machine Vision Conference (2011), pp. 16.1–16.11Google Scholar
  40. 130.
    K. Lee, J. Ho, D. Kriegman, Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 27, 684–698 (2005)CrossRefGoogle Scholar
  41. 137.
    Y. Lin, F. Guo, S. Li, Road obstacle detection in stereo vision based on UV-disparity. J. Inf. Comput. Sci. 11, 1137–1144 (2014)CrossRefGoogle Scholar
  42. 141.
    M. Ljung, H. Fagerlind, P. Lövsund, J. Sandin, Accident investigations for active safety at CHALMERS – new demands require new methodologies. Veh. Syst. Dyn. 45, 881–894 (2007)CrossRefGoogle Scholar
  43. 143.
    C. Long, X. Wang, G. Hua, M. Yang, Y. Lin, Accurate object detection with location relaxation and regionlets relocalization, in Proceedings of Asian Conference of Computer Vision (2014), pp. 260–275Google Scholar
  44. 144.
    A.M. Lopez, J. Serrat, C. Canero, F. Lumbreras, T. Graf, Robust lane markings detection and road geometry computation. Int. J. Automot. Technol. 11, 395–407 (2010)CrossRefGoogle Scholar
  45. 146.
    M.J. Lyons, S. Akamatsu, M. Kamachi, J. iro Gyoba, The Japanese female facial expression database (2013), www.kasrl.org/jaffe.html
  46. 148.
    A.M. Malla, P.R. Davidson, P.J. Bones, R. Green, R.D. Jones, Automated video-based measurement of eye closure for detecting behavioral microsleep, in Proceedings of IEEE International Conference on Engineering Medicine Biology Society (2010), pp. 6741–6744Google Scholar
  47. 150.
    J. Marin, D. Vazquez, A.M. Lopez, J. Amores, B. Leibe, Random forests of local experts for pedestrian detection, in Proceedings of IEEE International Conference on Computer Vision (2013), pp. 2592–2599Google Scholar
  48. 153.
    P. Martins, J. Batista, Monocular head pose estimation, in Proceedings of International Conference on Image Analysis Recognition (2008), pp. 357–368Google Scholar
  49. 156.
    J.C. McCall, M.M. Trivedi, Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation. IEEE Trans. Intell. Transp. Syst. 7, 20–37 (2006)CrossRefGoogle Scholar
  50. 159.
    F. Meng-Yin, H. Yuan-Shui, A survey of traffic sign recognition, in Proceedings of International Conference on Wavelet Analysis Pattern Recognition (2010), pp. 119–124Google Scholar
  51. 161.
    T.P. Michalke, F. Stein, U. Franke, Towards a closer fusion of active and passive safety: optical flow-based detection of vehicle side collisions, in Proceedings of IEEE Intelligent Vehicle Symposium (2011), pp. 181–188Google Scholar
  52. 163.
    M. Miyaji, M. Danno, H. Kawanaka, K. Oguri, Driver’s cognitive distraction detection using AdaBoost on pattern recognition basis, in Proceedings of IEEE International Conference on Vehicular Electronics Safety (2008), pp. 51–56Google Scholar
  53. 164.
    M. Miyaji, H. Kawanaka, K. Oguri, Effect of pattern recognition features on detection for driver’s cognitive distraction, in Proceedings of IEEE International Conference on Intelligent Transportation Systems (2010), pp. 605–610Google Scholar
  54. 165.
    A. Møgelmose, M.M. Trivedi, T.B. Moeslund, Vision based traffic sign detection and analysis for intelligent driver assistance systems: perspectives and survey. IEEE Trans. Intell. Transp. Syst. 13, 1484–1497 (2012)CrossRefGoogle Scholar
  55. 168.
    S. Morales, R. Klette, A third eye for performance evaluation in stereo sequence analysis, in Proceedings of International Conference on Computer Analysis Images Patterns. LNCS 5702 (2009), pp. 1078–1086Google Scholar
  56. 173.
    S. Müller-Schneiders, C. Nunn, M. Meuter, Performance evaluation of a real time traffic sign recognition system, in Proceedings of IEEE Conference on Intelligent Vehicles Symposium (2008), pp. 79–84Google Scholar
  57. 175.
    E. Murphy-Chutorian, M.M. Trivedi, Head pose estimation and augmented reality tracking: an integrated system and evaluation for monitoring driver awareness. IEEE Trans. Intell. Transp. Syst. 11, 300–311 (2010)CrossRefGoogle Scholar
  58. 176.
    W. Murray, Improving work-related road safety in New Zealand – a research report. Department of Labour, Wellington (2007)Google Scholar
  59. 179.
    New Zealand Ministry of Transport, Motor vehicle crash fact sheets (2010)Google Scholar
  60. 183.
    E. Ohn-Bar, M. Trivedi, Fast and robust object detection using visual subcategories, in Proceedings of IEEE Computer Vision Pattern Recognition Workshops (2014), pp. 179–184Google Scholar
  61. 185.
    R. O’Malley, M. Glavin, E. Jones, Vehicle detection at night based on tail-light detection, in Proceedings of International Symposium on Vehicular Computing Systems, vol. 2224 (2008)Google Scholar
  62. 191.
    M.T.R. Peiris, R.D. Jones, P.R. Davidson, P.J. Bones, Detecting behavioral microsleeps from EEG power spectra, in Proceedings of IEEE Conference on Engineering Medicine Biology Society (2006), pp. 5723–5726Google Scholar
  63. 192.
    M.T.R. Peiris, R.D. Jones, P.R. Davidson, G.J. Carroll, P.J. Bones, Frequent lapses of responsiveness during an extended visuomotor tracking task in non-sleep-deprived subjects. J. Sleep Res. 15, 291–300 (2006)CrossRefGoogle Scholar
  64. 198.
    D. Ponsa, A.M. Lopez, F. Lumbreras, J. Serrat, T. Graf, 3D vehicle sensor based on monocular vision, in Proceedings of IEEE Conference on Intelligent Transportation Systems (2005), pp. 1096–1101Google Scholar
  65. 199.
    D. Ponsa, A.M. Lopez, J. Serrat, F. Lumbreras, T. Graf, Multiple vehicle 3D tracking using an unscented Kalman filter, in Proceedings of IEEE Conference on Intelligent Transportation Systems (2005), pp. 1108–1113Google Scholar
  66. 200.
    E. Portouli, E. Bekiaris, V. Papakostopoulos, N. Maglaveras, On-road experiment for collecting driving behavioural data of sleepy drivers. Somnologie Schlafforschung Schlafmedizin 11, 259–267 (2007)CrossRefGoogle Scholar
  67. 204.
    L. Qiong, P. Guang-zheng, A robust skin color based face detection algorithm, in Proceedings of International Asian Conference on Informatics Control Automation Robotics (2010), pp. 525–528Google Scholar
  68. 207.
    M. Rezaei, R. Klette, Simultaneous analysis of driver behaviour and road condition for driver distraction detection. Int. J. Image Data Fusion 2, 217–236 (2011)CrossRefGoogle Scholar
  69. 211.
    M. Rezaei, R. Klette, Look at the driver, look at the road: No distraction! No accident! in Proceedings of IEEE Computer Vision Pattern Recognition (2014), pp. 129–136Google Scholar
  70. 224.
    H. Ryu, J. Yoon, S. Chun, S. Sull, Coarse-to-fine classification for image-based face detection, in Proceedings of International Conference on Image Video Retrieval (2006), pp. 291–299Google Scholar
  71. 230.
    M. Schreier, V. Willert, Robust free space detection in occupancy grid maps by methods of image analysis and dynamic B-spline contour tracking, in Proceedings of the IEEE Conference on Intelligent Transportation Systems (2012), pp. 514–521Google Scholar
  72. 233.
    R. Senaratne, B. Jap, S. Lal, A. Hsu, S. Halgamuge, P. Fischer, Comparing two video-based techniques for driver fatigue detection: classification versus optical flow approach. Mach. Vis. Appl. 22, 597–618 (2011)CrossRefGoogle Scholar
  73. 236.
    B.-S. Shin, D. Caudillo, R. Klette, Evaluation of two stereo matchers on long real-world video sequences. Pattern Recognit. 48, 113–1124 (2014)Google Scholar
  74. 237.
    B.-S. Shin, Z. Xu, R. Klette, Visual lane analysis and higher-order tasks: a concise review. Mach. Vis. Appl. 25, 1519–1547 (2014)CrossRefGoogle Scholar
  75. 238.
    J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, A. Blake, Real-time human pose recognition in parts from single depth images. Stud. Comput. Intell. 411, 119–135 (2013)Google Scholar
  76. 239.
    M.H. Sigari, Driver hypo-vigilance detection based on eyelid behavior, in Proceedings of the International Conference on Advances Pattern Recognition (2009), pp. 426–429Google Scholar
  77. 240.
    S. Sivaraman, M.M. Trivedi, Looking at vehicles on the road: a survey of vision-based vehicle detection, tracking and behavior analysis. IEEE Trans. Intell. Transp. Syst. 14, 1773–1795 (2013)CrossRefGoogle Scholar
  78. 241.
    S. Sivaraman, M.M. Trivedi, Looking at vehicles on the road: a survey of vision-based vehicle detection, tracking, and behavior analysis. IEEE Conf. Intell. Transp. Syst. 14, 1773–1795 (2013)CrossRefGoogle Scholar
  79. 248.
  80. 259.
    B. Triggs, P. McLauchlan, R. Hartley, A. Fitzgibbon, Bundle adjustment – a modern synthesis, in Proceedings of the Vision Algorithms Theory Practice (2000), pp. 298–375Google Scholar
  81. 265.
    U.S. Department of Transportation, National Highway Traffic Safety Administration. The impact of driver inattention on near-crash/crash risk. DOT HS 810 594 (2006)Google Scholar
  82. 267.
    M. Vargas, J.M. Milla, S.L. Toral, F. Barrero, An enhanced background estimation algorithm for vehicle detection in urban traffic scenes. IEEE Trans. Veh. Technol. 59, 3694–3709 (2010)CrossRefGoogle Scholar
  83. 272.
    Virginia Tech Transportation Institute, 100-car naturalistic driving study fact sheet (2005)Google Scholar
  84. 275.
    H. Wang, L.B. Zhou, Y. Ying, A novel approach for real time eye state detection in fatigue awareness system, in Proceedings of Robotics Automation Mechatronics (2010), pp. 528–532Google Scholar
  85. 276.
    R. Wang, L. Guo, B. Tong, L. Jin, Monitoring mouth movement for driver fatigue or distraction with one camera, in Proceedings of IEEE International Conference on Intelligent Transportation Systems (2004), pp. 314–319Google Scholar
  86. 278.
    X. Wang, M. Yang, S. Zhou, Y. Lin, Regionlets for generic object detection, in Proceedings of the IEEE Confernce on Computer Vision (2013), pp. 17–24Google Scholar
  87. 279.
    A. Wedel, H. Badino, C. Rabe, H. Loose, U. Franke, D. Cremers, B-spline modeling of road surfaces with an application to free-space estimation. IEEE Trans. Intell. Transp. Syst. 10, 572–583 (2009)CrossRefGoogle Scholar
  88. 280.
    A. Wedel, U. Franke, H. Badino, D. Cremers, B-spline modeling of road surfaces for freespace estimation, in Proceedings of the IEEE Intelligent Vehicles Symposium (2008), pp. 828–833Google Scholar
  89. 282.
    W. Wen, C. Xilin, Y. Lei, Detection of text on road signs from video. IEEE Trans. Intell. Transp. Syst. 6, 378–390 (2005)CrossRefGoogle Scholar
  90. 284.
    W.W. Wierwille, L.A. Ellsworth, Evaluation of driver drowsiness by trained raters. Accid. Anal. Prev. 26, 571–581 (1994)CrossRefGoogle Scholar
  91. 285.
    W.S. Wijesoma, K.R.S. Kodagoda, A.P. Balasuriya, Road-boundary detection and tracking using ladar sensing. IEEE Trans. Robot. Autom. 20, 456–464 (2004)CrossRefGoogle Scholar
  92. 289.
    P.I. Wilson, J. Fernandez, Facial feature detection using Haar classifiers. J. Comput. Sci. Coll. 21, 127–133 (2006)Google Scholar
  93. 300.
  94. 301.
    M.H. Yang, D. Kriegman, N. Ahuja, Detecting faces in images: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 24, 34–58 (2002)Google Scholar
  95. 303.
    J.J. Yebes, L.M. Bergasa, R. Arroyo, A. Lazaro, Supervised learning and evaluation of KITTI’s cars detector with DPM, in Proceedings of the IEEE Intelligent Vehicle Symposium (2014), pp. 768–773Google Scholar
  96. 310.
    C. Zhang, Z. Zhang, A survey of recent advances in face detection. Microsoft Research. Technical Report MSR-TR-2010-66 (2010)Google Scholar
  97. 313.
    Z. Zhang, Y. Shan, Incremental motion estimation through local bundle adjustment Microsoft Research. Technical report MSR-TR-01-54 (2001)Google Scholar
  98. 315.
    N. Zhiheng, S. Shiguang, Y. Shengye, C. Xilin, G. Wen, 2D cascaded AdaBoost for eye localization, in Proceedings of the International Conference on Pattern Recognition (2006), pp. 1216–1219Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Mahdi Rezaei
    • 1
  • Reinhard Klette
    • 2
  1. 1.Department of Computer EngineeringQazvin Islamic Azad UniversityQazvinIran
  2. 2.Department of Electrical and Electronic EngineeringAuckland University of TechnologyAucklandNew Zealand

Personalised recommendations