Vision-Based Driver-Assistance Systems

  • Mahdi Rezaei
  • Reinhard Klette
Part of the Computational Imaging and Vision book series (CIVI, volume 45)


This chapter outlines the general context of the book. Autonomous driving is still at a stage where drivers are expected to be in control of the vehicle at all times, but provided automated control features of the vehicle (based on input data generated by different sensors) already enhance safety and driver comfort. We especially consider automated control features possible by using camera data.


Face Detection Stereo Vision Adaptive Cruise Control Vehicle Detection Autonomous Driving 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 3.
    A. Ali, S. Afghani, Shadow based on-road vehicle detection and verification using Haar wavelet packet transform, in Proceedings of the IEEE Conference on Information Communication Technologies (2005), pp. 346–350Google Scholar
  2. 8.
    Y. Ban, S. Kim, S. Kim, A. Toh, S. Lee, Face detection based on skin color likelihood. Pattern Recognit. 47, 1573–1585 (2013)CrossRefGoogle Scholar
  3. 10.
    J. Barnes, Daimler’s double take sees machine vision move in-vehicle. ITS International, Nov/Dec 2013Google Scholar
  4. 15.
    K. Bengler, K. Dietmayer, B. Farber, M. Maurer, C. Stiller, H. Winner, Three decades of driver assistance systems: review and future perspectives. IEEE Intell. Transp. Syst. Mag. 6, 6–22 (2014)CrossRefGoogle Scholar
  5. 17.
    D. Bétaille, R. Toledo-Moreo, J. Laneurit, Making an enhanced map for lane location based services, in Proceedings of the IEEE Conference on Intelligent Transportation Systems (2008), pp. 711–716Google Scholar
  6. 25.
    A. Briassouli, I. Kompatsiaris, Change detection for temporal texture in the Fourier domain, in Proceedings of the Asian Conference on Computer Vision. LNCS 6492 (2010), pp. 149–160Google Scholar
  7. 28.
    H. Chang, A. Haizhou, L. Yuan, L. Shihong, High-performance rotation invariant multiview face detection. IEEE Trans. Pattern Anal. Mach. Intell. 29, 671–686 (2007)CrossRefGoogle Scholar
  8. 31.
    S.Y. Cheung, P.P. Varaiya, Traffic surveillance by wireless sensor networks: final report. Institute of Transportation Studies, University of California at Berkeley (2007)Google Scholar
  9. 32.
    J. Choi, Realtime on-road vehicle detection with optical flows and Haar-like feature detectors. University of Illinois at Urbana-Champaign (2007)Google Scholar
  10. 36.
    P. Corke, J. Lobo, J. Dias, An introduction to inertial and visual sensing. Int. J. Robot. Res. 26, 519–535 (2007)CrossRefGoogle Scholar
  11. 39.
    W. Cunningham, US requiring back-up cameras in cars by 2018. Road Show by Cnet (2014),
  12. 40.
    A.G. Daimler, First assistance system in the fight against vehicles driving on the wrong side of the road (2013), Google Scholar
  13. 42.
    R. Danescu, S. Nedevschi, Probabilistic lane tracking in difficult road scenarios using stereovision. IEEE Trans. Intell. Transp. Syst. 10, 272–282 (2009)CrossRefGoogle Scholar
  14. 44.
    M. Debord, 3 reasons why the auto industry should be terrified of the Google car. Business Insider Australia (2014)Google Scholar
  15. 47.
    H. Deusch, J. Wiest, S. Reuter, M. Szczot, M. Konrad, K. Dietmayer, A random finite set approach to multiple lane detection, in Proceedings of the IEEE Conference on Intelligent Transportation Systems (2012), pp. 270–275Google Scholar
  16. 51.
    A. Doshi, B. Morris, M. Trivedi, On-road prediction of driver’s intent with multimodal sensory cues. IEEE Pervasive Comput. 10, 22–34 (2011)CrossRefGoogle Scholar
  17. 55.
    M.L. Eichner, T.P. Breckon, Integrated speed limit detection and recognition from real-time video, in Proceedings of the IEEE Intelligent Vehicles Symposium (2008), pp. 626–631Google Scholar
  18. 56.
    The .enpeda.. image sequence analysis test site (2014),
  19. 60.
    Y. Fei, M. Adams, S. Roy, V2V wireless communication protocol for rear-end collision avoidance on highways, in Proceedings of the IEEE Conference on Communications Workshops (2008), pp. 375–379Google Scholar
  20. 61.
    P.F. Felzenszwalb, R.B. Girshick, D. McAllester, Cascade object detection with deformable part models, in Proceedings of the IEEE Computer Vision Pattern Recognition (2010), pp. 2241–2248Google Scholar
  21. 63.
    P.F. Felzenszwalb, R.B. Girshick, D. McAllester, D. Ramanan, Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1627–1645 (2010)CrossRefGoogle Scholar
  22. 68.
    U. Franke, D. Pfeiffer, C. Rabe, C. Knoeppel, M. Enzweiler, F. Stein, R.G. Herrtwich, Making Bertha see, in Proceedings of the IEEE Conference on Computer Vision Workshops (2013), pp. 214–221Google Scholar
  23. 72.
    T. Gandhi, M.M. Trivedi, Vehicle surround capture: survey of techniques and a novel omni-video-based approach for dynamic panoramic surround maps. IEEE Trans. Intell. Transp. Syst. 7, 293–308 (2006)CrossRefGoogle Scholar
  24. 74.
    F. Garcia, P. Cerri, A. Broggi, A. De la Escalera, J.M. Armingol, Data fusion for overtaking vehicle detection based on radar and optical flow, in Proceedings of the IEEE Intelligent Vehicles Symposium (2012), pp. 494–499Google Scholar
  25. 76.
    A. Geiger, J. Ziegler, C. Stiller, StereoScan: dense 3D reconstruction in real-time, in Proceedings of the IEEE Intelligent Vehicles Symposium (2011), pp. 963–968Google Scholar
  26. 79.
    H. Grabner, T.T. Nguyen, B. Gruber, H. Bischof, On-line boosting-based car detection from aerial images. ISPRS J. Photogramm. Remote Sens. 63, 382–396 (2008)CrossRefGoogle Scholar
  27. 80.
    G. Grubb, E. Jakobsson, A. Beutner, M. Ahrholdt, S. Bergqvist, Automatic queue assistance to aid under-loaded drivers, in Proceedings of the ITS World Congress Exhibition Intelligent Transport Systems Services (2009), pp. 1–8Google Scholar
  28. 82.
    R.A. Gupta, W. Snyder, W.S. Pitts, Concurrent visual multiple lane detection for autonomous vehicles, in Proceedings of the IEEE Conference on Robotics Automation (2010), pp. 2416–2422Google Scholar
  29. 84.
    S. Han, Y. Han, H. Hahn, Vehicle detection method using Haar-like feature on real time system. World Acad. Sci. Eng. Technol. (2009), pp. 455–459Google Scholar
  30. 86.
    A. Haselhoff, A. Kummert, G. Schneider, Radar-vision fusion for vehicle detection by means of improved Haar-like feature and AdaBoost approach, in Proceedings of the European Association Signal Processing (2007), pp. 2070–2074Google Scholar
  31. 87.
    Heidelberg robust vision challenge at ECCV (2012),
  32. 92.
    B. Hongliang, W. Jianping, L. Changpin, Motion and Haar-like features based vehicle detection, in Proceedings of the Multi-Media Modelling (2006), pp. 1–4Google Scholar
  33. 93.
    F. Huang, R. Klette, City-scale modeling towards street navigation applications. J. Inform. Converg. Commun. Eng. 10, 411–419 (2012)Google Scholar
  34. 100.
    S.H. Jeong, C.G. Choi, J.N. Oh, P.J. Yoon, B.S. Kim, M. Kim, K.H. Lee, Low cost design of parallel parking assist system based on an ultrasonic sensor. Int. J. Automot. Technol. 11, 409–416 (2010)CrossRefGoogle Scholar
  35. 107.
    B. Jun, D. Kim, Robust face detection using local gradient patterns and evidence accumulation. Pattern Recognit. 45, 3304–3316 (2012)CrossRefGoogle Scholar
  36. 108.
    S.-J. Jung, H.-S. Shin, W.-Y. Chung, Driver fatigue and drowsiness monitoring system with embedded electrocardiogram sensor on steering wheel. IET Intell. Transp. Syst. 8, 43–50 (2014)CrossRefGoogle Scholar
  37. 109.
    R.K. Jurgen (ed.), Adaptive Cruise Control (SAE International, Warrendale, 2006)Google Scholar
  38. 114.
    A. Kheyrollahi, T.P. Breckon, Automatic real-time road marking recognition using a feature driven approach. Mach. Vis. Appl. 23, 123–133 (2012)CrossRefGoogle Scholar
  39. 118.
    The KITTI Vision Benchmark Suite (2013),
  40. 119.
    R. Klette, Concise Computer Vision: An Introduction into Theory and Algorithms (Springer, London, 2014)CrossRefMATHGoogle Scholar
  41. 120.
    R. Klette, N. Krüger, T. Vaudrey, K. Pauwels, M. van Hulle, S. Morales, F. Kandil, R. Haeusler, N. Pugeault, C. Rabe, M. Lappe, Performance of correspondence algorithms in vision-based driver assistance using an online image sequence database. IEEE Trans. Veh. Technol. 60, 2012–2026 (2011)CrossRefGoogle Scholar
  42. 129.
    L. Le, A. Festag, R. Baldessari, W. Zhang, V2X communication and intersection safety, in Advanced Microsystems for Automotive Applications. VDI-Buch (Springer, Berlin/Heidelberg, 2009), pp. 97–107Google Scholar
  43. 138.
    P. Lindner, G. Wanielik, 3D LIDAR processing for vehicle safety and environment recognition, in Proceedings of IEEE Workshop Computational Intelligence in Vehicles and Vehicular Systems (2009), pp. 66–71Google Scholar
  44. 157.
    J.C. McCall, M.M. Trivedi, Driver behavior and situation aware brake assistance for intelligent vehicles. Proc. IEEE 95, 374–387 (2007)CrossRefGoogle Scholar
  45. 160.
    Mercedes Benz Tech Centre, Distronic Plus with Steering Assist (2013),
  46. 166.
    G. Monteiro, M. Ribeiro, J. Marcos, J. Batista, Wrongway drivers detection based on optical flow, in Proceedings of IEEE International Conference on Image Processing, vol. 5 (2007), pp. 141–144Google Scholar
  47. 167.
    S. Morales, Performance evaluation tools for stereo vision analysis in uncontrolled environments. PhD thesis, Department of Computer Science, The University of Auckland (2012)Google Scholar
  48. 170.
    S. Motoyama, T. Ohta, T. Watanabe, Y. Ito, Development of lane departure warning system, in Proceedings of ITS World Congress (2000), pp. 1–8Google Scholar
  49. 171.
    F. Moutarde, B. Stanciulescu, A. Breheret, Real-time visual detection of vehicles and pedestrians with new efficient AdaBoost features, in Proceedings of 2nd Workshop Planning Perception Navigation Intelligent Vehicles (2008), pp. 1–7Google Scholar
  50. 174.
    E. Murphy-Chutorian, M.M. Trivedi, Head pose estimation in computer vision: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 31, 607–626 (2009)CrossRefGoogle Scholar
  51. 180.
    J. Ocken, V2X telematics: taking ADAS to the next level (2011), Google Scholar
  52. 181.
    E. Ohn-Bar, A. Tawari, S. Martin, M.M. Trivedi, On surveillance for safety critical events: in-vehicle video networks for predictive driver assistance systems. Comput. Vis. Image Underst. 134, 130–140 (2015)CrossRefGoogle Scholar
  53. 182.
    E. Ohn-Bar, M.M. Trivedi, Beyond just keeping hands on the wheel: towards visual interpretation of driver and motion patterns, in Proceedings of IEEE Conference on Intelligent Transportation Systems (2014), pp. 1245–1250Google Scholar
  54. 184.
    T. Ojala, M. Pietikainen, T. Maenpaa, Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 971–987 (2002)CrossRefMATHGoogle Scholar
  55. 187.
    M. Osadchy, Y. Le Cun, M.L. Miller, Synergistic face detection and pose estimation with energy-based models. J. Mach. Learn. Res. 8, 1197–1215 (2007)Google Scholar
  56. 188.
    H. Pan, Y. Zhu, L. Xia, Efficient and accurate face detection using heterogeneous feature descriptors and feature selection. Comput. Vis. Image Underst. 117, 12–28 (2013)CrossRefGoogle Scholar
  57. 190.
    M. Pavlic, G. Rigoll, S. Ilic, Classification of images in fog and fog-free scenes for use in vehicles, in Proceedings of IEEE Intelligent Vehicles Symposium (2013), pp. 481–486Google Scholar
  58. 193.
    F. Peyret, J. Laneurit, D. Bétaille, A novel system using enhanced digital maps and WAAS for a lane-level positioning, in Proceedings of World Congress Intelligent Transport Systems (2008), pp. 1–12Google Scholar
  59. 201.
    R. Preiß, C. Gruner, T. Schilling, H. Winter, Mobile vision – developing and testing of visual sensors for driver assistance systems, in Proceedings of Advanced Microsystems Automotive Applications. VDI-Buch (2004), pp. 95–107Google Scholar
  60. 202.
    C. Premebida, G. Monteiro, U. Nunes, P. Peixoto, A Lidar and vision-based approach for pedestrian and vehicle detection and tracking, in Proceedings of IEEE Conference on Intelligent Transportation Systems Conference (2007), pp. 1044–1049Google Scholar
  61. 203.
    Z.M. Qian, H.X. Shi, J.K. Yang, Video vehicle detection based on local feature. Adv. Mater. Res. 186, 56–60 (2011)CrossRefGoogle Scholar
  62. 206.
    M. Rezaei, R. Klette, 3D cascade of classifiers for open and closed eye detection in driver distraction monitoring, in Proceedings of Computer Analysis Images Patterns. LNCS 6855 (2011), pp. 171–179Google Scholar
  63. 210.
    M. Rezaei, R. Klette, Novel adaptive eye detection and tracking for challenging lighting condition, in Proceedings of Asian Conference on Computer Vision Workshops. LNCS 7729 (2013), pp. 427–440Google Scholar
  64. 211.
    M. Rezaei, R. Klette, Look at the driver, look at the road: No distraction! No accident! in Proceedings of IEEE Computer Vision Pattern Recognition (2014), pp. 129–136Google Scholar
  65. 214.
    M. Rezaei, M. Terauchi, iROADS dataset (Intercity Roads and Adverse Driving Scenarios), EISATS, Set 10 (2014), Google Scholar
  66. 218.
    G. Ros, A. Sappa, D. Ponsa, A.M. Lopez, Visual SLAM for driverless cars: a brief survey, in Proceedings of IEEE Intelligent Symposium on Workshop (2012), pp. 1–6Google Scholar
  67. 219.
    G. Ros, S. Ramos, M. Granados, A.H. Bakhtiary, D. Vazquez, A.M. Lopez, in Proceedings of IEEE Winter Conference on Applications Computer Vision (2015), pp. 231–238Google Scholar
  68. 222.
    J.C. Rubio, J. Serrat, A.M. Lopez, D. Ponsa, Multiple target tracking for intelligent headlights control. IEEE Trans. Intell. Transp. Syst. 13, 594–609 (2012)CrossRefGoogle Scholar
  69. 224.
    H. Ryu, J. Yoon, S. Chun, S. Sull, Coarse-to-fine classification for image-based face detection, in Proceedings of International Conference on Image Video Retrieval (2006), pp. 291–299Google Scholar
  70. 226.
    M.J. Saberian, N. Vasconcelos, Learning optimal embedded cascades. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2005–2018 (2012)CrossRefGoogle Scholar
  71. 228.
    M. Sarshar, M. Rezaei, A novel system for advanced driver assistance systems, in Proceedings of the IEEE Systems Conference (2013), pp. 529–534Google Scholar
  72. 231.
    Y. Sekimoto, Y. Matsubayashi, H. Yamada, R. Imai, T. Usui, H. Kanasugi, Light weight lane positioning of vehicles using a smartphone GPS by monitoring the distance from the center line, in Proceedings of the IEEE Conference on Intelligent Transportation Systems (2012), pp. 1561–1565Google Scholar
  73. 232.
    A. Selloum, D. Bétaille, E. Le Carpentier, F. Peyret, Robustification of a map aided location process using road direction, in Proceedings of the IEEE Conference on Intelligent Transportation Systems (2010), pp. 1504–1510Google Scholar
  74. 236.
    B.-S. Shin, D. Caudillo, R. Klette, Evaluation of two stereo matchers on long real-world video sequences. Pattern Recognit. 48, 113–1124 (2014)Google Scholar
  75. 241.
    S. Sivaraman, M.M. Trivedi, Looking at vehicles on the road: a survey of vision-based vehicle detection, tracking, and behavior analysis. IEEE Conf. Intell. Transp. Syst. 14, 1773–1795 (2013)CrossRefGoogle Scholar
  76. 243.
    M.Á. Sotelo, J. Barriga, Blind spot detection using vision for automotive applications. J. Zhejiang Univ. Sci. A 9, 1369–1372 (2008)CrossRefGoogle Scholar
  77. 244.
    G.P. Stein, Y. Gdalyahu, A. Shashua, Stereo-assist: top-down stereo for driver assistance systems, in Proceedings of the IEEE Intelligent Vehicles Symposium (2010), pp. 723–730Google Scholar
  78. 246.
    V.B. Subburaman, S. Marcel, Alternative search techniques for face detection using location estimation and binary features. Comput. Vis. Image Underst. 117, 551–570 (2013)CrossRefGoogle Scholar
  79. 250.
    J. Tao, B.-S. Shin, R. Klette, Wrong roadway detection for multi-lane roads, in Proceedings of the International Conference on Computer Analysis Images Patterns (2013), pp. 50–58Google Scholar
  80. 251.
    A. Tawari, K.H. Chen, M.M. Trivedi, Where is the driver looking: analysis of head, eye and iris for robust gaze zone estimation, in Proceedings of the IEEE Intelligent Transportation Systems Conference (2014), pp. 988–994Google Scholar
  81. 252.
    A. Tawari, S. Sivaraman, M.M. Trivedi, T. Shanon, M. Tippelhofer, Looking-in and looking-out vision for urban intelligent assistance: estimation of driver attention and dynamic surround for safe merging and braking, in Proceedings of the IEEE Intelligent Vehicles Symposium (2014), pp. 115–120Google Scholar
  82. 253.
    Technology award for BLIS in 2006 (2006),
  83. 257.
    G. Toulminet, M. Bertozzi, S. Mousset, A. Bensrhair, A. Broggi, Vehicle detection by means of stereo vision-based obstacles features extraction and monocular pattern analysis. IEEE Trans. Image Process. 15, 2364–2375 (2006)CrossRefGoogle Scholar
  84. 258.
    C. Tran, A. Doshi, M.M. Trivedi, Modeling and prediction of driver behavior by foot gesture analysis. Comput. Vis. Image Underst. 116, 435–445 (2012)CrossRefGoogle Scholar
  85. 260.
    M.M. Trivedi, S.Y. Cheng, E. Childers, S. Krotosky, Occupant posture analysis with stereo and thermal infrared video: algorithms and experimental evaluation. IEEE Trans. Veh. Technol. 53, 1698–1712 (2004)CrossRefGoogle Scholar
  86. 261.
    M.M. Trivedi, T. Gandhi, J. McCall, Looking-in and looking-out of a vehicle: computer-vision-based enhanced vehicle safety. IEEE Trans. Intell. Transp. Syst. 8, 108–120 (2007)CrossRefGoogle Scholar
  87. 266.
    U.S. Department of Transportation, Research and Innovation Technology Administration, Vehicle-to-vehicle (V2V) communications for safety (2013),
  88. 268.
    N. Vinh Dinh, N. Thuy Tuong, N. Dung Duc, J. Jae Wook, Toward real-time vehicle detection using stereo vision and an evolutionary algorithm, in Proceedings of the Vehicular Technology Conference (2012), pp. 1–5Google Scholar
  89. 269.
    P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, in Proceedings of the IEEE Computer Vision Pattern Recognition, vol. 1 (2001), pp. 511–518Google Scholar
  90. 270.
    P. Viola, M. Jones, Robust real-time face detection, in IEEE International Conference on Computer Vision (2001), pp. 1–8Google Scholar
  91. 271.
    P. Viola, M. Jones, Robust real-time face detection. Int. J. Comput. Vis. 57, 137–154 (2004)CrossRefGoogle Scholar
  92. 274.
    C. Wang, H. Zhang, M. Yang, X. Wang, L. Ye, C. Guo, Automatic parking based on a bird’s eye view vision system. Adv. Mech. Eng. 6, 1–13 (2014)Google Scholar
  93. 281.
    Y. Wei, U. Stilla, Comparison of two methods for vehicle extraction from airborne LiDAR data toward motion analysis. Geosci. Remote Sens. Lett. 8, 607–611 (2011)CrossRefGoogle Scholar
  94. 283.
    WHO 2013 Global status report on road safety (2013),
  95. 286.
    Wikipedia, Automobile safety (2013),
  96. 288.
    Wikipedia, Radar gun (2013),
  97. 296.
    J. Xiao, T. Fang, P. Zhao, M. Lhuilier, L. Quan, Image-based street-side city modeling. ACM Trans. Graph. 28, article no.114 (2009)Google Scholar
  98. 299.
    W. Xuezhi, Y. Huai, Y. Chunyang, S. Chunyan, D. Bobo, Z. Hong, Improved Haar wavelet feature extraction approaches for vehicle detection, in Proceedings of the IEEE Conference on Intelligent Transportation Systems (2007), pp. 1050–1053Google Scholar
  99. 304.
    M. Yu, G. Ma, \(360^{\circ }\) surround view system with parking guidance. SAE Int. J. Commer. Veh. 7, 19–24 (2014)Google Scholar
  100. 308.
    S. Zehang, G. Bebis, R. Miller, On-road vehicle detection: a review. IEEE Trans. Pattern Anal. Mach. Intell. 28, 694–711 (2006)CrossRefGoogle Scholar
  101. 309.
    Y. Zeng, R. Klette, Multi-run 3D streetside reconstruction from a vehicle, in Proceedings of the International Conference on Computer Analysis Images Patterns. LNCS 8047 (2013), pp. 580–588Google Scholar
  102. 310.
    C. Zhang, Z. Zhang, A survey of recent advances in face detection. Microsoft Research. Technical Report MSR-TR-2010-66 (2010)Google Scholar
  103. 314.
    K. Zhao, M. Meuter, C. Nunn, D. Muller, S. Muller-Schneiders, J. Pauli, A novel multi-lane detection and tracking system, in Proceedings of the IEEE Intelligent Vehicles Symposium (2012), pp. 1084–1089Google Scholar
  104. 317.
    P. Zoratti, Automotive driver assistance systems: using the processing power of FPGAs. EE Catalog, vol. 1.0 (2011), pp. 1–8Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Mahdi Rezaei
    • 1
  • Reinhard Klette
    • 2
  1. 1.Department of Computer EngineeringQazvin Islamic Azad UniversityQazvinIran
  2. 2.Department of Electrical and Electronic EngineeringAuckland University of TechnologyAucklandNew Zealand

Personalised recommendations