Skip to main content
SpringerLink
Log in

Awareness of Road Scene Participants for Autonomous Driving

  • Reference work entry
Handbook of Intelligent Vehicles

Abstract

This chapter describes detection and tracking of moving objects (DATMO) for purposes of autonomous driving. DATMO provides awareness of road scene participants, which is important in order to make safe driving decisions and abide by the rules of the road. Three main classes of DATMO approaches are identified and discussed. First is the traditional approach, which includes data segmentation, data association, and filtering using primarily Kalman filters. Recent work within this class of approaches has focused on pattern recognition techniques. The second class is the model-based approach, which performs inference directly on the sensor data without segmentation and association steps. This approach utilizes geometric object models and relies on non-parametric filters for inference. Finally, the third class is the grid-based approach, which starts by constructing a low level grid representation of the dynamic environment. The resulting representation is immediately useful for determining free navigable space within the dynamic environment. Grid construction can be followed by segmentation, association, and filtering steps to provide object level representation of the scene. The chapter introduces main concepts, reviews relevant sensor technologies, and provides extensive references to recent work in the field. The chapter also provides a taxonomy of DATMO applications based on road scene environment and outlines requirements for each application.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 849.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Andrieu C, De Freitas N, Doucet A, Jordan M (2003) An introduction to MCMC for machine learning. Mach Learn 50(1):5–43

    Article  MATH  Google Scholar 

  • Arras K, Mozos O, Burgard W (2007) Using boosted features for the detection of people in 2d range data. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Rome, pp 3402–3407

    Google Scholar 

  • Arulampalam S, Maskell S, Gordon N, Clapp T (2002) A tutorial on particle filters for on-line nonlinear/non-gaussian bayesian tracking. IEEE Trans Signal Process 50:174–188

    Article  Google Scholar 

  • Badino H, Franke U, Mester R (2007) Free space computation using stochastic occupancy grids and dynamic programming. In: IEEE international conference on computer vision, workshop on dynamical vision, Rio de Janeiro

    Google Scholar 

  • Bar-Shalom Y, Jaffer A (1972) Adaptive nonlinear filtering for tracking with measurements of uncertain origin. In: Conference on decision and control and 11th symposium on adaptive processes, New Orleans. Institute of Electrical and Electronics Engineers, New York, pp 243–247

    Chapter  Google Scholar 

  • Bertozzi M, Broggi A, Castelluccio S (1997) A real-time oriented system for vehicle detection. J Syst Archit 43:317–325

    Article  Google Scholar 

  • Bertozzi M, Broggi A, Fascioli A (2000) Vision-based intelligent vehicles: state of the art and perspectives. Robot Auton Syst 32:1–16

    Article  Google Scholar 

  • Blackman S, Co R, El Segundo C (2004) Multiple hypothesis tracking for multiple target tracking. IEEE Aerosp Electron Syst Mag 19(1 Part 2):5–18

    Article  Google Scholar 

  • Blackwell D (1947) Conditional expectation and unbiased sequential estimation. Ann Math Stat 18(1):105–110

    Article  MathSciNet  MATH  Google Scholar 

  • Borgefors G (1986) Distance transformations in digital images. Comput Vision Graph Image Process 34:344–371

    Article  Google Scholar 

  • Braillon C, Pradalier C, Usher K, Crowley J, Laugier C (2006) Occupancy grids from stereo and optical flow data. In: Proceedings of international symposium on experimental robotics, Rio de Janeiro

    Google Scholar 

  • Broggi A, Cerri P, Ghidoni S, Grisleri P, Gi J (2008) Localization and analysis of critical areas in urban scenarios. In: Proceedings of IEEE international symposium on intelligent vehicles, Eindhoven, pp 1074–1079

    Google Scholar 

  • Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24:603–619

    Article  Google Scholar 

  • Coué C, Pradalier C, Laugier C, Fraichard T, Bessiére P (2006) Bayesian occupancy filtering for multitarget tracking: an automotive application. Int J Robot Res 25(1):19–30

    Article  Google Scholar 

  • Cox I (1993) A review of statistical data association techniques for motion correspondence. Int J Comput Vis 10(1):53–66

    Article  Google Scholar 

  • Cucchiara R, Piccardi M (1999) Vehicle detection under day and night illumination. In: Proceedings of the 3rd international ICSC symposium on intelligent industrial automation, Genova

    Google Scholar 

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), vol 1, San Diego, pp 886–893

    Google Scholar 

  • Dasarathy B (1997) Sensor fusion potential exploitation – innovative and illustrative applications. In: Proceedings of the IEEE special issue on sensor fusion, vol 85, pp 24–38. IEEE

    Google Scholar 

  • Diard J, Bessiere P, Mazer E (2003) A survey of probabilistic models using the bayesian programming methodology as a unifying framework. In: The second international conference on computational intelligence, robotics and autonomous systems (CIRAS 2003), Singapore

    Google Scholar 

  • Dickmanns E, Behringer R, Dickmanns D, Hilde-brandt T, Maurer M, Thomanek F, Schiehlen J (1994) The seeing passenger car “vamors-p”. In: Proceedings of the intelligent vehicles 1994 symposium, Paris, France, pp 68–73

    Google Scholar 

  • Dollar P, Wojek C, Schiele B, Perona P (2009) Pedestrian detection: a benchmark. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), Miami, USA, pp 304–311

    Google Scholar 

  • Douillard B, Fox D, Ramos F (2007) A spatio-temporal probabilistic model for multi-sensor object recognition. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, San Diego, CA, USA, pp 2402–2408

    Google Scholar 

  • Douillard B, Fox D, Ramos F (2008) Laser and vision based outdoor object mapping. In: Proceedings of robotics: science and systems (RSS), Zurich

    Google Scholar 

  • Enzweiler M, Gavrila D (2009) Monocular pedestrian detection: survey and experiments. IEEE Trans Pattern Anal Mach Intell 31(12):2179–2195

    Article  Google Scholar 

  • Ess A, Leibe B, Schindler K, Van Gool L (2009) Moving obstacle detection in highly dynamic scenes. In: Proceedings of IEEE international conference on Robotics and Automation (ICRA), Kobe, Japan

    Google Scholar 

  • Felzenszwalb P, McAllester D, Ramanan D (2008) A discriminatively trained, multiscale, deformable part model. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), Anchorage, Alaska, USA, pp 1–8

    Google Scholar 

  • Fod A, Howard A, Mataric M (2002) Laser-based people tracking. In: Proceedings of IEEE international conference on robotics and automation (ICRA), vol 3, Washington, DC, pp 3024–3029

    Google Scholar 

  • Fortmann T, Bar-Shalom Y, Scheffe M (1983) Sonar tracking of multiple targets using joint probabilistic data association. IEEE J Ocean Eng 8(3):173–184

    Article  Google Scholar 

  • Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139

    Article  MathSciNet  MATH  Google Scholar 

  • Gavrila D (2000) Pedestrian detection from a moving vehicle. In: European conference on computer vision (ECCV), Dublin, pp 37–49. IEEE Computer Society

    Google Scholar 

  • Gordon N (1993) Bayesian methods for tracking. PhD thesis, University of London

    Google Scholar 

  • Hastings WK (1970) Monte Carlo sampling methods using markov chains and their applications. Biometrika 57(1):97–109

    Article  MATH  Google Scholar 

  • Kleinhagenbrock M, Lang S, Fritsch J, Lömker F, Fink G, Sagerer G (2002) Person tracking with a mobile robot based on multi-modal anchoring. In: IEEE international workshop on robot and human interactive communication (ROMAN), Berlin, Germany

    Google Scholar 

  • Kuehnle A (1991) Symmetry-based recognition of vehicle rears. Pattern Recognit Lett 12:249–258

    Article  Google Scholar 

  • Kuncheva L (2004) Combining pattern classifiers: methods and algorithms. Wiley-Interscience, Hoboken

    Book  MATH  Google Scholar 

  • Labayrade R, Aubert D, Tarel J (2002) Real time obstacles detection on non flat road geometry through v-disparity representation. In: Proceedings of IEEE Intelligent Vehicle Symposium (IV), Versailles, France

    Google Scholar 

  • Lafferty J, McCallum A, Pereira F (2001) Conditional random fields: probabilistic models for segmentation and labeling sequence data. In: International conference on machine learning (ICML), Williamstown, pp 282–289

    Google Scholar 

  • Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2279–2324

    Article  Google Scholar 

  • Leibe B, Seemann E, Schiele B (2005) Pedestrian detection in crowded scenes. In: Proceedings of IEEE international conference on computer vision and pattern recognition (CVPR), vol 1, San Diego, pp 878–885

    Google Scholar 

  • Leibe B, Cornelis N, Cornelis K, Van Gool L (2007) Dynamic 3D scene analysis from a moving vehicle. In: Proceedings of IEEE international conference on computer vision and pattern recognition (CVPR), vol 1, Minneapolis, pp 1–8

    Google Scholar 

  • Leonard J, How J, Teller S, Berger M, Campbell S, Fiore G, Fletcher L, Frazzoli E, Huang A, Kara-man S et al (2008) A perception-driven autonomous urban vehicle. J Field Robot 25(10):727–774

    Article  Google Scholar 

  • Lundquist C, Schon T (2008) Road geometry estimation and vehicle tracking using a single track model. In: Intelligent vehicles symposium, 2008 IEEE, pp 144–149. IEEE, Eindhoven, The Netherlands

    Google Scholar 

  • MacKay DJC (1998) Introduction to Monte Carlo methods. In: Jordan MI (ed) Learning in graphical models, NATO science series. Kluwer Academic, Dordrecht, pp 175–204

    Google Scholar 

  • Mahlisch RS, Ritter W, Dietmayer K (2006) Sen-sorfusion using spatio-temporal aligned video and lidar for improved vehicle detection. In: Proceedings of IEEE international symposium on intelligent vehicles, Tokyo, Japan, pp 424–429

    Google Scholar 

  • Matthies L, Elfes A (1988) Integration of sonar and stereo range data using a grid-based representation. In: Proceedings of IEEE international conference on robotics and automation, Philadelphia

    Google Scholar 

  • Mekhnacha K, Mao Y, Raulo D, Laugier C (2008) Bayesian occupancy filter based Fast Clustering-Tracking algorithm. In: Proceedings of IEEE/RSJ international conference on intelligent robot and systems, Nice

    Google Scholar 

  • Metropolis N, Rosenbluth A, Rosenbluth M, Teller A, Teller E et al (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6):1087

    Article  Google Scholar 

  • Mikolajczyk K, Schmid C, Zisserman A (2004) Human detection based on a probabilistic assembly of robust part detectors. In: European conference on computer vision (ECCV), Prague, Czech Republic, pp 69–82

    Google Scholar 

  • Mohan A, Papageorgiou C, Poggio T (2001) Example-based object detection in images by components. IEEE Trans Pattern Anal Mach Intell 23(4):349–361

    Article  Google Scholar 

  • Montemerlo M (2003) FastSLAM: a factored solution to the simultaneous localization and mapping problem with unknown data association. PhD thesis, Robotics Institute, Carnegie Mellon University

    Google Scholar 

  • Montemerlo M, Becker J, Bhat S, Dahlkamp H, Dolgov D, Ettinger S, Hähnel D, Hilden T, Hoffmann G, Huhnke B, Johnston D, Klumpp S, Langer D, Levandowski A, Levinson J, Marcil J, Orenstein D, Paefgen J, Penny I, Petrovskaya A, Pflueger M, Stanek G, Stavens D, Vogt A, Thrun S (2008) Junior: the stanford entry in the urban challenge. J Field Robot 25(9):569–597

    Article  Google Scholar 

  • Moravec H (1988) Sensor fusion in certainty grids for mobile robots. AI Mag 9(2)

    Google Scholar 

  • Munder S, Gavrila D (2006) An experimental study on pedestrian classification. IEEE Trans Pattern Anal Mach Intell 28:1863–1868

    Article  Google Scholar 

  • Murphy K, Russell S (2001) Rao-blackwellized particle filtering for dynamic Bayesian networks. Springer, Heidelberg

    Google Scholar 

  • Murray D, Little J (2000) Using real-time stereo vision for mobile robot navigation. Auton Robot 8(2):161–171

    Article  Google Scholar 

  • Oliveira L, Nunes U (2010) Context-aware pedestrian detection using lidar. In: Proceedings of IEEE international symposium on intelligent vehicles, San Diego, CA, USA

    Google Scholar 

  • Oliveira L, Nunes U, Peixoto P (2010a) On exploration of classifier ensemble synergism in pedestrian detection. IEEE Trans Intell Transp Syst 11:16–27

    Article  Google Scholar 

  • Oliveira L, Nunes U, Peixoto P, Silva M, Moita F (2010b) Semantic fusion of laser and vision in pedestrian detection. Pattern Recognit 43:3648–3659

    Article  MATH  Google Scholar 

  • Papageorgiou C, Poggio T (2000) A trainable system for object detection. Int J Comput Vis 38(1):15–33

    Article  MATH  Google Scholar 

  • Perrollaz M, Spalanzani A, Aubert D (2010a) Probabilistic representation of the uncertainty of stereo-vision and application to obstacle detection. In: Proceedings of IEEE international symposium on intelligent vehicles, San Diego, CA, USA, pp 313–318

    Google Scholar 

  • Perrollaz M, Yoder J-D, Laugier C (2010b) Using obstacle and road pixels in the disparity space computation of stereovision based occupancy grids. In: Proceedings of IEEE international conference on intelligent transportation systems, Madeira, Portugal

    Google Scholar 

  • Petrovskaya A (2011) Towards dependable robotic perception. Ph D thesis, Stanford University, Stanford

    Google Scholar 

  • Petrovskaya A, Khatib O (2011) Global localization of objects via touch. IEEE Trans Robot 27(3):569–585

    Article  Google Scholar 

  • Petrovskaya A, Thrun S (2009) Model based vehicle detection and tracking for autonomous urban driving. Auton Robot 26(2):123–139

    Article  Google Scholar 

  • Premebida C, Monteiro G, Nunes U, Peixoto P (2007) A lidar and vision-based approach for pedestrian and vehicle detection and tracking. In: Proceedings of IEEE international conference on intelligent transportation systems, Seattle, USA, pp 1044–1049

    Google Scholar 

  • Prisacariu VA, Reid I (2009) fasthog- a realtime gpu implementation of hog technical report no. 2310/09

    Google Scholar 

  • Rao C (1945) Information and accuracy obtainable in one estimation of a statistical parameter. Bull Calcutta Math Soc 37:81–91

    MathSciNet  MATH  Google Scholar 

  • Reid DB (1979) An algorithm for tracking multiple targets. IEEE Trans Autom Control 24:843–854

    Article  Google Scholar 

  • Reisman P, Mano O, Avidan S, Shashua A (2004) Crowd detection in video sequences. In: Proceedings of IEEE international symposium on intelligent vehicles, Parma, Italy, pp 66–71. IEEE

    Google Scholar 

  • Richardson M, Domingos P (2006) Markov logic networks. Mach Learn 62:107–136

    Article  Google Scholar 

  • Richter E, Schubert R, Wanielik G (2008) Radar and vision based data fusion-advanced filtering techniques for a multi object vehicle tracking system. In: Proceedings of intelligent vehicles symposium, 2008 IEEE, Eindhoven, The Netherlands pp 120–125. IEEE

    Google Scholar 

  • Scheunert U, Mattern N, Lindner P, Wanielik G (2008) Generalized grid framework for multi sensor data fusion. J Inf Fusion, 814–820

    Google Scholar 

  • Schulz D, Burgard W, Fox D, Cremers AB (2003) People tracking with mobile robots using sample-based joint probabilistic data association filters. Int J Robot Res (IJRR) 22(2):99–116

    Article  Google Scholar 

  • Sittler R (1964) An optimal data association problem in surveillance theory. IEEE Trans Militar Electron 8(2):125–139

    Article  Google Scholar 

  • Spinello L, Macho A, Triebel R, Siegwart R (2009a) Detecting pedestrians at very small scales. In: Proceedings of IEEE international conference on intelligent robots and systems (IROS), St. Louis, pp 4313–4318

    Google Scholar 

  • Spinello L, Triebel R, Siegwart R (2009b) A trained system for multimodal perception in urban environments. In proceedings of the workshop on people detection and tracking of IEEE ICRA 2009, Kobe, Japan

    Google Scholar 

  • Spinello L, Triebel R, Siegwart R (2010) Multi-class multimodal detection and tracking in urban environments. Int J Robot Res (IJRR) 29(12):1498–1515

    Article  Google Scholar 

  • Sun Z, Bebis G, Miller R (2006) On-road vehicle detection: a review. IEEE Trans Pattern Anal Mach Intell 28:694–711

    Article  Google Scholar 

  • Szarvas M, Sakai U, Ogata J (2006) Real-time pedestrian detection using lidar and convolutional neural networks. In: Proceedings of IEEE international symposium on intelligent vehicles, Tokyo, Japan, pp 213–218

    Google Scholar 

  • Thrun S, Burgard W, Fox D (2005) Probabilistic robotics. MIT Press, Cambridge, MA

    MATH  Google Scholar 

  • Tuzel O, Porikli F, Meer P (2007) Human detection via classification on riemannian manifolds. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), Rio de Janeiro, Brazil, pp 1–8

    Google Scholar 

  • Vermaak J, Godsill S, Perez P (2005) Monte Carlo filtering for multi target tracking and data association. IEEE Trans Aerosp Electron Syst 41(1):309–332

    Article  Google Scholar 

  • Viola P, Jones MJ, Snow D (2003) Detecting pedestrians using patterns of motion and appearance. In: Proceedings of IEEE international conference on computer vision (ICCV), vol 2, Nice, pp 734–741

    Google Scholar 

  • Vu T (2009) Vehicle perception: localization, mapping with detection, classification and tracking of moving objects. PhD thesis, Institut National Polytechnique De Grenoble

    Google Scholar 

  • Vu T, Aycard O (2009) Laser-based detection and tracking moving objects using data-driven markov chain Monte Carlo. In: Proceedings of IEEE international conference on robotics and automation (ICRA 2009), Kobe, Japan, pp 3800–3806. IEEE

    Google Scholar 

  • Wang X, Han TX, Yan S (2009) An HOG-LBP human detector with partial occlusion handling. In: Proceedings of 2009 IEEE 12th international conference on computer vision, Kyoto, Japan, pp 32–39

    Google Scholar 

  • Wu B, Nevatia R (2005) Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors. In: Proceedings of IEEE international conference on computer vision (ICCV), Beijing, China

    Google Scholar 

  • Xavier J, Pacheco M, Castro D, Ruano A, Nunes U (2005) Fast line, arc/circle and leg detection from laser scan data in a player driver. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Barcelona, pp 3930–3935

    Google Scholar 

  • Zhang Q, Pless R (2004) Extrinsic calibration of a camera and laser range finder (improves camera calibration). In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, Sendai, Japan, vol 3, pp 2301–2306

    Google Scholar 

  • Zhu Q, Yeh MC, Cheng KT, Avidan S (2006) Fast human detection using a cascade of histograms of oriented gradients. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), vol 2, New York, pp 1491–1498

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Artificial Intelligence Laboratory, Stanford University, Stanford, CA, 94305-9010, USA

    Prof. Anna Petrovskaya

  2. e-Motion Project-Team, INRIA Grenoble Rhône-Alpes, 655 Avenue de l’Europe, Saint Ismier Cedex, 38334, France

    Dr. Mathias Perrollaz & Dr. Christian Laugier

  3. Faculty of Science and Technology, Coimbra University, Pólo II, Rua Silvio Lima, Coimbra, 3030-790, Portugal

    Dr. Luciano Oliveira & Dr. Urbano Nunes

  4. University of Freiburg, Freiburg, Germany

    Dr. Luciano Spinello

  5. University of Oxford, Parks Road, Oxford, OX1 3PJ, UK

    Dr. Rudolph Triebel

  6. Mechanical Engineering Department, Ohio Northern University, Science Annex 106 A, 525 S. Main Street, Ada, OH, 45810, USA

    Dr. John-David Yoder

  7. INRIA and Collège de France, Saint Ismier Cedex, France

    Dr. Pierre Bessiere

  8. Inst.f. Robotik u. Intelligente Systeme, ETH Zurich, Georges-Köhler-Allee 74, Zurich, Switzerland

    Dr. Luciano Spinello & Dr. Rudolph Triebel

  9. INRIA, Saint Ismier Cedex, France

    Dr. Alexandros Makris

Authors
  1. Prof. Anna Petrovskaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dr. Mathias Perrollaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dr. Luciano Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dr. Luciano Spinello
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dr. Rudolph Triebel
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dr. Alexandros Makris
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dr. John-David Yoder
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Dr. Christian Laugier
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Dr. Urbano Nunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Dr. Pierre Bessiere
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna Petrovskaya .

Editor information

Editors and Affiliations

  1. Center for Intelligent Systems Research, The George Washington University, 801 22nd Street, NW, Phillips Hall 630, Washington, DC, 20052, USA

    Azim Eskandarian

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag London Ltd.

About this entry

Cite this entry

Petrovskaya, A. et al. (2012). Awareness of Road Scene Participants for Autonomous Driving. In: Eskandarian, A. (eds) Handbook of Intelligent Vehicles. Springer, London. https://doi.org/10.1007/978-0-85729-085-4_54

Download citation

Publish with us

Policies and ethics

Search

Navigation