Dynamic 3D Environment Perception and Reconstruction Using a Mobile Rotating Multi-beam Lidar Scanner

  • Attila BörcsEmail author
  • Balázs Nagy
  • Csaba Benedek
Part of the Studies in Systems, Decision and Control book series (SSDC, volume 42)


In this chapter we introduce cooperating techniques for environment perception and reconstruction based on dynamic point cloud sequences of a single rotating multi-beam (RMB) Lidar sensor, which monitors the scene either from a moving vehicle top or from a static installed position. The joint aim of the addressed methods is to create 4D spatio-temporal models of large dynamic urban scenes containing various moving and static objects. Standalone RMB Lidar devices have been frequently applied in robot navigation tasks and proved to be efficient in moving object detection and recognition. However, they have not been widely exploited yet in video surveillance or dynamic virtual city modeling. We address here three different application areas of RMB Lidar measurements, starting from people activity analysis, through real time object perception for autonomous driving, until dynamic scene interpretation and visualization. First we introduce a multiple pedestrian tracking system with short term and long term person assignment steps. Second we present a model based real-time vehicle recognition approach. Third we propose techniques for geometric approximation of ground surfaces and building facades using the observed point cloud streams. This approach extracts simultaneously the reconstructed surfaces, motion information and objects from the registered dense point cloud completed with point time stamp information.


Point Cloud Iterative Close Point Vehicle Detection Building Facade Lidar Sensor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work is partially connected to the i4D project funded by the internal R&D grant of MTA SZTAKI, and it was partially supported by the Government of Hungary through a European Space Agency (ESA) Contract under the Plan for European Cooperating States (PECS), and by the Hungarian Research Fund (OTKA #101598).


  1. Benedek C (2014) 3D people surveillance on range data sequences of a rotating LIDAR. Pattern Recogni Lett 50:149–158 Special Issue on Depth Image AnalysisCrossRefGoogle Scholar
  2. Börcs A, Nagy B, Baticz M, Benedek C (2014a) A model-based approach for fast vehicle detection in continuously streamed urban LIDAR point clouds. In: Workshop on scene understanding for autonomous systems at ACCV, Lecture Notes in Computer Science, SingaporeGoogle Scholar
  3. Börcs A, Nagy B, Benedek C (2014b) Fast 3-D urban object detection on streaming point clouds. In: Workshop on computer vision for road scene understanding and autonomous driving at ECCV, Lecture Notes in Computer Science, Zürich, SwitzerlandGoogle Scholar
  4. Douillard B, Underwood J, Kuntz N, Vlaskine V, Quadros A, Morton P, Frenkel A (2011) On the segmentation of 3D LIDAR point clouds. In: Proceedings of the IEEE international conference on robotics and automation (ICRA). Shanghai, China, pp 2798–2805Google Scholar
  5. Douillard B, Quadros A, Morton P, Underwood J, de Deuge M, Hugosson S, Hallstrom M, Bailey T (2012) Scan segments matching for pairwise 3D alignment. In: IEEE international conference on robotics and automation (ICRA), St. Paul, MN, USA, pp 3033–3040. doi: 10.1109/ICRA.2012.6224788
  6. Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? the KITTI vision benchmark suite. Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR). Providence Rhode Island Convention Center Providence, RI, USA, pp 3354–3361Google Scholar
  7. Himmelsbach M, Müller A, Luettel T, Wuensche HJ (2008) LIDAR-based 3D object perception. In: Proceedings of 1st international workshop on cognition for technical systems, Munich, pp 1–7Google Scholar
  8. Józsa O, Börcs A, Benedek C (2013) Towards 4D virtual city reconstruction from LIDAR point cloud sequences. In: ISPRS workshop on 3D virtual city modeling, ISPRS annals of photogrammetry, remote sensing and spatial information sciences, vol II-3/W1, Regina, Canada, pp 15–20Google Scholar
  9. Kazhdan M, Bolitho M, Hoppe H (2006) Poisson surface reconstruction, In: Eurographics symposium on geometry processing, Eurographics Association, Aire-la-VilleGoogle Scholar
  10. Kovács L, Kovács A, Utasi A, Szirányi T (2012) Flying target detection and recognition by feature fusion. SPIE Opt Eng 51(11):117002CrossRefGoogle Scholar
  11. Kuhn HW (1955) The Hungarian method for the assignment problem. Nav Res Logist Q 2:83–97CrossRefGoogle Scholar
  12. Levinson J, Montemerlo M, Thrun S (2007) Map-based precision vehicle localization in urban environments. In: Proceedings of robotics: science and systems, Atlanta, GA, USAGoogle Scholar
  13. Magnusson M (2009) The three-dimensional normal-distributions transform—an efficient representation for registration, surface analysis, and loop detection. PhD thesis, Örebro University, SwedenGoogle Scholar
  14. McNaughton M, Urmson C, Dolan J, Lee JW (2011) Motion planning for autonomous driving with a conformal spatiotemporal lattice. In: Proceedings of the international conference on robotics and automation (ICRA). Shanghai, China, pp 4889–4895Google Scholar
  15. Quadros AJ, Underwood JP, Douillard B (2012) An occlusion-aware feature for range images. In: Proceedings of the IEEE international conference on robotics and automation (ICRA). St. Paul, MN, USA, pp 4428–4435Google Scholar
  16. Rusu RB, Cousins S (2011) 3D is here: Point cloud library (PCL). In: Proceedings of the IEEE international conference on robotics and automation (ICRA), Shanghai, ChinaGoogle Scholar
  17. Schiller I, Koch R (2011) Improved video segmentation by adaptive combination of depth keying and mixture-of-Gaussians. In: Proceedings of the Scandinavian conference on image analysis, Ystad, Sweden, Lecture Notes in Computer Science, vol 6688, pp 59–68,
  18. Stauffer C, Grimson W (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22:747–757CrossRefGoogle Scholar
  19. Teichman A, Levinson J, Thrun S (2011) Towards 3D object recognition via classification of arbitrary object tracks. In: Proceedings of the IEEE international conference on robotics and automation (ICRA). Shanghai, China, pp 4034–4041Google Scholar
  20. Xiong X, Munoz D, Bagnell J, Hebert M (2011) 3-D scene analysis via sequenced predictions over points and regions. In: IEEE international conference on robotics and automation (ICRA), Shanghai, China, pp 2609–2616, doi: 10.1109/ICRA.2011.5980125
  21. Zhang D, Lu G (2002) A comparative study of Fourier descriptors for shape representation and retrieval. Asian conference on computer vision (ACCV). Melbourne, Australia, pp 646–651Google Scholar
  22. Zhang Z (1994) Iterative point matching for registration of free-form curves and surfaces. Int J Comput Vis 13(2):119–152CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Distributed Events Analysis Research Laboratory, Institute for Computer Science and Control (MTA SZTAKI)Hungarian Academy of SciencesBudapestHungary

Personalised recommendations