3D object recognition method with multiple feature extraction from LiDAR point clouds

  • Yifei Tian
  • Wei SongEmail author
  • Su Sun
  • Simon Fong
  • Shuanghui Zou


During autonomous driving, fast and accurate object recognition supports environment perception for local path planning of unmanned ground vehicles. Feature extraction and object recognition from large-scale 3D point clouds incur massive computational and time costs. To implement fast environment perception, this paper proposes a 3D recognition system with multiple feature extraction from light detection and ranging point clouds modified by parallel computing. Effective object feature extraction is a necessary step prior to executing an object recognition procedure. In the proposed system, multiple geometry features of a point cloud that resides in corresponding voxels are computed concurrently. In addition, a scale filter is employed to convert feature vectors from uncertain count voxels to a normalized object feature matrix, which is convenient for object-recognizing classifiers. After generating the object feature matrices of all voxels, an initialized multilayer neural network (NN) model is trained offline through a large number of iterations. Using the trained NN model, real-time object recognition is realized using parallel computing technology to accelerate computation.


3D object recognition Feature extraction LiDAR point cloud Parallel computing 



This research was supported by National Natural Science Foundation of China (61503005), Beijing Natural Science Foundation (4184086), Beijing Young Topnotch Talents Cultivation Program (No. CIT&TCD201904009), the Great Wall Scholar Program (CIT&TCD20190304), NCUT “The Belt and Road” Talent Training Base Project, and NCUT “Yuyou” Project.


  1. 1.
    Yao J, Zhang K, Yang Y et al (2018) Emergency vehicle route oriented signal coordinated control model with two-level programming. Soft Comput 2(13):4283–4294CrossRefGoogle Scholar
  2. 2.
    Simony M, Milzy S, Amende K et al (2018) Complex-YOLO: an Euler-region-proposal for real-time 3D object detection on point clouds. In: Computer Vision-ECCV 2018 Workshops, pp 197–209Google Scholar
  3. 3.
    Chen X, Ma H, Wan J et al (2017) Multi-view 3D object detection network for autonomous driving. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1907–1915Google Scholar
  4. 4.
    Guo Y, Bennamoun M, Sohel F et al (2014) 3D object recognition in cluttered scenes with local surface features: a survey. IEEE Trans Pattern Anal Mach Intell 36(11):2270–2287CrossRefGoogle Scholar
  5. 5.
    Stamatis OK, Aouf N, Gray G et al (2018) Local feature based automatic target recognition for future 3D active homing seeker missiles. Aerosp Sci Technol 73:309–317CrossRefGoogle Scholar
  6. 6.
    Zeng H, Wang H, Dong J (2017) Robust 3D keypoint detection method based on double Gaussian weighted dissimilarity measure. Multimed Tools Appl 76(24):26377–26389CrossRefGoogle Scholar
  7. 7.
    Wang J, Lindenbergh R, Menenti M (2017) SigVox—a 3D feature matching algorithm for automatic street object recognition in mobile laser scanning point clouds. ISPRS J Photogramm Remote Sens 128:111–129CrossRefGoogle Scholar
  8. 8.
    Zeng H, Liu Y, Liu J et al (2018) Non-rigid 3D model retrieval based on quadruplet convolutional neural networks. IEEE Access 6:76087–76097CrossRefGoogle Scholar
  9. 9.
    Watanabe T, Yamazaki K, Yokokohji Y (2017) Survey of robotic manipulation studies intending practical applications in real environments-object recognition, soft robot hand, and challenge program and benchmarking. Adv Robot 31(19–20):1114–1132CrossRefGoogle Scholar
  10. 10.
    Li L, Ota K, Dong M (2018) Humanlike driving: empirical decision-making system for autonomous vehicles. IEEE Trans Veh Technol 67(8):6814–6823CrossRefGoogle Scholar
  11. 11.
    Buenoa M, González-Jorgea H, Sánchez JM et al (2017) Automatic point cloud coarse registration using geometric keypoint descriptors for indoor scenes. Autom Constr 81:134–148CrossRefGoogle Scholar
  12. 12.
    Sun J, Zhang J, Zhang G (2016) An automatic 3D point cloud registration method based on regional curvature maps. Image Vis Comput 56:49–58CrossRefGoogle Scholar
  13. 13.
    Persad RA, Armenakis C (2017) Automatic co-registration of 3D multi-sensor point clouds. ISPRS J Photogramm Remote Sens 130:162–186CrossRefGoogle Scholar
  14. 14.
    Ge X (2017) Automatic markerless registration of point clouds with semantic-keypoint-based 4-points congruent sets. ISPRS J Photogramm Remote Sens 130:344–357CrossRefGoogle Scholar
  15. 15.
    Hansch R, Webera T, Hellwich O (2014) Comparison of 3D interest point detectors and descriptors for point cloud fusion. ISPRS Ann Photogramm Remote Sens Spat Inf Sci II-3:57–64CrossRefGoogle Scholar
  16. 16.
    Weber T, Hänsch R, Hellwich O (2015) Automatic registration of unordered point clouds acquired by Kinect sensors using an overlap heuristic. ISPRS J Photogramm Remote Sens 102:96–109CrossRefGoogle Scholar
  17. 17.
    Yang J, Zhang Q, Cao Z (2017) The effect of spatial information characterization on 3D local feature descriptors: a quantitative evaluation. Pattern Recogn 66:375–391CrossRefGoogle Scholar
  18. 18.
    Yang J, Cao Z, Zhang Q (2016) A fast and robust local descriptor for 3D point cloud registration. Inf Sci 346–347:163–179CrossRefGoogle Scholar
  19. 19.
    Garcia AG, Escolano SO, Rodriguez JG et al (2018) Interactive 3D object recognition pipeline on mobile GPGPU computing platforms using low-cost RGB-D sensors. J Real-Time Image Proc 14:585–604CrossRefGoogle Scholar
  20. 20.
    Quan S, Ma J, Hu F et al (2018) Local voxelized structure for 3D binary feature representation and robust registration of point clouds from low-cost sensors. Inf Sci 444:153–171CrossRefGoogle Scholar
  21. 21.
    Zhu Q, Li Y, Hu H et al (2017) Robust point cloud classification based on multi-level semantic relationships for urban scenes. ISPRS J Photogramm Remote Sens 129:86–102CrossRefGoogle Scholar
  22. 22.
    Xu Y, Tuttas S, Hoegner L et al (2018) Voxel-based segmentation of 3D point clouds from construction sites using a probabilistic connectivity model. Pattern Recogn Lett 102:67–74CrossRefGoogle Scholar
  23. 23.
    Yang B, Dong Z, Zhao G et al (2015) Hierarchical extraction of urban objects from mobile laser scanning data. ISPRS J Photogramm Remote Sens 99:45–57CrossRefGoogle Scholar
  24. 24.
    Lei H, Jiang G, Quan L (2017) Fast descriptors and correspondence propagation for robust global point cloud registration. IEEE Trans Image Process 26(8):3614–3623MathSciNetzbMATHGoogle Scholar
  25. 25.
    Elbaz G, Avraham T, Fischer A (2017) 3D point cloud registration for localization using a deep neural network auto-encoder. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 4631–4640Google Scholar
  26. 26.
    Ligon J, Bein D, Ly P et al (2018) 3D point cloud processing using spin images for object detection. In: 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), pp 731–736Google Scholar
  27. 27.
    Yang J, Zhang Q, Xian K et al (2017) Rotational contour signatures for both real-valued and binary feature representations of 3D local shape. Comput Vis Image Underst 160:133–147CrossRefGoogle Scholar
  28. 28.
    Dong Z, Yang B, Liu Y et al (2017) A novel binary shape context for 3D local surface description. ISPRS J Photogramm Remote Sens 130:431–452CrossRefGoogle Scholar
  29. 29.
    Tu Y, Lin Y, Wang J et al (2018) Semi-supervised learning with generative adversarial networks on digital signal modulation classification. Comput Mater Contin 55(2):243–254Google Scholar
  30. 30.
    Zeng D, Dai Y, Li F et al (2018) Adversarial learning for distant supervised relation extraction. Comput Mater Contin 55(1):121–136Google Scholar
  31. 31.
    Dubé R, Gollub MG, Sommer H et al (2018) Incremental-segment-based localization in 3-D point clouds. IEEE Robot Autom Lett 3(3):1832–1839CrossRefGoogle Scholar
  32. 32.
    Soilán M, Riveiro B, Sánchez JM et al (2017) Segmentation and classification of road markings using MLS data. ISPRS J Photogramm Remote Sens 123:94–103CrossRefGoogle Scholar
  33. 33.
    Roveri R, Rahmann L, Oztireli AC et al (2018) A network architecture for point cloud classification via automatic depth images generation. In: IEEE Conference on Computer Vision Pattern Recognition (CVPR), pp 4176–4184Google Scholar
  34. 34.
    Bobkov D, Chen S, Jian R et al (2018) Noise-resistant deep learning for object classification in three-dimensional point clouds using a point pair descriptor. IEEE Robot Autom Lett 3(2):865–872CrossRefGoogle Scholar
  35. 35.
    Chen J, Cho YK, Ueda J (2018) Sampled-point network for classification of deformed building element point clouds. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp 2164–2169Google Scholar
  36. 36.
    Song W, Tian Y, Fong S et al (2016) GPU-accelerated foreground segmentation and labeling for real-time video surveillance. Sustainability 8(10):916CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.North China University of TechnologyBeijingChina
  2. 2.Department of Computer and Information ScienceUniversity of MacauTaipaChina
  3. 3.Beijing Key Lab on Urban Intelligent Traffic Control TechnologyBeijingChina

Personalised recommendations