3D object recognition from cluttered and occluded scenes with a compact local feature

  • Wulong GuoEmail author
  • Weiduo Hu
  • Chang Liu
  • Tingting Lu
Original Paper


This paper presents a novel local surface descriptor for 3D object recognition in the presence of clutter and occlusion. For a keypoint and its local neighbor points, a unique and repeatable local reference frame is first constructed. Then the depth information of those local points is encoded into a vector using multi-view method. Finally we obtain a compact feature with length of 27. Besides its compactness in length, the generated feature is tested to be not only descriptive on various datasets but also robust against Gaussian noise, short noise and varying mesh resolutions. Based on this descriptor, we propose a hierarchical algorithm for automatic object recognition. By feature matching, scene-point-to-model-point correspondences are established after potential planar clutters are removed from the scene. These correspondences are then utilized for object identification and pose estimation. Experimental results on three popular datasets demonstrate the effectiveness of our proposed technique which achieves the best results compared to the state of the art with all targets correctly recognized and their poses estimated.


3D descriptor Local reference frame Local surface feature Object recognition Point cloud 



The authors wish to thank owners of datasets used in this work for making those datasets publicly available. This work is supported by the National Natural Science Foundation of China (Grant No. 61703017).


  1. 1.
    Aldoma, A., Tombari, F., Di Stefano, L., Vincze, M.: A global hypotheses verification method for 3d object recognition. In: Computer Vision—ECCV 2012: 12th European Conference on Computer Vision. Florence, Italy, 7–13 October 2012, Proceedings, Part III, pp. 511–524. Springer, Berlin (2012)Google Scholar
  2. 2.
    Bariya, P., Novatnack, J., Schwartz, G., Nishino, K.: 3D geometric scale variability in range images: features and descriptors. Int. J. Comput. Vis. 99(2), 232–255 (2012)MathSciNetGoogle Scholar
  3. 3.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)Google Scholar
  4. 4.
    Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975). MathSciNetzbMATHGoogle Scholar
  5. 5.
    Besl, P.J., Mckay, H.D.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)Google Scholar
  6. 6.
    Buch, A.G., Petersen, H.G., Krüger, N.: Local shape feature fusion for improved matching, pose estimation and 3d object recognition. SpringerPlus 5(1), 1–33 (2016)Google Scholar
  7. 7.
    Chen, H.: 3D free-form object recognition in range images using local surface patches. Pattern Recognit. Lett. 3, 136 (2007)Google Scholar
  8. 8.
    Chen, X., Li, J., Shi, Z., Yang, W.: Distinctive local surface descriptor for three-dimensional objects based on bispectrum of spherical harmonics. J. Electron. Imaging 25(1), 013,021–013,021 (2016)Google Scholar
  9. 9.
    Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: International Conference on Computer Graphics and Interactive Techniques, pp. 303–312 (1996)Google Scholar
  10. 10.
    Flint, A., Dick, A., van den Hengel, A.: Thrift: local 3d structure recognition. In: 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007), pp. 182–188 (2007)Google Scholar
  11. 11.
    Frome, A., Huber, D., Kolluri, R., Bülow, T., Malik, J.: Recognizing objects in range data using regional point descriptors. In: Computer Vision—ECCV 2004: 8th European Conference on Computer Vision, Prague, Czech Republic, 11–14 May 2004. Proceedings, Part III, pp. 224–237. Springer, Berlin (2004)Google Scholar
  12. 12.
    Gao, Y., Zhang, N.: 3D model comparison using spatial structure circular descriptor. Pattern Recognit. 43(3), 1142–1151 (2010)zbMATHGoogle Scholar
  13. 13.
    Guennebaud, G., Germann, M., Gross, M.: Dynamic sampling and rendering of algebraic point set surfaces. Comput. Graph. Forum 27(2), 653–662 (2010)Google Scholar
  14. 14.
    Guennebaud, G., Gross, M.: Algebraic point set surfaces. ACM Trans. Graph. 26(3), 1–9 (2007)Google Scholar
  15. 15.
    Guo, Y., Bennamoun, M., Sohel, F., Lu, M., Wan, J.: 3D object recognition in cluttered scenes with local surface features: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 36(11), 2270–2287 (2014)Google Scholar
  16. 16.
    Guo, Y., Bennamoun, M., Sohel, F., Lu, M., Wan, J.: An integrated framework for 3-d modeling, object detection, and pose estimation from point-clouds. IEEE Trans. Instrum. Meas. 64(3), 683–693 (2015)Google Scholar
  17. 17.
    Guo, Y., Bennamoun, M., Sohel, F., Wan, J., Kwok, N.M.: A comprehensive performance evaluation of 3d local feature descriptors. Int. J. Comput. Vis. 116(1), 66–89 (2015)MathSciNetGoogle Scholar
  18. 18.
    Guo, Y., Sohel, F., Bennamoun, M., Lu, M., Wan, J.: Rotational projection statistics for 3d local surface description and object recognition. Int. J. Comput. Vis. 105(1), 63–86 (2013)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Guo, Y., Sohel, F., Bennamoun, M., Wan, J.: A novel local surface feature for 3d object recognition under clutter and occlusion. Inf. Sci. 293, 196–213 (2015)Google Scholar
  20. 20.
    Huang, C.H., Tombari, F., Navab, N.: Repeatable local coordinate frames for 3d human motion tracking: from rigid to non-rigid. In: 2015 International Conference on 3D Vision, pp. 371–379 (2015)Google Scholar
  21. 21.
    Iyer, N., Jayanti, S., Lou, K., Kalyanaraman, Y., Ramani, K.: Three-dimensional shape searching: state-of-the-art review and future trends. Comput. Aided Des. 37(5), 509–530 (2005)Google Scholar
  22. 22.
    Johnson, A.E., Hebert, M.: Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21(5), 433–449 (1999)Google Scholar
  23. 23.
    Kasaei, S.H., Tomé, A.M., Lopes, L.S., Oliveira, M.: Good: a global orthographic object descriptor for 3d object recognition and manipulation. Pattern Recognit. Lett. 83, 312–320 (2016)Google Scholar
  24. 24.
    Lei, Y., Guo, Y., Hayat, M., Bennamoun, M., Zhou, X.: A two-phase weighted collaborative representation for 3d partial face recognition with single sample. Pattern Recognit. 52, 218–237 (2016)Google Scholar
  25. 25.
    Liu, C., Hu, W.: Relative pose estimation for cylinder-shaped spacecrafts using single image. IEEE Trans. Aerosp. Electron. Syst. 50(4), 3036–3056 (2014)Google Scholar
  26. 26.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)Google Scholar
  27. 27.
    Lu, T., Hu, W., Liu, C., Yang, D.: Relative pose estimation of a lander using crater detection and matching. Optic. Eng. 55(2), 23102–23102 (2016)Google Scholar
  28. 28.
    Malassiotis, S., Strintzis, M.G.: Snapshots: a novel local surface descriptor and matching algorithm for robust 3d surface alignment. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1285–1290 (2007)Google Scholar
  29. 29.
    Mian, A., Bennamoun, M.: Keypoint detection and local feature matching for textured 3d face recognition. Int. J. Comput. Vis. 79(1), 1–12 (2008)Google Scholar
  30. 30.
    Mian, A., Bennamoun, M., Owens, R.: On the repeatability and quality of keypoints for local feature-based 3d object retrieval from cluttered scenes. Int. J. Comput. Vis. 89(2), 348–361 (2010)Google Scholar
  31. 31.
    Mian, A.S., Bennamoun, M., Owens, R.: Three-dimensional model-based object recognition and segmentation in cluttered scenes. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1584–1601 (2006)Google Scholar
  32. 32.
    Mian, A.S., Bennamoun, M., Owens, R.A.: A novel representation and feature matching algorithm for automatic pairwise registration of range images. Int. J. Comput. Vis. 66(1), 19–40 (2006)Google Scholar
  33. 33.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)Google Scholar
  34. 34.
    Pang, G., Neumann, U.: Fast and robust multi-view 3d object recognition in point clouds. In: 2015 International Conference on 3D Vision (3DV), pp. 171–179 (2015)Google Scholar
  35. 35.
    Petrelli, A., Stefano, L.D.: On the repeatability of the local reference frame for partial shape matching. In: International Conference on Computer Vision, pp. 2244–2251 (2011)Google Scholar
  36. 36.
    Petrelli, A., Stefano, L.D.: A repeatable and efficient canonical reference for surface matching. In: 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization Transmission, pp. 403–410 (2012)Google Scholar
  37. 37.
    Quan, S., Ma, J., Ma, T., Hu, F., Fang, B.: Representing local shape geometry from multi-view silhouette perspective: a distinctive and robust binary 3d feature. Signal Process. Image Commun. 65, 67–80 (2018). Google Scholar
  38. 38.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2016)Google Scholar
  39. 39.
    Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (fpfh) for 3d registration. In: International Conference on Robotics and Automation, pp. 3212–3217 (2009)Google Scholar
  40. 40.
    Rusu, R.B., Blodow, N., Marton, Z.C., Beetz, M.: Aligning point cloud views using persistent feature histograms. In: Intelligent RObots and Systems, pp. 3384–3391 (2008)Google Scholar
  41. 41.
    Salti, S., Tombari, F., Di Stefano, L.: Shot: unique signatures of histograms for surface and texture description. Comput. Vis. Image Underst. 125, 251–264 (2014)Google Scholar
  42. 42.
    Shah, S.A.A., Bennamoun, M., Boussaid, F.: A novel 3d vorticity based approach for automatic registration of low resolution range images. Pattern Recognit. 48(9), 2859–2871 (2015)Google Scholar
  43. 43.
    Shah, S.A.A., Bennamoun, M., Boussaid, F.: A novel feature representation for automatic 3d object recognition in cluttered scenes. Neurocomputing 205, 1–15 (2015)Google Scholar
  44. 44.
    Shang, L., Greenspan, M.: Real-time object recognition in sparse range images using error surface embedding. Int. J. Comput. Vis. 89(2), 211–228 (2010)Google Scholar
  45. 45.
    Sun, T., Liu, S., Liu, G., Zhu, S., Zhu, Z.: A 3d descriptor based on local height image. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5 (2018).
  46. 46.
    Taati, B., Greenspan, M.: Local shape descriptor selection for object recognition in range data. Comput. Vis. Image Underst. 115(5), 681–694 (2011). Special issue on 3D Imaging and ModellingGoogle Scholar
  47. 47.
    Tombari, F., Salti, S., Di Stefano, L.: Unique shape context for 3d data description. In: Proceedings of the ACM Workshop on 3D Object Retrieval, 3DOR ’10, pp. 57–62. ACM, New York, NY, USA (2010)Google Scholar
  48. 48.
    Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: Computer Vision - ECCV 2010: 11th European Conference on Computer Vision. Heraklion, Crete, Greece, 5–11 September 2010, Proceedings, Part III, pp. 356–369. Springer, Berlin (2010)Google Scholar
  49. 49.
    Tombari, F., Salti, S., Di Stefano, L.: Performance evaluation of 3d keypoint detectors. Int. J. Comput. Vis. 102(1), 198–220 (2013)Google Scholar
  50. 50.
    Tombari, F., Salti, S., Stefano, L.D.: A combined texture-shape descriptor for enhanced 3d feature matching. In: 2011 18th IEEE International Conference on Image Processing, pp. 809–812 (2011)Google Scholar
  51. 51.
    Yang, J., Cao, Z., Zhang, Q.: A fast and robust local descriptor for 3d point cloud registration. Inf. Sci. 346–347, 163–179 (2016). Google Scholar
  52. 52.
    Yang, J., Zhang, Q., Xian, K., Xiao, Y., Cao, Z.: Rotational contour signatures for both real-valued and binary feature representations of 3d local shape. Comput. Vis. Image Underst. 160, 133–147 (2017). Google Scholar
  53. 53.
    Yang, J., Zhang, Q., Xiao, Y., Cao, Z.: Toldi: An effective and robust approach for 3d local shape description. Pattern Recognit. 65, 175–187 (2017). Google Scholar
  54. 54.
    Zaharescu, A., Boyer, E., Varanasi, K., Horaud, R.: Surface feature detection and description with applications to mesh matching. In: Computer Vision and Pattern Recognition, pp. 373–380 (2009)Google Scholar
  55. 55.
    Zhong, Y.: Intrinsic shape signatures: a shape descriptor for 3d object recognition. In: Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on, pp. 689–696 (2009)Google Scholar
  56. 56.
    Zou, Y., Wang, X., Zhang, T., Liang, B., Song, J., Liu, H.: Broph: an efficient and compact binary descriptor for 3d point clouds. Pattern Recognit. 76, 522–536 (2018). Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of AstronauticsBeihang UniversityBeijingChina
  2. 2.Shenzhen Institutes of Advanced TechnologyChinese Academy of SciencesShenzhenChina
  3. 3.Research and Development DepartmentChina Academy of Launch Vehicle TechnologyBeijingChina

Personalised recommendations