Multimedia Tools and Applications

, Volume 78, Issue 23, pp 33223–33246 | Cite as

RGB-D point cloud registration via infrared and color camera

  • Teng Wan
  • Shaoyi DuEmail author
  • Yiting Xu
  • Guanglin Xu
  • Zuoyong Li
  • Badong Chen
  • Yue Gao


The iterative closest point (ICP) algorithm is widely used for rigid registration for its simplicity and speed, but the registration is easy to fail when point sets lack of obvious structure variety, such as smooth surface and hemisphere. RGB-D information obtained from infrared camera and color camera could use color information to compensate the shapes, so we propose a precise new algorithm for RGB-D point cloud registration, which is an extension of ICP algorithm. First of all, we introduce the color information as a constraint condition to establish correct correspondences between point clouds. Secondly, to reduce the impact of noises and outliers, we use maximum correntropy criterion (MCC) to increase the robustness and accuracy. Thirdly, we add both color information and correntropy into our objective function model and solve it with ICP algorithm. Finally, the compared experiments on simulation and real datasets prove that our algorithm can align two smooth surfaces more accurate and robust than other point set registration algorithms.


Infrared and color camera Iterative closest point RGB-D Maximum correntropy criterion 



This work was supported by the National Natural Science Foundation of China under Grant Nos. 61627811 and 61573274, the Fundamental Research Funds for the Central Universities under Grant Nos. xjj2017005 and xjj2017036, Fujian Provincial Key Laboratory of Information Processing and Intelligent Control (Minjiang University) under Grant No. MJUKF-IPIC201802.


  1. 1.
    Arun KS, Huang TS, Blostein SD (1987) Least-squares fitting of two 3-d point sets. IEEE Trans Pattern Anal Mach Intell 9(5):698–700CrossRefGoogle Scholar
  2. 2.
    M. S. Belshaw and M. A. Greenspan (2008) A high speed iterative closest point tracker on an FPGA platform. In: Proc. IEEE International Conference on Computer Vision Workshops, pp. 1449–1456Google Scholar
  3. 3.
    Benjemaa R, Schmitt F (1997) Fast Global Registration of 3D Sampled Surfaces using a Multi-Z-Buffer Technique. Image Vis Comput 17(2):113CrossRefGoogle Scholar
  4. 4.
    Besl PJ, Mckay ND (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14:239–256CrossRefGoogle Scholar
  5. 5.
    Censi A (2008) An ICP variant using a point-to-line metric. In: Proc. IEEE International Conference on Robotics and Automation, pp. 19–25Google Scholar
  6. 6.
    Chetverikov D, Stepanov D, Krsek P (2005) Robust Euclidean alignment of 3D point sets: the trimmed iterative closest point algorithm. Image Vis Comput 23(3):299–309CrossRefGoogle Scholar
  7. 7.
    Danelljan M, Meneghetti G, Khan FS, Felsberg M (2016) A Probabilistic Framework for Color-Based Point Set Registration. In: Proc. Computer Vision and Pattern Recognition, pp. 1818–1826Google Scholar
  8. 8.
    Davison AJ, Reid ID, Molton ND, Stasse O (2007) MonoSLAM: real-time single camera SLAM. IEEE Trans Pattern Anal Mach Intell 29(6):1052–1067CrossRefGoogle Scholar
  9. 9.
    Du S, Cui W, Wu L, Zhang S, Zhang X, Xu G, Xu M (2017) Precise iterative closest point algorithm with corner point constraint for isotropic scaling registration. Multimedia Systems 9:1–8Google Scholar
  10. 10.
    Du S, Liu J, Bi B, Zhu J, Xue J (2016) New iterative closest point algorithm for isotropic scaling registration of point sets with noise. J Vis Commun Image Represent 38(CAC):207–216CrossRefGoogle Scholar
  11. 11.
    Du S, Liu J, Zhang C, Zhu J, Li K (2015) Probability iterative closest point algorithm for m -D point set registration with noise. Neurocomputing 157(CAC):187–198CrossRefGoogle Scholar
  12. 12.
    Faugeras OD, Hebert M (1986) The Representation, Recognition, and Positioning of 3-D Shapes from Range Data. Machine Intelligence & Pattern Recognition 3:13–51CrossRefGoogle Scholar
  13. 13.
    Fitzgibbon AW (2003) Robust registration of 2D and 3D point sets. Image Vis Comput 21(13):1145–1153CrossRefGoogle Scholar
  14. 14.
    Granger S, Pennec X (2002) Multi-scale EM-ICP: A Fast and Robust Approach for Surface Registration. In: Proc. Computer Vision - ECCV 2002, European Conference on Computer Vision, Copenhagen, Denmark, Proceedings, pp. 418–432CrossRefGoogle Scholar
  15. 15.
    Greenspan M, Yurick M (2003) Approximate K-D Tree Search for Efficient ICP. In: Proc. International Conference on 3-D Digital Imaging and Modeling, pp. 442–448Google Scholar
  16. 16.
    He Y, Liang B, Yang J, Li S, He J (2017) An Iterative Closest Points Algorithm for Registration of 3D Laser Scanner Point Clouds with Geometric Features. Sensors 17(8):1862CrossRefGoogle Scholar
  17. 17.
    Horn BKP (1998) Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America, Series A 5(7):1127–1135CrossRefGoogle Scholar
  18. 18.
    Izadi S, Stamminger M (2013) Real-time 3D reconstruction at scale using voxel hashing. ACM Trans Graph 32(6):169Google Scholar
  19. 19.
    Kaneko S, Kondo T, Miyamoto A (2003) Robust matching of 3D contours using iterative closest point algorithm improved by M-estimation. Pattern Recogn 36(9):2041–2047CrossRefGoogle Scholar
  20. 20.
    Kim D, Kim D (2010) A Fast ICP Algorithm for 3-D Human Body Motion Tracking. IEEE Signal Processing Letters 17(4):402–405CrossRefGoogle Scholar
  21. 21.
    Korn M, Holzkothen M, Pauli J (2015) Color supported generalized-ICP. In: Proc. International Conference on Computer Vision Theory and Applications, pp. 592–599Google Scholar
  22. 22.
    Lai K, Bo L, Ren X, Fox D (2011) A large-scale hierarchical multi-view RGB-D object dataset. In: Proc. IEEE International Conference on Robotics and Automation, pp. 1817–1824Google Scholar
  23. 23.
    Masuda T, Yokoya N (1995) A robust method for registration and segmentation of multiple range images. CVIU 61(3):106–113Google Scholar
  24. 24.
    Men H, Gebre B, Pochiraju K (2011) Color point cloud registration with 4D ICP algorithm. In: Proc. IEEE International Conference on Robotics and Automation, pp. 1511–1516Google Scholar
  25. 25.
    Myronenko A, Song X (2010) Point Set Registration: Coherent Point Drift. IEEE Transactions on Pattern Analysis & Machine Intelligence 32(12):2262–2275CrossRefGoogle Scholar
  26. 26.
    Nchter A, Kai L, Hertzberg J (2007) Cached k-d tree search for ICP algorithms. In: Proc. International Conference on 3-D Digital Imaging and Modeling, pp. 419–426Google Scholar
  27. 27.
    Ridene T, Goulette F and Ois (2009) Registration of fixed-and-mobile- based terrestrial Laser data sets with DSM. In: Proc. IEEE International Symposium on Computational Intelligence in Robotics and Automation, pp. 375–380Google Scholar
  28. 28.
    S. Rusinkiewicz and M. Levoy (2002) Efficient Variants of the ICP Algorithm. In: Proc. International Conference on 3-D Digital Imaging and Modeling, Proceedings, pp. 145–152Google Scholar
  29. 29.
    Schneiderman H, Nashman M (2002) A discriminating feature tracker for vision-based autonomous driving. IEEE Trans Rob Autom 10(6):769–775CrossRefGoogle Scholar
  30. 30.
    Silva L, Bellon ORP, Boyer KL (2005) Precision range image registration using a robust surface interpenetration measure and enhanced genetic algorithms. IEEE Transactions on Pattern Analysis & Machine Intelligence 27(5):762–776CrossRefGoogle Scholar
  31. 31.
    Sturm J, Engelhard N, Endres F, Burgard W, Cremers D (2012) A benchmark for the evaluation of RGB-D SLAM systems. In: Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 573–580Google Scholar
  32. 32.
    Wachowiak MP, Smolikova R, Zheng Y, Zurada JM, Elmaghraby AS (2004) An approach to multimodal biomedical image registration utilizing particle swarm optimization. Evolutionary Computation IEEE Transactions on 8(3):289–301CrossRefGoogle Scholar
  33. 33.
    Walker MW, Shao L, Volz RA (1991) Estimating 3-D location parameters using dual number quaternions. Cvgip Image Understanding 54(3):358–367CrossRefGoogle Scholar
  34. 34.
    Xu G, Du S, Xue J (2016) Precise 2D point set registration using iterative closest algorithm and correntropy. In: Proc. International Joint Conference on Neural Networks, pp. 4627–4631Google Scholar
  35. 35.
    Yan P, Bowyer KW (2005) A Fast Algorithm for ICP-Based 3D Shape Biometrics. In: Proc. Automatic Identification Advanced Technologies, 2005. Fourth IEEE Workshop on, pp. 213–218Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Institute of Artificial Intelligence and Robotics, School of Electronic and Information EngineeringXi’an Jiaotong UniversityXi’anPeople’s Republic of China
  2. 2.Fujian Provincial Key Laboratory of Information Processing and Intelligent ControlMinjiang UniversityFuzhouPeople’s Republic of China
  3. 3.School of SoftwareTsinghua UniversityBeijingPeople’s Republic of China

Personalised recommendations