Advertisement

Topology-aware non-rigid point set registration via global–local topology preservation

  • Song Ge
  • Guoliang FanEmail author
Original Paper
  • 54 Downloads

Abstract

We propose a new topology-aware point set registration algorithm which can cope with multi-part articulated and non-rigid deformations. Point set registration is formulated as a maximum likelihood (ML) estimation problem where two topologically complementary constraints are jointly optimized in a probabilistic framework. The first is coherent point drift that keeps the overall spatial connectivity and associativity by moving the point set collectively and coherently. The second is local linear embedding that preserves the local topological structure during registration. Hence, the new algorithm is called global–local topology preservation (GLTP). Without any pre-segmentation and correspondence initialization, GLTP is particularly useful and effective in dealing with complex shape matching with non-coherent and non-rigid local deformations at different parts of a point set. We have derived the expectation maximization algorithm for the ML optimization constrained with both regularization terms. Experimental results on a large set of 2D and 3D examples show the advantages and robustness of GLTP over existing algorithms in the presence of outliers, noise and missing data, especially in the case of articulated non-rigid transformations.

Keywords

Point set registration Non-rigid registration Articulated deformation Local linear embedding Topological constraints 

Notes

Acknowledgements

This work is supported in part by the Oklahoma Center for the Advancement of Science and Technology (OCAST) under Grants HR12-030 and HR18-069 and the National Science Foundation (NSF) under Grant NRI-1427345.

References

  1. 1.
    Ye, M., Wang, X., Yang, R., Liu, R., Pollefeys, M.: Accurate 3D pose estimation from a single depth image. In: Proceedings of IEEE International Conference on Computer Vision, pp. 731–738 (2011)Google Scholar
  2. 2.
    Weiss, A., Hirshberg, D., Black, M.J.: Home 3D body scans from noisy image and range data. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1951–1958 (2011)Google Scholar
  3. 3.
    Park, S.-Y., Baek, J., Moon, J.: Hand-held 3D scanning based on coarse and fine registration of multiple range images. Mach. Vis. Appl. 22(3), 563–579 (2011)Google Scholar
  4. 4.
    Park, S.-Y., Choi, S.-I., Kim, J., Chae, J.S.: Real-time 3D registration using GPU. Mach. Vis. Appl. 22(5), 837–850 (2011)Google Scholar
  5. 5.
    Pribanić, T., Diez, Y., Roure, F., Salvi, J.: An efficient surface registration using smartphone. Mach. Vis. Appl. 27(4), 559–576 (2016)Google Scholar
  6. 6.
    Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)Google Scholar
  7. 7.
    Zhang, Z.: Iterative point matching for registration of free-form curves and surfaces. Int. J. Comput. Vis. 13(2), 119–152 (1994)Google Scholar
  8. 8.
    Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings of International Conference on 3D Digital Imaging and Modeling (3DIM) (2001)Google Scholar
  9. 9.
    Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24, 325–376 (1992)Google Scholar
  10. 10.
    Makadia, A., Patterson, A., Daniilidis, K.: Fully automatic registration of 3S point clouds. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1297–1304 (2006)Google Scholar
  11. 11.
    Chui, H., Rangarajan, A.: A feature registration framework using mixture models. In: Proceedings of IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, pp. 190–197 (2000)Google Scholar
  12. 12.
    Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. Comput. Vis. Image Underst. 89(2–3), 114–141 (2003)zbMATHGoogle Scholar
  13. 13.
    Myronenko, A., Song, X., Carreira-Perpinan, M.A.: Non-rigid point set registration: coherent point drift (CPD). In: Proceedings of Advances in Neural Information Processing Systems, pp. 1009–1016 (2006)Google Scholar
  14. 14.
    Myronenko, A., Song, X.: Point set registration: coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2262–2275 (2010)Google Scholar
  15. 15.
    Jian, B., Vemuri, B.C.: A robust algorithm for point set registration using mixture of Gaussians. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1246–1251 (2005)Google Scholar
  16. 16.
    Breitenreicher, D., Schnörr, C.: Robust 3D object registration without explicit correspondence using geometric integration. Mach. Vis. Appl. 21(5), 601–611 (2010)Google Scholar
  17. 17.
    Jian, B., Vemuri, B.C.: Robust point set registration using Gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1633–45 (2011)Google Scholar
  18. 18.
    Gerogiannis, D., Nikou, C., Likas, A.: The mixtures of Students’ t-distributions as a robust framework for rigid registration. Image Vis. Comput. 27(9), 1285–1294 (2009)Google Scholar
  19. 19.
    Sfikas, G., Nikou, C., Galatsanos, N.P.: Edge preserving spatially varying mixtures for image segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2008)Google Scholar
  20. 20.
    Bishop, C.M., Svensén, M.: Robust Bayesian mixture modelling. Neurocomputing 64, 235–252 (2005)Google Scholar
  21. 21.
    Zhou, Z., Zheng, J., Dai, Y., Zhou, Z., Chen, S.: Robust non-rigid point set registration using Student’s-t mixture model. PLoS ONE 9, e91381 (2014)Google Scholar
  22. 22.
    Tsin, Y., Kanade, T.: A correlation-based approach to robust point set registration. In: Proceedings of European Conference on Computer Vision, pp. 558–569 (2004)Google Scholar
  23. 23.
    Ding, M., Fan, G.: Articulated and generalized Gaussian kernel correlation for human pose estimation. IEEE Trans. Image Process. 25(2), 776–789 (2016)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Wang, G., Wang, Z., Chen, Y., Zhao, W.: A robust non-rigid point set registration method based on asymmetric Gaussian representation. Comput. Vis. Image Underst. 141, 67–80 (2015)Google Scholar
  25. 25.
    Kato, T., Omachi, S., Aso, H.: Asymmetric Gaussian and its application to pattern recognition. In: Structural, Syntactic, and Statistical Pattern Recognition, pp. 227–242 (2002)Google Scholar
  26. 26.
    Wang, G., Zhou, Q., Chen, Y.: Robust non-rigid point set registration using spatially constrained Gaussian fields. IEEE Trans. Image Process. 26(4), 1759–1769 (2017)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Boughorbel, F., Mercimek, M., Koschan, A., Abidi, M.: A new method for the registration of three-dimensional point-sets: the Gaussian fields framework. Image Vis. Comput. 28(1), 124–137 (2010)Google Scholar
  28. 28.
    Ma, J., Qiu, W., Zhao, J., Ma, Y., Yuille, A.L., Tu, Z.: Robust L2E estimation of transformation for non-rigid registration. IEEE Trans. Signal Process. 63(5), 1115–1129 (2015)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Ma, J., Zhao, J., Yuille, A.L.: Non-rigid point set registration by preserving global and local structures. IEEE Trans. Image Process. 25(1), 53–64 (2016)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)Google Scholar
  31. 31.
    Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)Google Scholar
  32. 32.
    Zheng, Y., Doermann, D.: Robust point matching for nonrigid shapes by preserving local neighborhood structures. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 643–649 (2006)Google Scholar
  33. 33.
    Ma, J., Zhao, J., Tian, J., Tu, Z., Yuille, A.L.: Robust estimation of nonrigid transformation for point set registration. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2147–2154 (2013)Google Scholar
  34. 34.
    Ma, J., Zhao, J., Tian, J.W., Yuille, A.L., Tu, Z.W.: Robust point matching via vector field consensus. IEEE Trans. Image Process. 23(4), 1706–1721 (2014)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Ge, S., Fan, G.: Non-rigid articulated point set registration with local structure preservation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2015)Google Scholar
  36. 36.
    Panaganti, V., Aravind, R.: Robust nonrigid point set registration using Graph-Laplacian regularization. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision, pp. 1137–1144 (2015)Google Scholar
  37. 37.
    Pellegrini, S., Schindler, K., Nardi, D.: A generalization of the ICP algorithm for articulated bodies. In: Proceedings of British Machine Vision Conference, pp. 87.1–87.10 (2008)Google Scholar
  38. 38.
    Horaud, R., Forbes, F., Yguel, M., Dewaele, G., Zhang, J.: Rigid and articulated point registration with expectation conditional maximization. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 587–602 (2011)Google Scholar
  39. 39.
    Ye, M., Yang, R.: Real-time simultaneous pose and shape estimation for articulated objects using a single depth camera. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2353–2360 (2014)Google Scholar
  40. 40.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)Google Scholar
  41. 41.
    Chappelow, J., Madabhushi, A., Rosen, M., Tomaszeweski, J., Feldman, M.: Multimodal image registration of ex vivo 4 Tesla MRI with whole mount histology for prostate cancer detection. In: Medical Imaging 2007: Image Processing, Proceeddings of SPIE 6512, 65121S (2007)Google Scholar
  42. 42.
    Aljabar, P., Robin W., Daniel, R.: Manifold learning for medical image registration, segmentation, and classification. In: Machine Learning in Computer-Aided Diagnosis: Medical Imaging Intelligence and Analysis, pp. 351–372. IGI Global (2012)Google Scholar
  43. 43.
    Mateus, D., Cuzzolin, F., Horaud, R., Boyer, E.: Articulated shape matching using locally linear embedding and orthogonal alignment. In: IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, pp. 1–8. IEEE (2007)Google Scholar
  44. 44.
    Ge, S., Fan, G., Ding, M.: Non-rigid point set registration with global–local topology preservation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 245–251 (2014)Google Scholar
  45. 45.
    Ge, S., Fan, G.: Articulated non-rigid point set registration for human pose estimation from 3D sensors. Sensors 15(7), 15218 (2015)Google Scholar
  46. 46.
    Ge, S., Fan, G.: Sequential non-rigid point registration for 3D human pose tracking. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 1105–1109. IEEE (2015)Google Scholar
  47. 47.
    Ge, S. Fan, G.: Non-rigid articulated point set registration for human pose estimation. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 94–101. IEEE (2015)Google Scholar
  48. 48.
    de Sousa, S., Kropatsch, W.G.: Graph-based point drift: graph centrality on the registration of point-sets. Pattern Recognit. 48(2), 368–379 (2015)zbMATHGoogle Scholar
  49. 49.
    Li, X., Yankeelov, T.E., Peterson, T.E., Gore, J.C., Dawant, B.M.: Constrained non-rigid registration for whole body image registration: method and validation. In: Proceedings of SPIE, Medical Imaging: Image Processing, pp. 651202.1–651202.8 (2007)Google Scholar
  50. 50.
    Mateus, D., Horaud, R., Knossow, D., Cuzzolin, F., Boyer, E.: Articulated shape matching using Laplacian eigenfunctions and unsupervised point registration. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  51. 51.
    Ye, M., Shen, Y., Du, C., Pan, Z., Yang, R.: Real-time simultaneous pose and shape estimation for articulated objects with a single depth camera. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1517–1532 (2016)Google Scholar
  52. 52.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, New York (1995)zbMATHGoogle Scholar
  53. 53.
    Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Comput. 7, 219–269 (1995)Google Scholar
  54. 54.
    Micchelli, C.A., Pontil, M.A.: On learning vector-valued functions. Neural Comput. 17(1), 177–204 (2005)MathSciNetzbMATHGoogle Scholar
  55. 55.
    Zhao, J., Ma, J., Tian, J., Ma, J., Zhang, D.: A robust method for vector field learning with application to mismatch removing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2977–2984 (2011)Google Scholar
  56. 56.
    Yuille, A.L., Grzywacz, N.M.: A mathematical analysis of the motion coherence theory. Int. J. Comput. Vis. 3(2), 155–175 (1989)Google Scholar
  57. 57.
    Ge, S., Fan, G.: Articulated non-rigid point set registration for human pose estimation from 3D sensors. Sensors 15(7), 15218–15245 (2015)Google Scholar
  58. 58.
    Sumner, R.W., Popović, J.: Deformation transfer for triangle meshes. ACM Trans. Graph. 23(3), 399–405 (2004)Google Scholar
  59. 59.
    Chen, X., Golovinskiy, A., Funkhouser, T.: A benchmark for 3D mesh segmentation. In: ACM Transactions on Graphics (Proceedings of the SIGGRAPH), vol. 28, no. 3 (2009)Google Scholar
  60. 60.
    Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: SCAPE: shape completion and animation of people. ACM Trans. Graph. 24, 408–416 (2005)Google Scholar
  61. 61.
    Christensen, G.E., Johnson, H.J.: Invertibility and transitivity analysis for nonrigid image registration. J. Electron. Imaging 12(1), 106–117 (2003)Google Scholar
  62. 62.
    Datteri, R.D., Liu, Y., D’Haese, P.-F., Dawant, B.M.: Validation of a nonrigid registration error detection algorithm using clinical MRI brain data. IEEE Trans. Med. Imaging 34(1), 86–96 (2015)Google Scholar
  63. 63.
    Greengard, L., Strain, J.: The fast Gauss transform. SIAM J. Sci. Stat. Comput. 12(1), 79–94 (1991)MathSciNetzbMATHGoogle Scholar
  64. 64.
    Jia, Z., Chang, Y.-J., Lin, T.-H., Chen, T.: Dense interpolation of 3D points based on surface and color. In: Proceedings of IEEE International Conference on Image Processing, pp. 869–872 (2011)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Electrical and Computer EngineeringOklahoma State UniversityStillwaterUSA
  2. 2.School of Automation and Information EngineeringXi’an University of TechnologyXi’anChina

Personalised recommendations