Advertisement

Using Isometry to Classify Correct/Incorrect 3D-2D Correspondences

  • Toby Collins
  • Adrien Bartoli
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8692)

Abstract

Template-based methods have been successfully used for surface detection and 3D reconstruction from a 2D input image, especially when the surface is known to deform isometrically. However, almost all such methods require that keypoint correspondences be first matched between the template and the input image. Matching thus exists as a current limitation because existing methods are either slow or tend to perform poorly for discontinuous or unsmooth surfaces or deformations. This is partly because the 3D isometric deformation constraint cannot be easily used in the 2D image directly. We propose to resolve that difficulty by detecting incorrect correspondences using the isometry constraint directly in 3D. We do this by embedding a set of putative correspondences in 3D space, by estimating their depth and local 3D orientation in the input image, from local image warps computed quickly and accurately by means of Inverse Composition. We then relax isometry to inextensibility to get a first correct/incorrect classification using simple pairwise constraints. This classification is then efficiently refined using higher-order constraints, which we formulate as the consistency between the correspondences’ local 3D geometry. Our algorithm is fast and has only one free parameter governing the precision/recall trade-off. We show experimentally that it significantly outperforms state-of-the-art.

Keywords

Input Image Deformable Model Pairwise Constraint Deformable Surface Correct Correspondence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Tran, Q.-H., Chin, T.-J., Carneiro, G., Brown, M.S., Suter, D.: In defence of ransac for outlier rejection in deformable registration. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 274–287. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  2. 2.
    Bartoli, A., Gérard, Y., Chadebecq, F., Collins, T.: On template-based reconstruction from a single view: Analytical solutions and proofs of well-posedness for developable, isometric and conformal surfaces. In: CVPR (2012)Google Scholar
  3. 3.
    Salzmann, M., Fua, P.: Linear local models for monocular reconstruction of deformable surfaces. PAMI 33, 931–944 (2011)CrossRefGoogle Scholar
  4. 4.
    Östlund, J., Varol, A., Ngo, D.T., Fua, P.: Laplacian meshes for monocular 3D shape recovery. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 412–425. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  5. 5.
    Salzmann, M., Fua, P.: Reconstructing sharply folding surfaces: A convex formulation. In: CVPR (2009)Google Scholar
  6. 6.
    Pilet, J., Lepetit, V., Fua, P.: Fast non-rigid surface detection, registration and realistic augmentation. IJCV (2008)Google Scholar
  7. 7.
    Alcantarilla, P.F., Bartoli, A.: Deformable 3D reconstruction with an object database. In: BMVC (2012)Google Scholar
  8. 8.
    Torresani, L., Hertzmann, A., Bregler, C.: Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors. PAMI 30 (2008)Google Scholar
  9. 9.
    Taylor, J., Jepson, A.D., Kutulakos, K.N.: Non-rigid structure from locally-rigid motion. In: CVPR (2010)Google Scholar
  10. 10.
    Zhou, F., De la Torre, F.: Deformable graph matching. In: CVPR (2013)Google Scholar
  11. 11.
    Duchenne, O., Bach, F.R., Kweon, I.S., Ponce, J.: A tensor-based algorithm for high-order graph matching. In: CVPR (2009)Google Scholar
  12. 12.
    Torresani, L., Kolmogorov, V., Rother, C.: Feature correspondence via graph matching: Models and global optimization. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 596–609. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Pizarro, D., Bartoli, A.: Feature-based deformable surface detection with self-occlusion reasoning. IJCV (2012)Google Scholar
  14. 14.
    Leordeanu, M., Hebert, M.: A spectral technique for correspondence problems using pairwise constraints. In: ICCV (2005)Google Scholar
  15. 15.
    Shaji, A., Varol, A., Torresani, L., Fua, P.: Simultaneous point matching and 3D deformable surface reconstruction. In: CVPR (2010)Google Scholar
  16. 16.
    Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. IJCV 60, 63–86 (2004)CrossRefGoogle Scholar
  17. 17.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV (2004)Google Scholar
  18. 18.
    Bartoli, A., Gerard, Y., Chadebecq, F., Collins, T.: On template-based reconstruction from a single view: Analytical solutions and proofs of well-posedness for developable, isometric and conformal surfaces. In: CVPR (2012)Google Scholar
  19. 19.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparison of affine region detectors. IJCV (2005)Google Scholar
  20. 20.
    Matthews, I., Baker, S.: Active appearance models revisited. IJCV (2004)Google Scholar
  21. 21.
    Brunet, F., Gay-Bellile, V., Bartoli, A., Navab, N., Malgouyres, R.: Feature-driven direct non-rigid image registration. IJCV (2011)Google Scholar
  22. 22.
    Bartoli, A., Collins, T.: Template-based isometric deformable 3D reconstruction with sampling-based focal length self-calibration. In: CVPR (2013)Google Scholar
  23. 23.
    Collins, T., Bartoli, A.: Infinitesimal plane-based pose estimation. IJCV (July 2014)Google Scholar
  24. 24.
    Li, S., Xu, C., Xie, M.: A robust O(n) solution to the perspective N point problem. PAMI (2012)Google Scholar
  25. 25.
    Schweighofer, G., Pinz, A.: Robust pose estimation from a planar target. Pattern Analysis and Machine Intelligence (PAMI) 28, 2024–2030 (2006)CrossRefGoogle Scholar
  26. 26.
    Varol, A., Salzmann, M., Fua, P., Urtasun, R.: A constrained latent variable model. In: CVPR (2012)Google Scholar
  27. 27.
    Salzmann, M., Hartley, R., Fua, P.: Convex optimization for deformable surface 3-d tracking. In: ICCV (2007)Google Scholar
  28. 28.
    Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. IJCV (2011)Google Scholar
  29. 29.
    Huang, Q.X., Adams, B., Wicke, M., Guibas, L.J.: Non-rigid registration under isometric deformations. Comput. Graph. Forum (2008)Google Scholar
  30. 30.
    Tam, G.K.L.: quan Cheng, Z., kun Lai, Y., Langbein, F.C., Liu, Y., Marshall, D., Martin, R.R., fang Sun, X., Rosin, P.L.: Registration of 3D point clouds and meshes: A survey from rigid to non-rigid. Visualization and Computer Graphics (2013)Google Scholar
  31. 31.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV (2004)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Toby Collins
    • 1
  • Adrien Bartoli
    • 1
  1. 1.ALCoV-ISIT, UMR 6284 CNRS/Université d’AuvergneClermont-FerrandFrance

Personalised recommendations