# A Differential–Algebraic Projective Framework for the Deformable Single-View Geometry of the 1D Perspective Camera

- 176 Downloads

## Abstract

Single-View Geometry (SVG) studies the world-to-image mapping or warp, which is the relationship that exists between a body’s model and its image. For a rigid body observed by a projective camera, the warp is described by the usual camera matrix and its properties. However, it is clear that for a body whose deformation state changes between the body’s model and its image, the ‘simple,’ globally parameterized warp described solely by the camera matrix, breaks down. Existing work has exploited deformation to reconstruct the deformed body from its image, but did not establish the properties of the deformable warp. Studying these properties is part of deformable SVG and forms a recent research topic. Because deformations may take place anywhere in the object’s body, and because they may be uncorrelated, the warp is local in nature. Using a differential framework is thus an obvious choice. We propose a differential–algebraic projective framework based on modeling the body’s surface by a locally rational projective embedding and on the 1D projective camera. We show that this leads, via the study of univariate rational functions, to differential invariants that the warp must satisfy. It may seem surprising, given the generic hypothesis made on the observed body, hardly stronger than mere local smoothness, that constraints can still be found. Our framework generalizes the Schwarzian derivative, the first-order projective differential invariant, which holds under the assumption that the body’s shape is locally linear. Our invariants may be used to construct regularizers to be used in warp estimation. We report experimental results of two types on simulated and real data. The first type shows that the proposed invariants hold well for an independently estimated warp. The second type shows that the proposed regularizers improve warp estimation from point correspondences compared to the classical derivative-penalizing regularizers.

## Keywords

Visual geometry Deformation Rational function Differential invariant## Notes

### Acknowledgements

This research has received funding from the EU’s FP7 through the ERC research grant 307483 FLEXABLE. We thank the authors of [11] for the real dataset and Yan Gérard for his kind feedback on the paper.

## Supplementary material

## References

- 1.Bartoli, A., Gérard, Y., Chadebecq, F., Collins, T., Pizarro, D.: Shape-from-template. IEEE Trans. Pattern Anal. Mach. Intell.
**37**(10), 2099–2118 (2015)CrossRefGoogle Scholar - 2.Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Berlin (2006)zbMATHGoogle Scholar
- 3.Bookstein, F .L.: Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell.
**11**(6), 567–585 (1989)CrossRefzbMATHGoogle Scholar - 4.Duchon, J.: Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces. RAIRO Anal. Numér.
**10**, 5–12 (1976)MathSciNetGoogle Scholar - 5.Fabbri, R., Kimia, B.B.: Multiview differential geometry of curves. Int. J. Comput. Vis.
**117**(3), 1–23 (2016)MathSciNetzbMATHGoogle Scholar - 6.Faugeras, O.: Three-Dimensional Computer Vision. MIT Press, Cambridge (1993)Google Scholar
- 7.Faugeras, O., Mourrain, B.: On the geometry and algebra of the point and line correspondences between \(n\) images. In: International Conference on Computer Vision (1995)Google Scholar
- 8.Faugeras, O., Quan, L., Sturm, P.: Self-calibration of a 1d projective camera and its application to the self-calibration of a 2d projective camera. IEEE Trans. Pattern Anal. Mach. Intell.
**22**(10), 1179–1185 (2000)CrossRefGoogle Scholar - 9.Forsyth, D., Ponce, J.: Computer Vision—A Modern Approach, International edn. Pearson, London (2012)Google Scholar
- 10.Fortun, D., Bouthemy, P., Kervrann, C.: Optical flow modeling and computation: a survey. Comput. Vis. Image Underst.
**134**, 1–21 (2015)CrossRefzbMATHGoogle Scholar - 11.Gallardo, M., Pizarro, D., Bartoli, A., Collins, T.: Shape-from-template in flatland. In: International Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
- 12.Gumerov, N.A., Zandifar, A., Duraiswami, R., Davis, L.S.: 3D structure recovery and unwarping surfaces applicable to planes. Int. J. Comput. Vis.
**66**(3), 261–281 (2006)CrossRefGoogle Scholar - 13.Hartley, R .I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
- 14.Kaminski, J., Shashua, A.: Multiple view geometry of general algebraic curves. Int. J. Comput. Vis.
**56**(3), 195–219 (2004)CrossRefGoogle Scholar - 15.Ovsienko, V., Tabachnikov, S.: Projective Differential Geometry Old and New. Cambridge University Press, Cambridge (2005)zbMATHGoogle Scholar
- 16.Ovsienko, V., Tabachnikov, S.: What is... the schwarzian derivative? Not. AMS
**56**(1), 34–36 (2009)MathSciNetzbMATHGoogle Scholar - 17.Parashar, S., Pizarro, D., Bartoli, A.: Isometric non-rigid shape-from-motion with riemannian geometry solved in linear time. IEEE Trans. Pattern Anal. Mach. Intell.
**40**(10), 2442–2454 (2018)CrossRefGoogle Scholar - 18.Perriollat, M., Hartley, R., Bartoli, A.: Monocular template-based reconstruction of inextensible surfaces. Int. J. Comput. Vis.
**95**(2), 124–137 (2011)MathSciNetCrossRefzbMATHGoogle Scholar - 19.Piegl, L., Tiller, W.: The NURBS Book. Monographs in Visual Communication, 2nd edn. Springer, Berlin (1997)zbMATHGoogle Scholar
- 20.Pizarro, D., Khan, R., Bartoli, A.: Schwarps: Locally projective image warps based on 2D schwarzian derivatives. Int. J. Comput. Vis.
**119**(2), 93–109 (2016)MathSciNetCrossRefzbMATHGoogle Scholar - 21.Press, W .H., Teukolsky, S .A., Vetterling, W .T., Flannery, B.P.: Numerical Recipes—The Art of Scientific Computing, 3rd edn. Cambridge University Press, Cambridge (2007)zbMATHGoogle Scholar
- 22.Quan, L., Kanade, T.: Affine structure from line correspondences with uncalibrated affine cameras. IEEE Trans. Pattern Anal. Mach. Intell.
**19**(8), 834–845 (1997)CrossRefGoogle Scholar - 23.Salzmann, M., Pilet, J., Ilic, S., Fua, P.: Surface deformation models for nonrigid 3D shape recovery. IEEE Trans. Pattern Anal. Mach. Intell.
**29**(8), 1–7 (2007)CrossRefGoogle Scholar - 24.Schmid, C., Zisserman, A.: The geometry and matching of lines and curves over multiple views. Int. J. Comput. Vis.
**40**(3), 199–234 (2000)CrossRefzbMATHGoogle Scholar - 25.Sendra, J.R., Winkler, F., Prez-Daz, S.: Rational Algebraic Curves A Computer Algebra Approach. Springer, Berlin (2008)CrossRefGoogle Scholar
- 26.Walker, R.J.: Algebraic Curves. Springer, Berlin (1978)CrossRefGoogle Scholar