A Differential–Algebraic Projective Framework for the Deformable Single-View Geometry of the 1D Perspective Camera

Abstract

Single-View Geometry (SVG) studies the world-to-image mapping or warp, which is the relationship that exists between a body’s model and its image. For a rigid body observed by a projective camera, the warp is described by the usual camera matrix and its properties. However, it is clear that for a body whose deformation state changes between the body’s model and its image, the ‘simple,’ globally parameterized warp described solely by the camera matrix, breaks down. Existing work has exploited deformation to reconstruct the deformed body from its image, but did not establish the properties of the deformable warp. Studying these properties is part of deformable SVG and forms a recent research topic. Because deformations may take place anywhere in the object’s body, and because they may be uncorrelated, the warp is local in nature. Using a differential framework is thus an obvious choice. We propose a differential–algebraic projective framework based on modeling the body’s surface by a locally rational projective embedding and on the 1D projective camera. We show that this leads, via the study of univariate rational functions, to differential invariants that the warp must satisfy. It may seem surprising, given the generic hypothesis made on the observed body, hardly stronger than mere local smoothness, that constraints can still be found. Our framework generalizes the Schwarzian derivative, the first-order projective differential invariant, which holds under the assumption that the body’s shape is locally linear. Our invariants may be used to construct regularizers to be used in warp estimation. We report experimental results of two types on simulated and real data. The first type shows that the proposed invariants hold well for an independently estimated warp. The second type shows that the proposed regularizers improve warp estimation from point correspondences compared to the classical derivative-penalizing regularizers.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. 1.

    We have that \({{\,\mathrm{sgn}\,}}(s)=1\) if s can be generated by an even number of interchanges of the elements of \([1,b+1]\), and \({{\,\mathrm{sgn}\,}}(s)=-1\) otherwise.

References

  1. 1.

    Bartoli, A., Gérard, Y., Chadebecq, F., Collins, T., Pizarro, D.: Shape-from-template. IEEE Trans. Pattern Anal. Mach. Intell. 37(10), 2099–2118 (2015)

    Article  Google Scholar 

  2. 2.

    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Berlin (2006)

    MATH  Google Scholar 

  3. 3.

    Bookstein, F .L.: Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 11(6), 567–585 (1989)

    Article  MATH  Google Scholar 

  4. 4.

    Duchon, J.: Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces. RAIRO Anal. Numér. 10, 5–12 (1976)

    MathSciNet  Google Scholar 

  5. 5.

    Fabbri, R., Kimia, B.B.: Multiview differential geometry of curves. Int. J. Comput. Vis. 117(3), 1–23 (2016)

    MathSciNet  MATH  Google Scholar 

  6. 6.

    Faugeras, O.: Three-Dimensional Computer Vision. MIT Press, Cambridge (1993)

    Google Scholar 

  7. 7.

    Faugeras, O., Mourrain, B.: On the geometry and algebra of the point and line correspondences between \(n\) images. In: International Conference on Computer Vision (1995)

  8. 8.

    Faugeras, O., Quan, L., Sturm, P.: Self-calibration of a 1d projective camera and its application to the self-calibration of a 2d projective camera. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1179–1185 (2000)

    Article  Google Scholar 

  9. 9.

    Forsyth, D., Ponce, J.: Computer Vision—A Modern Approach, International edn. Pearson, London (2012)

    Google Scholar 

  10. 10.

    Fortun, D., Bouthemy, P., Kervrann, C.: Optical flow modeling and computation: a survey. Comput. Vis. Image Underst. 134, 1–21 (2015)

    Article  MATH  Google Scholar 

  11. 11.

    Gallardo, M., Pizarro, D., Bartoli, A., Collins, T.: Shape-from-template in flatland. In: International Conference on Computer Vision and Pattern Recognition (2015)

  12. 12.

    Gumerov, N.A., Zandifar, A., Duraiswami, R., Davis, L.S.: 3D structure recovery and unwarping surfaces applicable to planes. Int. J. Comput. Vis. 66(3), 261–281 (2006)

    Article  Google Scholar 

  13. 13.

    Hartley, R .I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  14. 14.

    Kaminski, J., Shashua, A.: Multiple view geometry of general algebraic curves. Int. J. Comput. Vis. 56(3), 195–219 (2004)

    Article  Google Scholar 

  15. 15.

    Ovsienko, V., Tabachnikov, S.: Projective Differential Geometry Old and New. Cambridge University Press, Cambridge (2005)

    MATH  Google Scholar 

  16. 16.

    Ovsienko, V., Tabachnikov, S.: What is... the schwarzian derivative? Not. AMS 56(1), 34–36 (2009)

    MathSciNet  MATH  Google Scholar 

  17. 17.

    Parashar, S., Pizarro, D., Bartoli, A.: Isometric non-rigid shape-from-motion with riemannian geometry solved in linear time. IEEE Trans. Pattern Anal. Mach. Intell. 40(10), 2442–2454 (2018)

    Article  Google Scholar 

  18. 18.

    Perriollat, M., Hartley, R., Bartoli, A.: Monocular template-based reconstruction of inextensible surfaces. Int. J. Comput. Vis. 95(2), 124–137 (2011)

    MathSciNet  Article  MATH  Google Scholar 

  19. 19.

    Piegl, L., Tiller, W.: The NURBS Book. Monographs in Visual Communication, 2nd edn. Springer, Berlin (1997)

    MATH  Google Scholar 

  20. 20.

    Pizarro, D., Khan, R., Bartoli, A.: Schwarps: Locally projective image warps based on 2D schwarzian derivatives. Int. J. Comput. Vis. 119(2), 93–109 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  21. 21.

    Press, W .H., Teukolsky, S .A., Vetterling, W .T., Flannery, B.P.: Numerical Recipes—The Art of Scientific Computing, 3rd edn. Cambridge University Press, Cambridge (2007)

    MATH  Google Scholar 

  22. 22.

    Quan, L., Kanade, T.: Affine structure from line correspondences with uncalibrated affine cameras. IEEE Trans. Pattern Anal. Mach. Intell. 19(8), 834–845 (1997)

    Article  Google Scholar 

  23. 23.

    Salzmann, M., Pilet, J., Ilic, S., Fua, P.: Surface deformation models for nonrigid 3D shape recovery. IEEE Trans. Pattern Anal. Mach. Intell. 29(8), 1–7 (2007)

    Article  Google Scholar 

  24. 24.

    Schmid, C., Zisserman, A.: The geometry and matching of lines and curves over multiple views. Int. J. Comput. Vis. 40(3), 199–234 (2000)

    Article  MATH  Google Scholar 

  25. 25.

    Sendra, J.R., Winkler, F., Prez-Daz, S.: Rational Algebraic Curves A Computer Algebra Approach. Springer, Berlin (2008)

    Book  Google Scholar 

  26. 26.

    Walker, R.J.: Algebraic Curves. Springer, Berlin (1978)

    Book  Google Scholar 

Download references

Acknowledgements

This research has received funding from the EU’s FP7 through the ERC research grant 307483 FLEXABLE. We thank the authors of [11] for the real dataset and Yan Gérard for his kind feedback on the paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Adrien Bartoli.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Higher Dimensions

Fig. 6
figure6

Virtual 1D setup. We use this construction to show that the proposed invariants for the 1D setup form a subset of the invariants for the 2D setup

We give general points about extending our framework to higher dimensions. We then show that the 1D invariants provide a partial set of invariants for the 2D setup using a virtual 1D setup.

General Points

Extending our 1D framework to higher dimensions presents different types of difficulties, whether one extends the dimension of the source or the target space. Note that extending the framework to handle 2D warps requires one to extend the dimension of both the source and target spaces. Extending the dimension of the target space, for example to 2D in order to model the image of a deformable 3D curve, leads to \(\eta ^\top =[\eta _1\quad \eta _2]=\frac{1}{\gamma }[\alpha _1\quad \alpha _2]\). It is straightforward to see that this warp satisfies two invariants, \({\mathcal {I}}_{a,a}[\eta _1]\) and \({\mathcal {I}}_{a,a}[\eta _2]\), for a polynomial embedding of degree a. However, these two invariants, according to Proposition 1, guarantee that \(\eta _i=\frac{\alpha _i}{\gamma _i}\) for some \(\alpha _i,\gamma _i\in {\mathcal {P}}_a\), \(i=1,2\), but they do not guarantee that \(\gamma _1=\gamma _2\), which is a necessary condition for the SVG to be valid. It is quite likely that this requirement adds an invariant that \(\eta \) must fulfill. Extending the dimension of the source space, for example to 2D in order to model a deformable 3D surface, means that the partial derivatives will become tensor-valued functions. Though this will require one to use a tensorial notation, such as Einstein’s, this will also allow one to follow the proposed mathematical framework.

The Virtual 1D Setup

Our goal is to show that the 1D invariants form a subset of the 2D invariants. For that purpose, we construct a virtual 1D setup, as shown in Fig. 6. We start from the 2D setup, shown in Fig. 1, where the warp \(\eta ':\Omega '\rightarrow \mathbb {R} ^2\), \(\Omega ' \subset \mathbb {R} ^2\), is a 2D rational mapping. Therefore, both its source and target space dimensions must be reduced by one in order to apply our 1D invariants. Importantly, these reductions must preserve rationality. Our first step is to reduce the source space dimension. This is done by choosing a rational plane curve parameterized by \(\kappa :\Omega \rightarrow \mathbb {R} ^2\), a 1D rational embedding. Clearly, \(\eta '\circ \kappa :\Omega \rightarrow \mathbb {R} ^2\) is a rational function. The domain \(\Omega \subset \mathbb {R} \) of \(\kappa \) stands for the virtual 1D body’s model. Our second step is to reduce the target space dimension. This is done by introducing a virtual 1D camera, represented by \(\Pi :\mathbb {P}^2\rightarrow \mathbb {R} \), in the 2D image. Clearly, \(\eta =\Pi \circ \eta '\circ \kappa :\Omega \rightarrow \mathbb {R} \) is a rational function. Therefore, according to Proposition 1, there exist \(a,b\in \mathbb {N}\) such that the invariant \({\mathcal {I}}_{a,b}[\eta ]\) holds. The value of ab is upper-bounded by a combination of the degrees of \(\kappa \) and \(\eta '\) and the parameters of \(\Pi \). We can easily derive interesting cases out of this general formulation. For instance, choosing the rational plane curve as a straight horizontal line in the 2D model, we have \(\kappa (p)=[p\quad y]^\top \), where \(y\in \mathbb {R} \) is a constant, and the degree of \(\eta '\circ \kappa \) is at most \((a',b')\), the degree of \(\eta '\). Then, placing the virtual 1D camera at the origin with its principal axis along the vertical direction, we have \(\Pi ([q_1\quad q_2]^\top )=\frac{q_1}{q_2}\), and the degree of \(\eta \) becomes at most \((a',a')\). The real example shown in the experiments of §§6.1.3 and 6.2.2 represents a special case of this general principle.

We have shown that the 1D invariants can be applied to simple restrictions of the 2D warp, based on choosing a simple rational curve in the 2D model and a simple 1D projection in the 2D image. This forms a possible way to study the generalization of our 1D invariants to higher dimensions. A first step would be to study how to form a minimal set of restrictions among the many possible ones to form a set of mutually independent invariants. A second step would be to study whether the so-obtained invariant set forms sufficient conditions to characterize the set of 2D rational functions.

Proof of Proposition 1

We prove Proposition 1 in two parts. Part I is the forward implication and part II the reverse implication. Part I requires the following three lemmas. It constructs the canonical invariant \({\mathcal {I}}_{a,b}[\mu ]\) by differentiating the relationship \(\gamma \mu =\alpha \), called \({\mathcal {G}}[\mu ,\alpha ,\gamma ]\) in Lemma 1, at the orders \({\mathcal {Z}}\subset \mathbb {N}\), which we will show can be chosen as \({\mathcal {Z}}=[a+1,a+b+1]\) in Lemma 2. It then combines the different orders in Lemma 3 to eliminate all orders of \(\gamma \) and \(\alpha \) from the equations, resulting in the sought invariant, an equation depending on \(\mu \) only.

Lemma 1

We have \(\forall \mu \in C^\infty ,a,b\in \mathbb {N},\alpha \in {\bar{{\mathcal {P}}}}_a,\gamma \in {\bar{{\mathcal {P}}}}_b\):

$$\begin{aligned} \mu =\frac{\alpha }{\gamma } \,\,\,\Rightarrow \,\,\, \{ {\mathcal {H}}_{e,b}[\mu ,\gamma ]~|~e>a \}, \end{aligned}$$

with:

$$\begin{aligned} {\mathcal {H}}_{e,b}[\mu ,\gamma ] \,\,\,{\mathop {\Leftrightarrow }\limits ^{{\mathrm{def}}}}\,\,\, \left( \sum _{k=0}^{b} { e \atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k)} \,\,\,=\,\,\,0 \right) . \end{aligned}$$
(10)

Proof

We have that \(\mu =\frac{\alpha }{\gamma }\) implies \({\mathcal {G}}[\mu ,\alpha ,\gamma ]\), with:

$$\begin{aligned} {\mathcal {G}}[\mu ,\alpha ,\gamma ] \,\,\,{\mathop {\Leftrightarrow }\limits ^{{\mathrm{def}}}}\,\,\, \left( \gamma \mu \,\,\,=\,\,\,\alpha \right) . \end{aligned}$$
(11)

By taking the eth derivative of \({\mathcal {G}}[\mu ,\alpha ,\gamma ]\) using Leibniz’s rule, we obtain:

$$\begin{aligned} D^e{\mathcal {G}}[\mu ,\alpha ,\gamma ] \,\,\,\Leftrightarrow \,\,\, \left( \sum _{k=0}^e {e\atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k)} \,\,\,=\,\,\,\alpha ^{(e)} \right) . \end{aligned}$$
(12)

Using Eq. 1, we have that \(\alpha ^{(e)}=0\) for \(e>a\) and \(\gamma ^{(e)}=0\) for \(e>b\). We can thus rewrite \(D^e{\mathcal {G}}[\mu ,\alpha ,\gamma ]\) as:

$$\begin{aligned} {\mathcal {K}}_{e,a,b}[\mu ,\alpha ,\gamma ] \,\,\,{\mathop {\Leftrightarrow }\limits ^{{\mathrm{def}}}}\,\,\, \left( \sum _{k=0}^{\min (e,b)} { e \atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k)} \,\,\,=\,\,\,\mathbb {1}_{e\le a}\alpha ^{(e)} \right) , \end{aligned}$$

where \(\mathbb {1}\) is the indicator function, with \(\mathbb {1}_{\mathrm {true}} = 1\) and \(\mathbb {1}_{\mathrm {false}}=0\). For \(e>a\) we can simplify \({\mathcal {K}}_{e,a,b}[\mu ,\alpha ,\gamma ]\) to \({\mathcal {H}}_{e,b}[\mu ,\gamma ]\), using the relationship \(\min (e,b) \le b\) and \({e\atopwithdelims ()k}=0\) for \(k>e\) to set a fixed summation count. \(\square \)

Lemma 2

For \(\alpha \in {\bar{{\mathcal {P}}}}_a,\gamma \in {\bar{{\mathcal {P}}}}_b\) constructing a differential invariant \({\mathcal {I}}_{a,b}[\mu ]\) requires one to differentiate \({\mathcal {G}}[\mu ,\alpha ,\gamma ]\) at the orders in \({\mathcal {Z}}\subset \mathbb {N}\) with \(\min ({\mathcal {Z}})>a\) and \(|{\mathcal {Z}}|=b+1\).

Proof

Recall that deriving a differential invariant \({\mathcal {I}}_{a,b}[\mu ]\) on \(\mu \) from equations \(D^e{\mathcal {G}}[\mu ,\alpha ,\gamma ]\) formed at several orders e requires that \(\alpha \) and \(\gamma \) be eliminated, as well as their derivatives at all orders. We start with the elimination of \(\alpha \) and its derivatives. By construction \(D^e{\mathcal {G}}[\mu ,\alpha ,\gamma ]\) involves \(\alpha ^{(e)}\) and no other orders of \(\alpha \), as can be seen in Eq. 12. Therefore, the only way to cancel out all orders of \(\alpha \) to form an invariant on \(\mu \) is by differentiation of \({\mathcal {G}}[\mu ,\alpha ,\gamma ]\) at an order greater than a, the degree of \(\alpha \), thanks to Eq. 1. This proves \(\min ({\mathcal {Z}})>a\). Importantly, once \(\alpha \) has been canceled, the equations \(D^e{\mathcal {G}}[\mu ,\alpha ,\gamma ]\) become homogeneous, as can trivially be seen from Eq. 12. We now turn to the elimination of \(\gamma \) and its derivatives. The situation is different as each equation \(D^e{\mathcal {G}}[\mu ,\alpha ,\gamma ]\) depends on \(\{ \gamma ^{(0)},\dots ,\gamma ^{(e)} \}\). This means that we will have to form a linear system in \(\gamma \) and its derivatives. Forcing this system to have a solution will then allow us to eliminate \(\gamma \) and its derivatives, as will be shown in Proof of Lemma 3. This also means that differentiating \(D^e{\mathcal {G}}[\mu ,\alpha ,\gamma ]\) to form \(D^{e+1}{{\mathcal {G}}}[\mu ,\alpha ,\gamma ]\) introduces a new order \(\gamma ^{(e+1)}\) of \(\gamma \). Therefore, the first e orders in \({\mathcal {G}}[\mu ,\alpha ,\gamma ]\) involve the first e orders of \(\gamma \). This means that to eliminate all orders of \(\gamma \) to form an invariant on \(\mu \), one has to differentiate \({\mathcal {G}}[\mu ,\alpha ,\gamma ]\) to an order greater than b, the degree of \(\gamma \), thanks to Eq. 1, and that the number of orders involved must be greater than b. This is equivalent to requiring the size of \({\mathcal {Z}}\) to be greater than b. Each order beyond b then allows one to form a linear system, and thus an invariant. Because we want to form one invariant only, this proves \(|{\mathcal {Z}}|=b+1\). \(\square \)

Lemma 3

Defining \({\hat{{\mathcal {H}}}}_{a,b}[\mu ,\gamma ] {\mathop {=}\limits ^{{\mathrm{def}}}}\{ {\mathcal {H}}_{e,b}[\mu ,\gamma ]~|~e\in [a+1,a+b+1]\}\), we have:

$$\begin{aligned} \forall \mu ,\gamma \in C^\infty ,a,b\in \mathbb {N}^* \quad \quad {\hat{{\mathcal {H}}}}_{a,b}[\mu ,\gamma ] \,\,\,\Rightarrow \,\,\, {\mathcal {I}}_{a,b}[\mu ]. \end{aligned}$$

Proof

We choose \({\mathcal {U}},{\mathcal {V}}\subset \mathbb {N}\) to contain, respectively, the orders of \(\mu \) and \(\gamma \) involved in Eq. 10, namely \({\mathcal {U}}=[\max (a+1-b,0),a+b+1]\) and \({\mathcal {V}}=[0,b]\). The relationship \({\mathcal {H}}_{e,b}[\mu ,\gamma ]\) in Eq. 10 is bilinear in \(\mu ^{({\mathcal {U}})}\) and \(\gamma ^{({\mathcal {V}})}\), and we thus have \(\forall e \in \mathbb {N},a,b\in \mathbb {N},\exists {{\texttt {A}}} _{e,b} \in \mathbb {N}^{|{\mathcal {U}}|\times |{\mathcal {V}}|}\):

$$\begin{aligned} {\mathcal {H}}_{e,b}[\mu ,\gamma ] \,\,\,\Leftrightarrow \,\,\, \left( {\mu ^{({\mathcal {U}})}}^\top {{\texttt {A}}} _{e,b} \gamma ^{({\mathcal {V}})} \,\,\,=\,\,\,0 \right) . \end{aligned}$$

Together with \({\hat{{\mathcal {H}}}}_{a,b}[\mu ,\gamma ]\), this shows that \(\gamma ^{({\mathcal {V}})}\) is up to scale a non-trivial element in the kernel of the matrix-valued function \(\beta _{a,b}[\mu ] \in C^\infty \left( \mathbb {R},\mathbb {R} ^{(b+1)\times |{\mathcal {V}}|}\right) \) whose ‘rows’ are functions \({\mu ^{({\mathcal {U}})}}^\top {{\texttt {A}}} _{e,b} \in C^\infty \left( \mathbb {R},\mathbb {R} ^{1\times |{\mathcal {V}}|}\right) \) for \(e\in [a+1,a+b+1]\). Because \(|{\mathcal {V}}|=b+1\), we thus have \(\forall \mu ,\gamma \in C^\infty ,a,b\in \mathbb {N},\exists \beta _{a,b}[\mu ] \in C^\infty \left( \mathbb {R}, \mathbb {R} ^{(b+1)\times (b+1)} \right) \):

$$\begin{aligned} {\hat{{\mathcal {H}}}}_{a,b}[\mu ,\gamma ] \,\,\,\Leftrightarrow \,\,\, \left( \beta _{a,b}[\mu ] \gamma ^{([0,b])} = 0 \right) . \end{aligned}$$

In order to form a non-trivial invariant on \(\mu \), we force the kernel of \(\beta _{a,b}[\mu ]\) to be non-empty. This invariant may thus be found by expanding \(\det \left( \beta _{a,b}[\mu ] \right) = 0\). Inspecting \({\mathcal {H}}_{e,b}[\mu ,\gamma ]\) in Eq. 10, we find that the entry of matrix \(\beta _{a,b}[\mu ]\) at (ij) is given by:

$$\begin{aligned} \beta _{a,b;i,j}[\mu ] \,\,\,=\,\,\,{ a+i \atopwithdelims ()j-1 } \mu ^{(a+i-j+1)}. \end{aligned}$$
(13)

Using Leibniz’s formula, we finally expand \(\det \left( \beta _{a,b}[\mu ] \right) = 0\) to arrive at the canonical invariant 6. \(\square \)

Proof of Proposition 1, part I

We want to show the forward implication, \({\bar{{\mathcal {R}}}}_{a,b}[ \mu ] \Rightarrow {\mathcal {I}}_{a,b}[\mu ]\). We use the definition 4 to obtain \({\mathcal {R}}_{a,b}[\mu ] \Rightarrow \exists \alpha \in {\bar{{\mathcal {P}}}}_a,\gamma \in {\bar{{\mathcal {P}}}}_b \text {~s.t.~ }\mu =\frac{\alpha }{\gamma }\). From Lemma 1, we then have that this implies \(\{{\mathcal {H}}_{e,b}[\mu ,\gamma ]~|~e>a\}\). Lemma 2 shows that instantiating \({\mathcal {H}}_{e,b}\) at the orders in \([a+1,a+b+1]\), which is equivalent to \({\hat{{\mathcal {H}}}}_{a,b}[\mu ,\gamma ]\), is required to form the canonical invariant \({\mathcal {I}}_{a,b}[\mu ]\). These orders lead to the canonical invariant, and any other orders respecting Lemma 2 lead to a non-canonical invariant for the degree (ab) (see comment 1). Finally, applying Lemma 3 proves the forward implication in Eq. 6. \(\square \)

Comment 1

(Non-canonical invariants and number of invariants) The way the invariants are constructed depends on \({\mathcal {Z}}\subset \mathbb {N}\), which must satisfy the constraints from Lemma 2. We have defined the canonical invariant as the one obtained with \({\mathcal {Z}}=[a+1,a+b+1]\). However, for a given degree (ab), there exist an infinite number of \({\mathcal {Z}}\) subsets, each leading to a different invariant. There thus exist an infinite number of invariants.

Part II of our proof of Proposition 1 uses Lemma 4 to show that given the invariant \({\mathcal {I}}_{a,b}[\mu ]\) there exists a smooth vector-value function \({\varvec{\gamma }}\in (C^\infty )^{b+1}\), \({\varvec{\gamma }}^\top {\mathop {=}\limits ^{{\mathrm{def}}}}( \gamma _1 , \dots , \gamma _{b+1} )\) with \(\gamma _k \in C^\infty \), \(k=1,\dots ,b+1\), so that \(\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,{\varvec{\gamma }}]\) holds. The latter is defined as \(\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,{\varvec{\gamma }}] {\mathop {=}\limits ^{{\mathrm{def}}}}\{ \varvec{{\mathcal {H}}}_{e,b}[\mu ,{\varvec{\gamma }}]~|~e\in [a+1,a+b+1]\}\), where \(\varvec{{\mathcal {H}}}_{e,b}[\mu ,{\varvec{\gamma }}]\) is an extension of \({\mathcal {H}}_{e,b}[\mu ,\gamma ]\) where \(\gamma \)’s derivatives are replaced by the scalar-valued functions forming the vector-valued function \({\varvec{\gamma }}\) as:

$$\begin{aligned} \varvec{{\mathcal {H}}}_{e,c}[\mu ,{\varvec{\gamma }}] \,\,\,\Leftrightarrow \,\,\, \left( \sum _{k=0}^{c} { e \atopwithdelims ()k} \gamma _{k+1} \mu ^{(e-k)} \,\,\,=\,\,\,0 \right) . \end{aligned}$$

It then shows that this implies that at each point in \(\Omega \) either \(\mu \) or \(\gamma \) is a polynomial of some bounded degree. In the former case, \(\mu \) is also a univariate rational function. In the latter case, this is because the entries of \({\varvec{\gamma }}\) correspond to the derivatives of a scalar-valued function \(\gamma \). We can then show that there exist a polynomial \(\alpha \) such that \(\mu =\frac{\alpha }{\gamma }\). More specifically, our proof requires the following three lemmas.

Lemma 4

We have \(\forall \mu \in C^\infty ,a,b\in \mathbb {N}\):

$$\begin{aligned} {\mathcal {I}}_{a,b}[\mu ]\Rightarrow & {} \left( \exists {\varvec{\gamma }}\in (C^\infty )^{b+1} \text {~s.t.~ }\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,{\varvec{\gamma }}] \right) \end{aligned}$$
(14)
$$\begin{aligned} {\mathcal {I}}_{a,b}[\mu ]\Rightarrow & {} \left( \exists \gamma \in C^\infty \text {~s.t.~ }\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,\gamma ^{([0,b])}] \right) . \end{aligned}$$
(15)

Proof

We first prove Eq. 14. The way the invariant \({\mathcal {I}}_{a,b}\) was derived in the proof of Lemma 3 is by nullifying the determinant of a matrix-valued operator \(\beta _{a,b}\) on \(\mu \). In other words, \({\mathcal {I}}_{a,b}[\mu ]\) implies \(\det (\beta _{a,b}[\mu ])=0\), and thus that \(\beta _{a,b}[\mu ]\) has a non-empty null-space, which we represent by the vector-valued function \({\varvec{\gamma }}\), defined such that \(\beta _{a,b}[\mu ]{\varvec{\gamma }}= 0\). The smoothness of \({\varvec{\gamma }}\) is implied by the smoothness of \(\mu \) and \(\beta _{a,b}[\mu ]\), and the fact that the null-space of a rank-deficient matrix is obtained as a multi-linear combination of the matrix’ entries.

We now prove Eq. 15 by contradiction. Assuming that \(\forall \mu \in C^\infty \) s.t. \({\mathcal {I}}_{a,b}[\mu ]\), we have \(\not \exists \gamma \in C^\infty \) s.t. \(\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,\gamma ^{([0,b])}]\). By using the contrapositive of Lemma 1, this implies that \(\not \exists \alpha \in {\mathcal {P}}_{a},\gamma \in {\mathcal {P}}_{b} \text {~s.t.~ }\mu =\frac{\alpha }{\gamma }\). In other words, this implies the falsity of \({\mathcal {R}}_{a,b}[\mu ]\). However, the forward implication of Proposition 1, which is proved in part I of the proof, implies that the kernel of the canonical invariant comprises at least the rational functions of the appropriate degree, which contradicts the falsity of \({\mathcal {R}}_{a,b}[\mu ]\). \(\square \)

Lemma 5

We have \(\forall \mu ,\gamma \in C^\infty ,c,e\in \mathbb {N},e\ge c\):

$$\begin{aligned} \left( {\mathcal {H}}_{e,c}[\mu ,\gamma ] \wedge {\mathcal {H}}_{e+1,c}[\mu ,\gamma ] \right) \,\,\,\Rightarrow \,\,\, \gamma ^{(c+1)}\mu ^{(e-c)}=0. \end{aligned}$$

Proof

We first differentiate \({\mathcal {H}}_{e,c}[\mu ,\gamma ]\):

$$\begin{aligned} D{\mathcal {H}}_{e,c}[\mu ,\gamma ]\Leftrightarrow & {} \Bigg ( \sum _{k=0}^{c} {e\atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k+1)} \\&\,\, + \, \sum _{k=0}^{c} {e\atopwithdelims ()k} \gamma ^{(k+1)} \mu ^{(e-k)} \,\,\,=\,\,\,0 \Bigg ). \end{aligned}$$

We expand the first term as:

$$\begin{aligned}&\sum _{k=0}^{c} {e\atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k+1)} \\&\,\,\,=\,\,\,{e \atopwithdelims ()0}\gamma \mu ^{(e+1)} + \sum _{k=1}^{c} {e\atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k+1)} \\&\,\,\,=\,\,\,{e+1 \atopwithdelims ()0}\gamma \mu ^{(e+1)} + \sum _{k=1}^{c} {e\atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k+1)}, \end{aligned}$$

where the last equality follows from \({e \atopwithdelims ()0} = {e+1 \atopwithdelims ()0} = 1\) since \(e\ge 0\). We expand the second term as:

$$\begin{aligned}&\sum _{k=0}^{c} {e\atopwithdelims ()k} \gamma ^{(k+1)} \mu ^{(e-k)} \\&\,\,\,=\,\,\,\sum _{k=1}^{c+1} {e\atopwithdelims ()k-1} \gamma ^{(k)} \mu ^{(e-k+1)} \\&\,\,\,=\,\,\,\sum _{k=1}^{c} {e\atopwithdelims ()k-1} \gamma ^{(k)} \mu ^{(e-k+1)} + {e \atopwithdelims ()c} \gamma ^{(c+1)}\mu ^{(e-c)}. \end{aligned}$$

Using the relation \({e+1 \atopwithdelims ()k} = {e \atopwithdelims ()k-1} + {e \atopwithdelims ()k}\), we obtain:

$$\begin{aligned} D{\mathcal {H}}_{e,c}[\mu ,\gamma ]\Leftrightarrow & {} \Bigg ( \sum _{k=0}^{c} {e+1\atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k+1)} \\&\,\, + \, {e \atopwithdelims ()c} \gamma ^{(c+1)}\mu ^{(e-c)} \,\,\,=\,\,\,0 \Bigg ). \end{aligned}$$

Expanding \(D{\mathcal {H}}_{e,c}[\mu ,\gamma ]-{\mathcal {H}}_{e+1,c}[\mu ,\gamma ]\) yields \({e \atopwithdelims ()c}\gamma ^{(c+1)}\mu ^{(e-c)}=0\). Because \(e\ge c\), \({e \atopwithdelims ()c}>0\) and we arrive at \(\gamma ^{(c+1)}\mu ^{(e-c)}=0\). \(\square \)

Lemma 6

We have \(\forall \mu \in C^\infty ,c,e\in \mathbb {N},\gamma \in C^\infty ,\gamma ^{(c+1)}=0\):

$$\begin{aligned} {\mathcal {H}}_{e,c}[\mu ,\gamma ] \,\,\,\Rightarrow \,\,\, (\gamma \mu )^{(e)}=0. \end{aligned}$$

Proof

Let \(\alpha = \gamma \mu \). We thus have that \({\mathcal {G}}[\mu ,\alpha ,\gamma ]\) from Eq. 11 holds. We then form \(D^e{\mathcal {G}}[\mu ,\alpha ,\gamma ]\) from Eq. 12 and \(D^e{\mathcal {G}}[\mu ,\alpha ,\gamma ]-{\mathcal {H}}_{e,c}[\mu ,\gamma ]\) as:

$$\begin{aligned}&\left( \sum _{k=0}^e {e\atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k)} \,\,\,=\,\,\,\alpha ^{(e)} \right) \\&\,\,\,-\,\,\, \left( \sum _{k=0}^c {e\atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k)} \,\,\,=\,\,\,0 \right) . \end{aligned}$$

For \(e\le c\), we immediately obtain \(\alpha ^{(e)}=0\) because the second summation goes only up to e since \({e\atopwithdelims ()k}\) vanishes for \(k>e\). For \(e>c\), the relationship is rewritten as:

$$\begin{aligned} \left( \sum _{k=c+1}^e {e\atopwithdelims ()k} \gamma ^{(k)} \mu ^{(e-k)} \,\,\,=\,\,\,\alpha ^{(e)} \right) \end{aligned}$$

Because \(\gamma ^{(c+1)}=0\), this also leads to \(\alpha ^{(e)}=0\). \(\square \)

Proof of Proposition 1, part II

We want to show the reverse implication, \({\bar{{\mathcal {R}}}}_{a,b}[ \mu ] \Leftarrow {\mathcal {I}}_{a,b}[\mu ]\). We first prove the case \(b>0\). We use Lemma 4, Eq. 14, which is \({\mathcal {I}}_{a,b}[\mu ]\Rightarrow \left( \exists {\varvec{\gamma }}\in (C^\infty )^{b+1} \text{ s.t. } \varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,{\varvec{\gamma }}] \right) \). We first show \(\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,{\varvec{\gamma }}]\) admits as solution \((\mu ,{\varvec{\gamma }})\) for which \({\bar{{\mathcal {R}}}}_{a,b}[\mu ]\) holds and \({\varvec{\gamma }}= \gamma ^{([0,b])}\). We then show this is the only solution to \(\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,{\varvec{\gamma }}]\). We have from Lemma 4, Eq. 15, that \(\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,\gamma ^{([0,b])}]\) is solvable. We have that \(\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,\gamma ^{([0,b])}] \Leftrightarrow {\hat{{\mathcal {H}}}}_{a,b}[\mu ,\gamma ]\). By definition, \({\hat{{\mathcal {H}}}}_{a,b}[\mu ,\gamma ]\) implies \({\mathcal {H}}_{a+b,b}[\mu ,\gamma ]\wedge {\mathcal {H}}_{a+b+1,b}[\mu ,\gamma ]\) and we can thus apply Lemma 5 with \(c=b\) and \(e=a+b\), leading to:

$$\begin{aligned} {\hat{{\mathcal {H}}}}_{a,b}[\mu ,\gamma ] \,\,\,\Rightarrow \,\,\, \gamma ^{(b+1)}\mu ^{(a)}=0. \end{aligned}$$

This implies that there exists \(\Lambda \subset \mathbb {R} \) such that (i)\(\mu ^{(a)}=0\) on \(\Lambda \) and (ii)\(\gamma ^{(b+1)}\) on \(\mathbb {R} \backslash \Lambda \). In case (i), because \(\mu \) is smooth from a premise of Proposition 1, using Eq. 2 for \(\mu \) in \(\Lambda \) implies \(\mu \in {\mathcal {P}}_n\) with \(n \le a\), which is equivalent to \({\bar{{\mathcal {R}}}}_{a,0}[\mu ]\). In case (ii), because \(\gamma \) is smooth from Lemma 4, using Eq. 2 for \(\gamma \) in \(\mathbb {R} \backslash \Lambda \) implies \(\gamma \in {\mathcal {P}}_n\) with \(n<b+1\). We can use Lemma 6 with \(c=b\) and \(e=a+1\). Defining \(\alpha = \gamma \mu \), we obtain that \(\alpha ^{(a+1)}=0\), which implies through Eq. 2 that \(\alpha \in {\mathcal {P}}_n\) with \(n<a\). We can therefore write \(\mu =\frac{\alpha }{\gamma }\) and thus \({\bar{{\mathcal {R}}}}_{a,b}[\mu ]\). Combining cases (i) and (ii), we obtain that \({\mathcal {R}}_{a,b}[\mu ]\) holds since \(\max (0,b)=b\). We finally prove the case \(b=0\) with \(a\ge 0\). Using Eq. 6, we have that \({\mathcal {I}}_{a,0}[\mu ] \Leftrightarrow {a+1\atopwithdelims ()1} \mu ^{(a+1)} = 0\). The factor \({a+1\atopwithdelims ()1} = a+1\) and because \(a+1>0\) we arrive at \(\mu ^{(a+1)} = 0\) and thus \({\bar{{\mathcal {R}}}}_{a,0}\) holds.

We now show that the above solution is the only one to \(\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,{\varvec{\gamma }}]\). We use an induction on b. The key observation to this proof is that for \(b>0\) the leading submatrix of \(\beta _{a,b}[\mu ]\) is \(\beta _{a,b-1}[\mu ]\). We start with \(b=0\). In that case \(\beta _{a,0}[\mu ]= {a+1\atopwithdelims ()0} \mu ^{(a+1)}=\mu ^{(a+1)}\) where the last equality follows from \(a+1>0\). Because \({\mathcal {I}}_{a,0}[\mu ]\Leftrightarrow (\mu ^{(a+1)}=0)\), as can easily be found from Eq. 6, we directly have that \(\mu \in {\bar{{\mathcal {P}}}}_a\) which implies \({\bar{{\mathcal {R}}}}_{a,0}[\mu ]\). We now examine \(b>0\). In that case, \(\beta _{a,b}[\mu ] \in C^\infty (\mathbb {R},\mathbb {R} ^{(b+1)\times (b+1)})\), and by definition of the invariant we have \(\det (\beta _{a,b}[\mu ])=0\) and so \(\mathrm {rank}(\beta _{a,b}[\mu ]) \le b\). We then face two cases. In the first case, we have \(\mathrm {rank}(\beta _{a,b}[\mu ]) < b\). This means that any \((b\times b)\) minor of \(\beta _{a,b}[\mu ]\) vanishes. In particular, deleting the last row and column of \(\beta _{a,b}[\mu ]\), we observe that the obtained submatrix is \(\beta _{a,b-1}[\mu ]\). This is easily understood from Eq. 13, where we observe that the entries of \(\beta _{a,b}[\mu ]\) do not depend on b (just its size depends on b). Therefore, from the induction hypothesis, we have \({\bar{{\mathcal {R}}}}_{a,b-1}[\mu ]\) which implies \({\bar{{\mathcal {R}}}}_{a,b}[\mu ]\). In the second case, we have \(\mathrm {rank}(\beta _{a,b}[\mu ]) = b\). The equations \(\beta _{a,b}[\mu ]{\varvec{\gamma }}=0\) from \(\varvec{{\hat{{\mathcal {H}}}}}_{a,b}[\mu ,{\varvec{\gamma }}]\) are homogeneous. Any two solutions \({\varvec{\gamma }}\) and \({\varvec{\gamma }}'\) must thus be related by \({\varvec{\gamma }}= \tau {\varvec{\gamma }}'\) for some \(\tau \in C^\infty (\mathbb {R},\mathbb {R} ^*)\). Therefore, we have that \({\varvec{\gamma }}= \tau \gamma ^{([0,b])}\). Because \(\varvec{{\hat{{\mathcal {H}}}}}[\mu ,{\varvec{\gamma }}] = \varvec{{\hat{{\mathcal {H}}}}}[\mu ,\tau \gamma ^{([0,b])}] = \tau \varvec{{\hat{{\mathcal {H}}}}}[\mu ,\gamma ^{([0,b])}] = \tau {\hat{{\mathcal {H}}}}[\mu ,\gamma ]\), we conclude that all solutions to \(\varvec{{\hat{{\mathcal {H}}}}}[\mu ,{\varvec{\gamma }}]\) are equivalent. \(\square \)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bartoli, A. A Differential–Algebraic Projective Framework for the Deformable Single-View Geometry of the 1D Perspective Camera. J Math Imaging Vis 61, 1051–1068 (2019). https://doi.org/10.1007/s10851-019-00887-y

Download citation

Keywords

  • Visual geometry
  • Deformation
  • Rational function
  • Differential invariant