1 Introduction

Conic as an important image primitive has been studied very well in the early 1990s [13]. Many major problems in computer vision, such as reconstruction, motion estimation, and pose determination, can be solved with two coplanar conics. For instance, Forsyth et al. use projective invariants of coplanar conic pairs to recognize curved planar objects [4]. Rothwell et al. use four intersection points of two conics to obtain homography which in fact results in the solution of relative motion and pose [5]. However, the above works all assume that the conics’ parameters are known which is not always available in practice. If the a priori knowledge about the object conics is scarce or imprecise, there is no method to handle these cases. More importantly, the methods of computing homography in [1, 5] are not suitable for calibration because the proposed posterior rules of removing the correspondence ambiguity are only effective when the intrinsic parameters of the camera are given.

In the 2000s, people explored various 2D conics patterns, such as concentric circles, confocal conics, coplanar circles, Principal-Axes Aligned (PAA) conics, conics with a common axis of symmetry, enclosing ellipses, and degenerate conics with double complex contact to calibrate a camera [613]. The core step in such algorithms is to determine the homography between the model plane and its image. But it is hard to extend their methods as they only find one special pattern. Moreover, due to lack of the natural link between different conditions, there is no explanation for the confusing question why this pattern is valid while others are not.

In recent years, Zhao [14] proposed a novel method of the 2D Euclidean structure recovery from the conic feature correspondences. The conic features are transformed from the homogeneous coordinates to the lifted coordinates to represent the geometric objects without considering the conic dual to the absolute points. Wang et al. [15] propose an algorithm which is efficient and easy to estimate the pose of camera based on the conic correspondences from world plane to image plane system. The above method needs more than two conics to work. However, it is not common to see many conics in the nature scene.

In fact, the key of camera calibration based on two coplanar conics is how to obtain homography using partial information of the conics in the model plane. In other words, what is the minimal condition used to compute the homography for two coplanar conics? Based on this idea, we propose a two-step method to give a unified solution of conics-based camera calibration. The proposed method not only easily explains all existing conics patterns, but also finds other possibilities. The first step is to get the unknown parameters by using geometric invariance. In this step, we discuss which form of projective invariants should be chosen to construct the equations on the unknown parameters followed by the algebraic and geometric explanation of ambiguous solutions. This ambiguity problem results in the uncertain parameters of the conics. The second step is to compute homography by utilizing four intersection points of two conics and their correspondences in the image, no matter whether they are real, complex, or partially complex. This correspondence ambiguity will lead to multiple possibilities of correspondences and the ambiguous homographies. To obtain the only correct solution, these two kinds of ambiguities must be removed. Thereby, owing to the Centre Circle constraint [16, 17] which provides the geometrical explanation of the world-to-image homography in perspective transformation, we set a cuboid bound for the camera center [18] to judge whether the computed homography is correct. Finally, the experiments with real and simulated data verify the correctness of the proposed method.

The contributions of this paper could be concluded as follows:

  • A unified explanation to the problem of calibration with coplanar conics is proposed.

  • We show how to use projective invariants to compute unknown parameters of conics.

  • We analyze the correspondence ambiguity of four intersection points of conics.

  • We apply the Centre Circle constraint to remove the above two ambiguities.

The paper is organized as follows. Section 2 includes some preliminaries. Section 3 gives the problem statements. Section 4 describes different invariant forms of conics and the algebraic and geometric reasons of ambiguous solutions. Section 5 first classifies the reasons of the points-correspondence ambiguity into three aspects. Then, several constraints are adopted to seriously reduce the number of the ambiguous correspondences. The degenerate situations are also discussed in this section. Section 6 provides our experimental results with simulated and real data.

2 Preliminaries

2.1 Basic equations

Suppose the object plane is located on Z=0 plane in the world coordinate system. The world-to-image homography matrix H is given by

$$ \mathbf{H}=\mathbf{K}[\!\mathbf{r}_{1}, \mathbf{r}_{2}, \mathbf{t}], \quad \text{where} \quad \mathbf{K} = \left[ \begin{array}{ccc} f_{u} & & u_{0} \\ & f_{v} & v_{0} \\ & & 1 \end{array} \right] $$
(1)

where (u0,v0) are the pixel coordinates of the principal point and fu and fv are the scale factors in the image’s u and v axes. r1, r2 are the first two vectors of rotation matrix R. t denotes the translation vector between the world coordinate system and the camera coordinate system. K is called the intrinsic parameter matrix and (R, t) are called the extrinsic parameters.

Two constraints on K provided by the homography are:

$$ \mathbf{h}_{1}^{T}\mathbf{K}^{-T}\mathbf{K}^{-1}\mathbf{h}_{2}=0 $$
(2)
$$ \mathbf{h}_{1}^{T}\mathbf{K}^{-T}\mathbf{K}^{-1}\mathbf{h}_{1}=\mathbf{h}_{2}^{T}\mathbf{K}^{-T}\mathbf{K}^{-1}\mathbf{h}_{2} $$
(3)

where h1, h2 are the first two columns of H. It is well known in [1921] that given the correct correspondences of four coplanar points, the 2D homography H can be solved.

2.2 Projective invariants related to two coplanar conics

Invariant theory as an important technique has been investigated in previous literatures [2, 4, 22, 23]. There are mainly three forms of invariants used by researchers as follows:

1) The form of trace. For two coplanar conics C1 and C2 (here, determinants of the matrices must be 1 through normalization), two scalar invariants [4, 22] are given by

$$ {}I_{C_{1}C_{2}} = Trace\left(\mathbf{C}_{1}^{-1}\mathbf{C}_{2}\right), \quad I_{C_{2}C_{1}} = Trace\left(\mathbf{C}_{2}^{-1}\mathbf{C}_{1}\right) $$
(4)

2) The form of the generalized eigenvalues. The generalized eigenvalues denoted by λ1,λ2,λ3 of two conics are projective invariants as a set, up to a scale factor. So two ratios from three generalized eigenvalues are the scalar invariants [4, 10]. One simple choice is:

$$ \lambda_{1}/\lambda_{3}, \quad \lambda_{2}/\lambda_{3} $$
(5)

3) The form of real invariants. Gros and Quan [23] utilized a generalization method to obtain two real invariants of two conics in which the matrices of the conics can be multiplied by a scalar.

$$\begin{array}{*{20}l} I_{1} &= \frac{Trace\left(\mathbf{C}_{2}^{-1} \mathbf{C}_{1}\right)\det \mathbf{C}_{2}} {\left(Trace\left(\mathbf{C}_{1}^{-1}\mathbf{C}_{2}\right)\right)^{2}\det\mathbf{C}_{1}}, \\ I_{2} &= \frac{Trace\left(\mathbf{C}_{1}w^{-1}\mathbf{C}_{2}\right)\det\mathbf{C}_{1}} {\left(Trace\left(\mathbf{C}_{2}^{-1}\mathbf{C}_{1}\right)\right)^{2}\det\mathbf{C}_{2}} \end{array} $$
(6)

Note that Eq. 4 is closely related to Eq. 6. Detailed distinction in application is shown in Section 4.1 and more specific discussion is stated in [23].

2.3 The Centre Circle constraint and the cuboid bound for the optical center

Gurdjos et al. [16] proposed a Centre Circle theory to explain the constraints imposed from a 2D homography on the optical center. When a planar figure is the central projection of another planar figure, the center of projection then lies on a spatial circle (called Centre Circle) which is intersected by a sphere S (called Centre Sphere) and a plane perpendicular to this intersection (called Centre Plane). The two equations determining the position of the Centre Circle can be deduced from Eqs. (2) and (3) respectively.

Owing to the Centre Circle constraint described above, Cai [18] builds a geometric criterion to judge the computed homography’s correctness, i.e., the Centre Circle must appear near the camera center. This criterion actually includes two basic physical meanings: the principal point should be near the image center and the focal length should be limited by the prior knowledge of the lens. Here, judging whether the camera centers lie in the geometric cuboid bound can be divided into two steps. First, judge whether the Centre Line intersected by the Centre Plane and the image plane passes through the given square region. Second, if the first condition suffices, judge whether the focal length varying with the changes in principal point is in the given range (see Fig. 1). Thus, for a cuboid bound, there are two controlling parameters: the one denoting the radius of the square controls the varying scope of the principal point and the other one denoting the height of the cuboid controls the varying range of the focal length.

Fig. 1
figure 1

The Centre Circle and the cuboid bound

3 Problem statements

Assume that the correspondences between two coplanar conics and their images are given. The basic equations about two conics and 2D homography are expressed by

$$ \tilde{\mathbf{C}}_{1}\equiv \mathbf{H}^{-T}\mathbf{C}_{1}\mathbf{H}^{-1}, \quad \tilde{\mathbf{C}}_{2}\equiv \mathbf{H}^{-T}\mathbf{C}_{2}\mathbf{H}^{-1} $$
(7)

where H denotes the homography. C1 and C2 are the matrices of the two conics in the model plane. Symbol \(\tilde {\mathbf {X}}\) denotes the image of the object X in the model plane. The symbol “ ≡” denotes the equivalence up to a scale factor.

Note that in the previous works [1, 5], C1 and C2 are totally known. Thus, there are 10 constraints on 8 unknowns in the process of computing homography and 10 constraints on 6 unknowns in pose estimation. By counting unknowns, if C1 and C2 are partially known, we can easily obtain the minimal condition of computing homography, i.e., C1 and C2 have two unknown parameters and the equations provide 10 constraints on 10 unknowns. Although the equations can be easily written, it is impossible to directly solve them due to the complicated nonlinearity.

Let us reconsider this problem by observing Fig. 2. After abandoning a global scale factor for simplicity, two general coplanar conics C1 and C2 can be denoted by six meaningful parameters: the eccentricities e1 and e2, the displacements between the centers of two conics dx and dy, the relative rotation θ, and the relative scale factor k between two conics. In another way, C1 and C2 (that are two ellipses here for simplicity) with six parameters are expressed by the following equations:

$$ {}x^{2}+axy+by^{2}-1=0, \quad cx^{2}+dy^{2}+ex+fy-1=0 $$
(8)
Fig. 2
figure 2

Two general conics with six parameters in our coordinate system

According to the number of prior conditions imposed on two base conics, we classify two coplanar conics into six classes which are two general conics, two Principal-Axis Parallel (PAP) conics (i.e., a=0), two Center Coaxial (CC) conics (i.e., f=0), two One Principal-Axis Superposed (OPAS) conics (i.e., a=0 and f=0), two concentric conics (i.e., e=0 and f=0), and two Principal-Axes Aligned (PAA) conics (i.e., a=0, e=0, and f=0). For most of the classes, we find that the quadrangle formed by four intersection points has different shapes and degrees of freedom (DOF) (see Fig. 3 and the proof is provided in Appendix). Such a classification not only gives us many geometrical explanations, but also provides some conveniences in further computing.

Fig. 3
figure 3

Four conics classes with different shapes. From left to right, they are respectively PAP conics with three DOF, OPAS conics with two DOF, concentric conics with two DOF, and PAA conics with one DOF. Their quadrangles are respectively a quadrangle whose four vertices lie on a common circle, a trapezoid, a parallelogram, and a rectangle

Based on the above analysis, we can propose an important argument about the number of the unknowns and given constraints as follows.

Proposition 1

A necessary and sufficient condition of obtaining the world-to-image homography up to a transformation for two coplanar conics is that four effective constraints on six parameters are given.

Proof

(⇒) Assume four effective constraints on the above six parameters are given (the simplest form is to give exact values of four parameters; the term “effective” means that the four constraints must be independent and different from the two constraints provided by geometric invariance), we can use two scalar invariants of two conics to establish two equations (be nonlinear in most cases) of two unknown parameters. By solving them, we can get all parameters. Thus, the four intersection points can be obtained followed by the computation of homography. (⇐) Assuming the homography up to a similarity transformation is given, we can compute back-projected points in the model plane. Note that the quadrangle formed by four intersection points of C1 and C2 actually has four DOF. Since the four independent DOF imposed on the quadrangle are invariant under the similarity transformation, we can obtain four equations in six unknowns, i.e., four effective constraints on six parameters. □

To further describe these four effective constraints, we can divide them into two parts: conics class constraints and additional constraints. Conics class constraints refer to the constraints imposed on the conics class mentioned above. Additional constraints refer to the constraints a pair of coplanar conics still have after removing conics class constraints. Moreover, the number of additional constraints should be equal to the DOF of the quadrangle formed by four common points. Under the minimal conditions, the number of constraints should satisfy the following equation:

$$ conics \:\: class + additional = 4 $$
(9)

Proposition 1 and the above equation actually answer the confusing question why one specific conics pattern can be used to get the homography. For example, any two coplanar circles belong to OPAS conics which have two DOF. Thereby, we have two conics class constraints and two additional constraints (i.e., their eccentricities) which satisfy the above equation about the minimal condition of computing the homography. Two coplanar circles belonging to OPAS conics have two unknown parameters, the displacement dx and the scale factor k. As these two unknowns can be computed by using projective invariants, all parameters and the trapezoid formed by four common points will be obtained. Adding the projective invariants, (9) for coplanar circles can be expressed by

$$ \underbrace{OPAS \:\: conics}_{2} + \underbrace{additional}_{2} + \underbrace{projective}_{2} = 6 $$
(10)

Now, we discuss confocal conics [9] which belong to PAA conics. As PAA conics already has three class constraints and one DOF, it needs to be given one effective and additional constraint to compute the homography. Therefore, the additional constraint for confocal conics is that two conics are confocal. Ying and Zha [7] also propose a condition of computing homography for PAA conics: giving the eccentricity of any one conic. This constraint could also be considered as a specially additional constraint.

For concentric circles, the analysis is a little complicated. Because of belonging to PAA conics, concentric circles belong to the degenerate conics system with double contact which cannot be explained by (9). On the conics system with double contact, interested readers could see [13] for the detailed properties. Here, we only give the conclusion: any two degenerate conics with double contact except concentric circles will result in a one-parameter family of homographies with an uncertain rotation parameter. For concentric circles, although the computed homography also has one unknown rotation parameter, the image of circular points is not influenced as the unknown variable is just in a similarity transformation under which the true homography is transformed into the computed one.

Although this question has been explained in principle, there still exist two problems that need to be solved. One is how to choose the suitable form of invariants to solve two unknown parameters and remove the ambiguous solutions and the other is after calculating four intersection points, how to automatically determine their correspondences in the image to compute the correct homography, especially for the case with complex intersections. These two significant problems will be demonstrated in the next two sections.

4 The ambiguous solutions of unknown parameters

In this section, we focus on how to use the projective invariants to compute two unknown parameters of a pair of coplanar conics. To our knowledge, this idea is first implemented by Gurdjos [10] to compute two unknown parameters for two coplanar circles. Note that coplanar circles belong to OPAS conics, whose configuration is not general as their two parameters are equal to zero. Because of the complexity of two general conics, some new problems arise and Gurdjos’s method could not handle them. Thus, we revisit this problem which actually includes three sub-questions as follows:

  • Geometric invariants involving a pair of conics have various forms. Which form should be chosen to establish the equations of the unknown parameters?

  • Why ambiguous solutions occur?

  • How to remove the wrong solutions?

Gros and Quan [23] reveal that all the mentioned invariant forms in Section 2.2 are related to each other. They also imply that the complexities of the four invariant forms are different. Owing to that work, the first two questions can be answered. The removing method based on the reasonability of the intrinsic parameters is depicted in the last subsection.

4.1 Computing unknown parameters using the real invariant form

For the general conics C1 and C2 in (8), it is impossible to obtain their cross ratios and the generalized eigenvalues by using Matlab’s symbolic computation. Moreover, the invariant form of the trace requires normalizing the determinants of two matrices, which will result in the fractional power functions needing to be further changed. Therefore, we finally choose the real invariants to compute the two unknown parameters. After some simple calculations, we have

$$ {}I_{1} = \frac{\left(be^{2} - aef + 4bc + 4cd + f^{2} + 4d \right) \left(4b - a^{2} \right)} {\left(4bc -a^{2} + 4d + 4b \right)^{2}} $$
(11)
$$ {}I_{2} = \frac{\left(4bc-a^{2} + 4d + 4b \right) \left(cf^{2} + de^{2} + 4cd \right)} {\left(be^{2} - aef + 4bc + 4cd + f^{2} + 4d \right)^{2}} $$
(12)

For different combinations of unknowns, the polynomial functions have different degrees and solutions. For example, if a, b, e, f are given, the two polynomials functions are expressed by:

$$ \frac{(\square cd+\square d+\square c+\square)\square}{(\square c+\square d+\square)^{2}} = \tilde{I}_{1} $$
(13)
$$ \frac{(\square c + \square d + \square)(\square cd + \square d + \square c)}{(\square cd + \square d + \square c + \square)^{2}} = \tilde{I_{2}} $$
(14)

where \(\square \) denotes a real number. Substituting \((\square cd + \square d + \square c + \square)\) in (13) to (14), we have

$$ \frac{(\square cd+\square d+\square c)}{(\square c+\square d+\square)^{3}} = \tilde{I_{2}}/{\tilde{I}_{1}}^{2} $$
(15)

It is obvious that (13) is a polynomial function with 2 degree and (15)’s degree is 3. Therefore, the maximum number of solutions of the unknowns c and d is 6. Do the similar manipulations and all results about degrees of the polynomials (DoP), and the maximum number of solutions (MNoS) are shown in Table 1.

Table 1 DoP and MNoS for all combinations of two unknowns

Here, we do not pay more attention to obtain the exact number of solutions of the polynomial equations for each combination of two unknowns, because we experimentally find that Matlab can directly solve these polynomial equations in two variables, which allow us not to care about the solving details. Notably, when some parameters equal to zero (corresponding to the five non-general classes mentioned in Section 3), the above polynomial equations will be further simplified and the number of solutions becomes smaller.

4.2 Reasons of the invariant ambiguity

It is easy to verify that the projective invariance is only a necessary condition, not a sufficient condition for a pair of plane conics. In other words, different pairs of coplanar conics may have the same projective invariants that will result in the ambiguity of solving unknown parameters. Now, we will explain the underlying reasons of the ambiguous solutions in geometry and algebra.

Consider the projective property that two general conics can be transformed into two PAA conics under a projective transformation (the direct conclusion in [24, pp.157]). The equations of two PAA conics in the model plane up to a global scale factor are:

$$ x^{2} + by^{2} - 1 =0, \quad cx^{2} + dy^{2} - 1=0 $$
(16)

The generalized eigenvalues of two PAA conics are:

$$ \lambda_{1} =1/c, \quad \lambda_{2}=b/d, \quad \lambda_{3}=1 $$
(17)

There is also a twofold ambiguity caused by the confusion of λ1 and λ2. As pointed out in [23], two independent invariants, λ1/λ3 and λ2/λ3, can be interpreted geometrically by the above two cross ratios. Therefore these two kinds of invariant forms arise from the uncertainty of correspondence in geometry.

The real invariants are related to the invariants of the generalized eigenvalues by the following equations ([23, pp.23]):

$$ I_{1} = \frac{\lambda_{3}/\lambda_{1}+\lambda_{3}/\lambda_{2}+\lambda_{3}^{2}/\lambda_{1}\lambda_{2}} {(1+\lambda_{3}/\lambda_{1}+\lambda_{3}/\lambda_{2})^{2}} $$
(18)
$$ I_{2} = \frac{\lambda_{1}\lambda_{2}+\lambda_{3}\lambda_{1}^{2}/\lambda_{2}+\lambda_{3}\lambda_{2}^{2}/\lambda_{1}} {(\lambda_{1}+\lambda_{2}+\lambda_{3})^{2}} $$
(19)

Note that in the above expressions, λ1 and λ2 can be interchanged, which implies the real invariants conceal the geometrical ambiguity. Moreover, the real invariants also bring some new ambiguous solutions because of the increase of polynomial complexity. Therefore, the ambiguity produced in solving real invariant equations can be seen as a purely algebraic problem.

4.3 Ambiguity removal

Because of the invariant ambiguity, the conics whose configuration is wrong could be projected to the image of the correct conics. Thus, it is impossible to remove them in theory. In other words, for one image of two partially unknown conics, some ambiguous solutions are projectively correct. However, the principle of removing the invariant ambiguity is the same as the principle of removing the correspondence ambiguity described in Section 2.3. Given a homography, the Centre Circle which the camera center should lie in can be used to evaluate the reasonability of the homography.

5 Computation of homography between totally known conics and their images

After obtaining all parameters of the pair of conics, the four intersection points of two conics on the model plane and their projections can be computed. But consequently, the problem of how to find the correct correspondences arises. For real intersection points, people may manually select the correct correspondences in the image as the popular calibration tool does [25]. For complex intersection points without physical position, it is impossible to directly find their correspondences in the image.

Rothwell et al. [5] point out that there are 24 ways to match 4 image points to 4 object points. For real intersection points and complex intersection points, the possibilities can be reduced from 24 to 4 and 8 respectively. Then, they used several posterior rules to eliminate this ambiguity. The similar argument is given in [1]. However, these methods are only effective for estimating the pose as the intrinsic parameters must be given. If we use N images of object to calibrate the camera, there will be 8N combinations to compute the intrinsic parameters. It is impossible to verify them in terms of such posterior rules. Although [6, 10] propose some methods to distinguish the circular points from the four complex intersections of two circles, it is still far away from the final solution to the general case.

In this section, we pay more attention to analyze the reasons of correspondence ambiguity and present the possibilities of correspondences (PoC) in theory for different cases of conics. In this way, the number of PoC we must deal with decreases obviously. Then, based on the Centre Circle constraint (as described in Section 2.3) which provides the geometrical explanation of the world-to-image homography in perspective transformation, we set a cuboid bound for the optical center to remove wrong correspondences, no matter whether the intersections are complex or real.

5.1 Possibilities of correspondences for different cases

Two coplanar conics with four distinguished intersection points could be divided into four cases: (1) quasi-enclosing conics with four complex intersections, (2) quasi-separate conics with four complex intersections, (3) intersecting conics with four real intersections, and (4) intersecting conics with two complex and two real intersections. Here, the terms “quasi-enclosing” and “quasi-separate” arising from the geometric properties of two conics will be explained later. We start our analysis from the cases with four complex intersections (including (1) and (2) cases above) and the other two cases will be demonstrated last. Denote these four intersection points by A, B, C, D and their correct image points by \(\tilde {A}\), \(\tilde {B}\), \(\tilde {C}\), \(\tilde {D}\). The PoC for four complex points mapping to image points are given by [5]

$$ \begin{aligned} \left[ \begin{array}{c} \tilde{A}\\ \tilde{B}\\ \tilde{C}\\ \tilde{D} \end{array} \right] \rightarrow &\left[ \begin{array}{c} A\\ B\\ C\\ D \end{array} \right] \left[ \begin{array}{c} A\\ B\\ D\\ C \end{array} \right] \left[ \begin{array}{c} B\\ A\\ C\\ D \end{array} \right] \left[ \begin{array}{c} B\\ A\\ D\\ C \end{array} \right] \\ &\left[ \begin{array}{c} C\\ D\\ A\\ B \end{array} \right] \left[ \begin{array}{c} C\\ D\\ B\\ A \end{array} \right] \left[ \begin{array}{c} D\\ C\\ A\\ B \end{array} \right] \left[ \begin{array}{c} D\\ C\\ B\\ A \end{array} \right] \end{aligned} $$
(20)

where either {A,B} or {C,D} is a pair of conjugate complex points. The above possibilities are obtained by only using the projective property that all complex conjugate pairs should be projected to conjugate pairs, i.e., the real lines which the complex points lie in still correspond to real lines. Given all possibilities of the numbers 1–8 from left to right for analyzing convenience, we find that the generation of these eight possibilities can be concluded by three different states as follows.

  • The correspondences of real lines. If the correspondences of two real lines in the quadrangle are clear, we can distinguish the PoC 1–4 from the PoC 5–8.

  • The adjacency of four complex intersections. Assuming that there indeed exists the adjacent relationship between the complex points no matter their order is clockwise or anti-clockwise, the PoC 1,4,5,8 can be distinguished from the PoC 2,3,6,7 if this adjacency is known.

  • The order of points around the conic. The PoC 1,3,5,7 and 2,4,6,8 can be respectively treated as the clockwise order and the anti-clockwise order.

Next, we will show how to utilize these three states to further reduce the number of PoC.

5.2 The adjacency of four complex points

Here, we will show that the adjacency of four complex intersection points can be found. Let us turn back to observe the geometric relationships of the separate conics and the enclosing conics both of which have four complex intersections (see Fig. 4). The quadrangle ABCD has two real lines AB and CD. Two complex lines AD and BC intersect at the real vertex Z of the self-polar triangle XYZ. The complex lines BD and AC intersect at the real vertex Y. Assume the two conics are in visible half-space segmented by the principal plane through the camera center parallel to the image plane (e.g., two ellipses). Thus, their four complex intersections also should be in visible half-space and the possible points lying in invisible half-space are X and Z.Footnote 1 Because the real point Y always lies inside one conic, the line AC and BD intersect at Y both in the world plane and in the image. This implies that A should be adjacent to D, not to C. Therefore, as the position of the vertex Y intersected by two complex conjugate lines is projectively invariant, we obtain the adjacency of four complex intersections. Consequently, the number of PoC for these two cases is reduced to 4.

5.3 The correspondences of two real lines

The correspondence problem of two real lines is previously investigated in [6, 10]. Specifically, they propose to divide two coplanar circles with four complex intersections into two cases: the separate case and the enclosing case. Note that the terms “separate” and “enclosing” are visually suitable for a pair of circles or ellipses since they are closed and finite in Euclidean geometry. However, to the visually infinite conics, some adjustment should be given. Thus, we slightly extend the previous works to define the term “quasi-separate” and “quasi-enclosing” to cover all conics cases with four complex.

Definition 1

For two coplanar conics C1 and C2 with four distinguished complex intersections, let the two real lines consisted of the two pairs of complex conjugate points be l1 and l2. We name

  1. (i)

    C1 and C2 are quasi-separate if and only if they lie on the adjacent regions divided by l1 and l2.

  2. (ii)

    C1 and C2 are quasi-enclosing if and only if they lie on the same region or the opposite regions divided by l1 and l2.

Figure 5 gives some examples of the quasi-separate and the quasi-enclosing conics. For example, Fig. 5e depicts an ellipse and a half hyperbola. Their four imaginary intersections lie on two real lines, only one of which separates two conics. Thus, the correspondences of two real lines can be distinguished. The correct PoC only ranges from 1 to 4 or 5 to 8. Furthermore, as Wu proves for a pair of circles in [6], the above relationships are invariant under a quasi-affine transformation [26]. In other words, the real line between two conics can be distinguished for the quasi-separate case. Thus, the number of PoC for the quasi-separate conics is reduced to 2 (see Table 2).

Fig. 5
figure 5

Most of the quasi-separate and quasi-enclosing cases for two conics and their associated lines. a, c, and e depict the quasi-separate ellipses, hyperbolas, and an ellipse and a hyperbola respectively. The dashed conic indicates the position of a half hyperbola and only the other half hyperbola can be seen. b, d, and f depict the quasi-enclosing ellipses, hyperbolas, and an ellipse and a hyperbola respectively

Table 2 PoC for two cases with four complex intersections

After analyzing the cases with four complex intersections, it is not difficult for us to obtain the PoC for the other two cases. For the case with two real and two complex intersections, as both the adjacency of four points and the real line passing through two real points can be distinguished, the correspondences only have two possibilities (see Table 3). For the real case, it is easy to verify that the PoC B, C, D, A and D, A, B, C will result in the wrong correspondences of the self-polar triangle XYZ followed by the wrong re-projection image (see Fig. 4) Therefore, the number of PoC for the real case is finally reduced to 2.

Table 3 PoC for two cases with real intersections

5.4 The order of four points and geometric symmetry

The notion of adjacency or order is not existing in projective geometry as any complex point does not have the physical position. Thus, the order of four complex points cannot be distinguished and the PoC that arose from the order must be considered in general cases. Tables 2 and 3 together show the PoC of four intersection points in four general conics cases. Now, we will reveal the relationship between PoC and the geometric symmetry of the quadrangle ABCD.

The influence of centrosymmetric to PoC is obvious. When the quadrangle ABCD is centrosymmetric, the PoC ABCD and CDAB will generate the same homographies no matter the intersections are complex or real. As a result, we can pick up 2 PoC out of 4 to compute the homography.

When the quadrangle is axisymmetric, such as the trapezoid formed by two OPAS conics (see Fig. 2), the situation is a little complicated. Axisymmetry which lets the PoC ABCD and BADC generate the same homography can only influence the three cases without four real intersections. For the four-real-intersecting case, there is still 2 PoC needed to be considered.

The rectangle formed by PAA conics is both centrosymmetric and axisymmetric. Thus, for two enclosing PAA conics, all 4 PoC will generate the same homographies. It is also worth noting that, for a geometrically symmetric quadrangle, the number of the PoC is reduced at the cost of the ambiguous extrinsic parameters.

5.5 Summary

After analyzing the three states which influence the PoC, the correspondence ambiguity of four intersections could be concluded in three points as follows:

  • The adjacency of four intersections for all cases of two coplanar conics can always be distinguished.

  • The correspondences of real lines can be distinguished for the quasi-separate case and the two-real-intersecting case.

  • For the geometrically symmetric conics, the number of PoC often can be further reduced.

Compared to the previous works, the number of the PoC we must deal with in theory declines obviously. For the residual PoC, we adopt the two steps described in Section 2.3 to judge whether the Centre Circle corresponding to one computed homography intersects with the cuboid bound around the optical center. Our experiments will verify this method can remove the invariant ambiguity and the correspondence ambiguity simultaneously.

The pseudo code of whole calibration method for two coplanar conics is illustrated in Algorithm 1.

5.6 Degenerate situations

After discussing the general case of two conics, we now analyze in which conditions the degenerate cases will occur. There are four types of degenerate conics systems which are the simple-contact system, the three-point contact system, the system with double contact, and the system with four-point contact [24, pp.158-160]. As the degenerate situations rarely happen in practice, we only give the conclusion about whether the degenerate conics can be used to compute a homography.

For the simple-contact conics system (see Fig. 6a), two common tangents l1 and l2 are coincident. But four points intersected by two conics and the other common tangents l3 and l4 respectively can be always obtained. In the similar way, for the three-point contact system (see Fig. 6b), two common points and two points on the tangent l4 can be obtained. Thus, for the above two degenerate cases, the complete homography can be computed.

Fig. 6
figure 6

Four degenerate cases. a the simple-contact system; b the three-point contact system; c the system with double contact; d the system with four-point contact

For the system with double contact (see Fig. 6c), only three points including two common points and the point intersected by the tangent l3 and l4 can be distinguished from the object plane and its image. The undetermined thing is the correspondence of another general point P of one conic which has one DOF. As a result, a one-parameter family of homographies will map this degenerate pattern to its image (interested readers could see the detailed derivation in [13]). Similarly, for the system with four-point contact, any line t passing through the common point has one DOF. The points M and N as the intersections of conics and t have one common unknown. Subsequently, the other two points intersected by the common tangent and the tangents passing through M and N still have one unknown. As a result, an incomplete homography with one parameter can be obtained.

6 Experiments

6.1 Computer simulations

In our experiments, the simulated camera has the following intrinsic parameters: fu = 1200, fv = 1190, u0 = 500, and v0 = 400. The image resolution is 1024 × 768. The model plane is composed of a hyperbola and six ellipses as shown in Fig. 7, which actually covers all the cases in Table 4. C1, C2, C3, and C4 are four concentric ellipses centered at (0,0). The centers of C5, C6, and C7 are (80,0), (90,20), and (− 120,20) respectively. The radii of all conics are not listed here as the combination relationship between two of them is clear in Fig. 7. Project the pattern (approximately 500 points evenly extracted from each ellipse) to the simulated image planes at several different positions. The conic fitting algorithm presented in [27] is used to obtain the equations of the projected conics.

Fig. 7
figure 7

Simulated seven conics in the model plane

Table 4 Number of the residual PoC in theory. The symbol “ ⊗” denotes that this situation does not exist

Performance w.r.t the invariant ambiguity We first pick up several pairs of conics to compute the unknown parameters. The results of the number of their solutions are shown in Table 5. Note that all the numbers of the solutions are equal or less than the MNoS shown in Table 1.

Table 5 The two unknown parameters and the number of their solutions

For the general conics pair {C1,C6}, the solutions of {c,d} are shown in Table 6. After removing the last two complex solutions, the conics corresponding to the other four solutions of unknown parameters are depicted in Fig. 8. The first solution is correct. Both the third and the fourth solutions do not satisfy the perspective relationship. The second one which we have to deal with is meaningfully wrong. We set the region parameter of the principal point and the range parameter of the focal length 50 and 20% respectively. Using these two different solutions to compute the homographies, there is only one correct Centre Circle passing through the cuboid. Therefore, the correct unknowns, PoC, and homography can be obtained. Detailed results are shown in Fig. 9 and Table 7.

Fig. 8
figure 8

Conics corresponding to four real solutions of {c,d}

Fig. 9
figure 9

Centre Lines computed from 4 PoC for { C1, C6} and the square region of the principal point. a is depicted with the correct parameters while b with the wrong parameters

Table 6 Solutions of {c,d} for the conics pair {C1,C6}
Table 7 Range of the focal length computed from four PoC for each solution of {c,d}

Performance w.r.t the correspondence ambiguity Assume the conics are totally known. For all the PoC shown in Tables 2 and 3, the number of the corresponding homographies satisfying the cuboid constraint is shown in Table 8.

Table 8 The number of the computed homographies satisfying the cuboid constraint

The above results verify the correctness of Table 4 and our propositions. For instance, five pairs of conics {C2,C3}, {C6,C7}, {C1,C2}, {C2,C6}, and {C1,C7} are quasi-enclosing and belong to PAA, OPAS, concentric, PAP, and general classes of conics respectively. In the original eight PoC of four complex points, only the projections of the conics from the residual four PoC in Table 7 are correct. The example of the pair of conics {C1,C7} is shown in Fig. 10. Then, we apply the cuboid constraint to verify the residual four PoC. For the geometrically asymmetrical conics {C2,C6} and {C1,C7}, only one homography from the correct PoC satisfies the cuboid constraint. For the geometrically symmetrical conics {C2,C3}, {C6,C7}, and {C1,C2}. The homographies obtained from two or four PoC of four complex intersections are same and their Centre Circles all pass through the cuboid at the cost of the ambiguous extrinsic parameters. Thus, we can choose any one PoC to compute the correct homography.

Fig. 10
figure 10

The correct image (blue and postfix is “-i”) of { C1, C7} and their wrong re-projection (red and postfix is “-p”) from the PoC 2, 3, 6, 7 in Eq. 20

Performance w.r.t the noise level The Gaussian noise with mean 0 and standard deviation ranging from 0.3 to 1.5 is added to the projected image points. We perform 100 times independent experiments to solve camera calibration with three pairs of conics { C1,C6}, { C2,C5}, and { C3,C4}. The averaged results are shown in Fig. 11. Note that the largest pair of conics { C2,C5} has the least errors, which implies that the larger the size of conics, the better the anti-noise performance.

Fig. 11
figure 11

Performance of different intrinsic parameters vs. the noise level of the image points

6.2 Real images

Complete camera calibration The first real experiment is run to compute the complete intrinsic parameters using the proposed algorithm with the following setup:

  • Print a model plane with two general ellipses and a parabola as shown in Fig. 12. The left ellipse (denoted by E1) can be obtained by rotating the ellipse x2/16+y2/9=1 30° anti-clockwise. The equation of the right ellipse (denoted by E2) is (x−8)2/1+(y−6)2/4=1. The equation of the parabola (denoted by P1) is \(y-5=\frac {3}{7}\left (x-3\right)^{2}\).

    Fig. 12
    figure 12

    a One real image of the model plane. b Its rectified metric image

  • Use a CCD camera (Point Grey FL2-08S2M-C) with 4-mm lens (uTron FV0420) to take 12 photos of the model plane. The image resolution is 1024×768.

  • Use Canny operator to detect the edge and Fitzgibbon’s conic fitting method [27] to obtain the conics.

First, we compute the unknown translation parameters dx and dy for two conics pairs { E1,E2} and { E1,P1} respectively. Here, four priori conditions that include the eccentricities of two conics, the relative rotation, and the relative scale factor between two conics are given. The two pairs of real solutions for two conics pairs are listed in Table 9. Note that the first group of solution is correct while the second one is wrong. The mean value of every solution is very close to its theoretical value.

Table 9 Two solutions of the translation parameters

We then applied the cuboid constraint with the size parameters 50 and 20% to remove the two kinds of ambiguities in the process of computing homography. For each solution of the unknown parameters of the conics pair { E1,E2}, the computed Centre Lines from four PoC are shown in Fig. 13. Note that for each case, only two Centre Lines pass through the given square region of the principal point. Table 10 further shows the ranges of the focal length computed from the above two PoC. Finally, only one PoC obtained from the correct parameters satisfies the cuboid constraint.

Fig. 13
figure 13

Centre Lines computed from four PoC and the square region of the principal point. a is depicted with the correct parameters while b with the wrong parameters

Table 10 Range of the focal length computed from two PoC for the correct and wrong parameters

After obtaining the homography for each image, the intrinsic parameters are computed. To verify the accuracy of our method, we also use the Camera Calibration Toolbox [25] to calibrate the same camera. In addition, to robustly extract the chessboard corner, we refer to the literature [28] which shows the better performance under severe distortion or illumination. The results are shown in Table 11. The metric rectification result of the depicted image is shown in Fig. 12b.

Table 11 Comparison of three results

Focal length calibration and AR The second experiment focuses on a specific AR application in practice.

There is a round box on a table. Put a printed ellipse on anyplace of this table (see Fig. 14). Thus, an ellipse and a coplanar circle form PAP conics (shown in Fig. 3) on the table plane. The semi-major axis and the semi-minor axis of the ellipse is 4 cm and 2 cm, respectively. The radius of the bottom circle of the box is 3.25 cm. With these four known parameters, we can compute the unknown translations between the centers of two conics, followed by the homography between the table plane and one image. Three images captured by an iPhone 8 are shown in Fig. 14. Note that the ellipse positions in three images are different. We still use the above method to extract the conics (colored in pink and cyan respectively) in the image. Assuming the principal point is given on the image center, here we only calibrate the focal length using the computed homography from each image individually. This is because the slight offset of the principal point to image center has very little impact on the results of focal length calibration and AR effects. The calibration results from three images and the Camera Calibration Toolbox are shown in Table 12. After obtaining the intrinsic parameters, the pose of every ellipse can be estimated and a virtual teapot with its local coordinate system are drawn. Constructing a moving AR marker with an ellipse shows the practical use of the proposed method, even if under the case that the camera has a zoom lens.

Fig. 14
figure 14

Three images with the added AR teapot

Table 12 Comparison of the focal length computed by our method and the Camera Calibration Toolbox

7 Conclusions

In this paper, a two-step method is developed to give a unified explanation of camera calibration based on coplanar conics. We also display a particular description of the invariant and correspondence ambiguity problems existing in the proposed method. Based on the analysis of the solutions of residual ambiguity in various cases, the wrong homographies can be removed by setting a cuboid bound for the camera center for each image and we can directly obtain the unambiguous calibration results. Furthermore, as the natural link between various conics patterns has been established, readers could get a full comprehension of the conics-based calibration.

8 Appendix: Proof of Fig. 3

In this appendix, we use analytic geometry knowledge to verify the shape of the quadrangle formed by four intersection points of two conics for every case shown in Fig. 3.

See Fig. 15. C1 and C2 are two PAP conics whose axes are parallel. A, B, C, and D are four intersection points. Now, we will prove D is on the circumcircle defined by the points A, B, and C.

Fig. 15
figure 15

Two PAP conics and the circle passing through their four intersection points

The isogonal conjugate of the circumconic C1 with respect to the triangle ABC is the line l1. Meanwhile, the isogonal conjugate of C2 with respect to the triangle ABC is the line l2. For a given triangle, the direction of the isogonal conjugate line of a circumconic is only related to the direction of the axes of the circumconic. Therefore, l1 and l2 are parallel. The isogonal conjugate of the point D as a common point of C1 and C2 will be the point at infinity intersected by l1 and l2. Moreover, as the isogonal conjugate of the circumcircle with respect to the triangle ABC is the line at infinity, D must be on the circumcircle.

For OPAS conics, as both of them are axisymmetric, the four intersections must be axisymmetric. Thus, the quadrangle formed by these four intersections can only be a trapezoid. In the similar way, two concentric conics are centrosymmetric, so are their intersections. Thus, the quadrangle can only be a parallelogram. For PAA conics, the quadrangle should be axisymmetric and centrosymmetric simultaneously. Thus, it is a rectangle.