Abstract
This chapter describes the algorithms for calibrating the image acquisition system (normally a single camera or a stereo vision) that are fundamental for detecting metric information (detecting an object’s size or determining accurate measurements of object–observer distance—the pose) of the scene from the images. The various camera calibration methods are presented that determine the relative intrinsic parameters and the extrinsic parameters that define the geometric transformation to pass from the reference system of the world to that of the camera. The epipolar geometry introduced in Chap. 5 is exploited to solve the problem of correspondence of homologous points in a stereo vision system with the cameras calibrated and not. With the epipolar geometry is simplified the search for the homologous points between the stereo images (the correspondence problem) by introducing the Essential matrix (calibrated approach) and the Fundamental matrix (weak calibrated approach). The algorithms for estimating these matrices are also described, known a priori the corresponding points of a calibration platform. This is also accomplished with the image alignment procedure, known as stereo image rectification. Finally, the triangulation procedures for the 3D reconstruction of the geometry of the scene without ambiguity are described, given the 2D projections of the homologous points of the stereo images, given the calibration parameters of the stereo system. If only the intrinsic parameters are known, the 3D geometry of the scene is reconstructed by estimating the extrinsic parameters of the system at less than a non-determinable scale factor. If the calibration parameters of the stereo system are not available but only the correspondences between the stereo images are known, the structure of the scene is recovered through an unknown homography transformation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
By definition, an orthogonal matrix R has the following properties:
-
1.
R is normalized, that is, the sum of the squares of the elements of any row or column is 1 (\(\parallel \mathbf {r}_i\parallel _2=1, i=1,2,3\)).
-
2.
The inner product of any pair of rows or columns is zero (\( \mathbf {r}_1^T \mathbf {r}_3=\mathbf {r}_2^T \mathbf {r}_3=\mathbf {r}_2^T \mathbf {r}_1=0\)).
-
3.
From the first two properties follows the orthonormality of R, or \(RR^T=R^TR= I \), its invertibility, that is, \(R^{-1}=R^T\) (its inverse coincides with the transpose), and the determinant of R has module 1 (\( det (R) = 1 \)), where \( I \) indicates the identity matrix.
-
4.
The rows of R represent the coordinates in the space of origin of unit vectors along the coordinate axes of the rotated space.
-
5.
The columns of R represent the coordinates in the rotated space of unit vectors along the axes of the space of origin. In essence, they represent the directional cosines of the triad axes rotated with respect to the triad of origin.
-
6.
Each row and each column are orthonormal to each other, as they are orthonormal bases of space. It, therefore, follows that given two vectors row or column of the matrix \( \varvec{r} _1 \) and \( \varvec{r} _2 \) it is possible to determine the third base vector as a vector product given by \(\varvec{r}_3=\varvec{r}_1\times \varvec{r}_2\).
.
-
1.
- 2.
Triangle Orthocenter: located at the intersection of the heights of the triangle.
- 3.
The coordinates (u, v) of the points in the image plane in Eq. (7.58) are expressed in homogeneous coordinates while in the nonlinear equations (7.59) are not homogeneous \((u/\lambda ,v/\lambda )\) but for simplicity they remain with the same notation. Once calculated \(\varvec{H} \), from the third equation obtainable from (7.58), we can determine \(\lambda = h_{31} u+ h_{32} v+1\).
- 4.
We know that a 3D point, according to the pinhole model, projected in the image plane in \( \varvec{P} _L \) defines a ray passing through the optical center \( \varvec{C} _L \) (in this case the origin of the stereo system) locus of 3D points aligned represented by \(\lambda \varvec{P}_L\). These points can be observed by the right camera and referenced in its reference system to determine the homologous points using the epipolar geometry approach. We will see that this is possible so that we will neglect the parameter \( \lambda \) in the following.
- 5.
From now on, the perspective projection matrices will be indicated with \(\varvec{{\mathcal {P}}}\) to avoid confusion with the points of the scene indicated with \(\varvec{P}\).
- 6.
With reference to Note 1, we recall that the matrix \(\varvec{R}\) provides the orientation of the camera \(\varvec{C}_R\) with respect to the \(\varvec{C}_L\) one. The column vectors are the direction cosines of \(\varvec{C}_L\) axes rotated with respect to the \(\varvec{C}_R\).
- 7.
A matrix A is said to be antisymmetric when it satisfies the following properties:
$$\begin{aligned}\nonumber \varvec{A}+\varvec{A}^T=\varvec{0}\quad \quad \quad \quad \quad \quad \varvec{A}^T=-\varvec{A} \end{aligned}$$It follows that the elements on the main diagonal are all zeroes while those outside the diagonal satisfy the relation \(a_{ij}=-a_{ji}\). This means that the number of elements is only \( n (n-1) / 2 \) and for \( n = 3 \) we have a matrix of the size \(3\times 3\) of only 3 elements that can be considered as the components of a generic three-dimensional vector \( \varvec{v} \). In this case, we use the symbolism \([\varvec{v}]_{\times }\) or \(\varvec{S}(\varvec{v})\) to indicate the operator that transforms the vector \( \varvec{v } \) in an antisymmetric matrix as reported in (7.103). Often this dual form of representation between vector and antisymmetric matrix is used to write the vector product or outer product between two three-dimensional vectors with the traditional form \(\varvec{x}\times \varvec{y}\) in the form of simple product \([\varvec{x}]_{\times }\varvec{y}\) or \(\varvec{S(x)}\varvec{y}\).
- 8.
With reference to the figure, if we denote by \(\tilde{\varvec{p}}_L=(x_L,y_L,1)\) the projection vector of \(\varvec{P}\) in the left image plane, expressed in homogeneous coordinates, and with \(\varvec{l}_L=(a,b,c)\) the epipolar line expressed as 3D vector whose equation in the image plane is \(ax_L+by_L+c=0\), then the constraint that the point \(\tilde{\varvec{p}}_L\) is on the epipolar line \(\varvec{l}_L\) induces \(\varvec{l}_L\tilde{\varvec{p}}_L=0\) or \(\varvec{l}_L^T\tilde{\varvec{p}}_L=0\) or \(\tilde{\varvec{p}}_L^T\varvec{l}_L=0\). Furthermore, we recall that a line \(\varvec{l}\) passing through two points \(\varvec{p}_1\) and \(\varvec{p}_2\) is given by the following vector product \(\varvec{l}=\varvec{p}_1\times \varvec{p}_2\). Finally, a point \(\varvec{p}\) as the intersection of two lines \(\varvec{l}_1\) and \(\varvec{l}_2\) is given by \(\varvec{p}=\varvec{l}_1\times \varvec{l}_2\).
- 9.
The Frobenius norm is an example of a matrix norm that can be interpreted as the norm of the vector of the elements of a square matrix \( \varvec{A} \) given by
$$\begin{aligned}\nonumber \parallel \mathbf {A}\parallel _F=\sqrt{\sum _{i=1}^{n}\sum _{j=1}^{n}a_{ij}^2}=\sqrt{Tr(\mathbf {A}^T\mathbf {A})}=\sqrt{\sum _{i=1}^{r}\lambda _{i}}=\sqrt{\sum _{i=1}^{r}\sigma ^{2}_{i}} \end{aligned}$$where \(\mathbf {A}\) is the \(n\times n\) square matrix of real elements, \(r\le n\) is the rank of \( \varvec{A} \), \(\lambda _{i}=\sigma ^{2}_{i}\) is the ith nonzero eigenvalues of \( \varvec{A} ^ {T} \varvec{A} \), and \(\sigma _{i}=\sqrt{\lambda _{i}}\) is the ith singular value of the SVD decomposition of \( \varvec{A} \). It should be considered \(Tr(\varvec{A}^{*}\varvec{A})\) with the transposed conjugate \( \varvec{A} ^ {*} \) in the more general case.
- 10.
For the decomposition predicted by (7.140), it must result that the solution of \(\varvec{T}=\varvec{U}[0\ 0\ 1]^T\) since it must satisfy (7.150) that is \(\varvec{S}\varvec{T}=0\) according to the property of an antisymmetric matrix. In fact, for \(\varvec{T}=\varvec{u}_3\) the following condition is satisfied:
$$\begin{aligned}\nonumber \varvec{S}\varvec{u}_3=\scriptstyle \varvec{UZU}^T\varvec{u}_3=\varvec{U} \left| {\begin{matrix} 0 &{} -1 &{} 0\\ 1 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 \end{matrix}}\right| \varvec{U}^T\varvec{u}_3=[\varvec{u}_2\ -\varvec{u}_1\ 0][\varvec{u}_1\ \varvec{u}_2\ \varvec{u}_3]^T\varvec{u}_3=\varvec{u}_2\varvec{u}_1^T\varvec{u}_3-\varvec{u}_1\varvec{u}_2^T\varvec{u}_3=0 \end{aligned}$$.
- 11.
According to epipolar geometry, we know that the epipolar lines intersect in the relative epipoles. Given the noise is present in the the correspondence coordinates, in reality the epipolar lines intersect not in a single point \(\varvec{e}\) but in a small area. Therefore, it is required to optimize the calculation of the position of each epipole considering the center of gravity of this area and this is achieved with the least squares method to minimize this error. Remembering that each line is represented with a 3D vector of the type \(\varvec{l}_i=(a_i,b_i,c_i)\) the set of epipolar lines \(\{\varvec{l}_1,\varvec{l}_2,\ldots ,\varvec{l}_n\}\) can be grouped in a \(n\times 3\) matrix \( \varvec{L} \) and form a homogeneous linear system \(\varvec{L\cdot e=0}\) in the unknown the epipole vector \( \varvec{e} \), solvable with the SVD (singular value decomposition) method.
- 12.
The same equations can be obtained, for each camera, considering the properties of the vector product \(\varvec{p}\times (\varvec{{\mathcal {P}}}\varvec{X})=0\), that is, by imposing the constraint of parallel direction between the vectors. Once the vector product has been developed, three equations are obtained but only two are linearly independent of each other.
References
S.J. Maybank, O.D. Faugeras, A theory of self-calibration of a moving camera. Int. J. Comput. Vis. 8(2), 123–151 (1992)
B. Caprile, V. Torre, Using vanishing points for camera calibration. Int. J. Comput. Vis. 4(2), 127–140 (1990)
R.Y. Tsai, A versatile camera calibration technique for 3d machine vision. IEEE J. Robot. Autom. 4, 323–344 (1987)
J. Heikkila, O. Silvén, A four-step camera calibration procedure with implicit image correction, in IEEE Proceedings of Computer Vision and Pattern Recognition (1997), pp 1106–1112
O.D. Faugeras, G. Toscani, Camera calibration for 3d computer vision, in International Workshop on Machine Vision and Machine Intelligence (1987), pp. 240–247
Z. Zhengyou, A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)
R.K. Lenz, R.Y. Tsai, Techniques for calibration of the scale factor and image center for high accuracy 3-d machine vision metrology. IEEE Trans. Pattern Anal Mach Intell 10(5), 713–720 (1988)
G.H. Golub, C.F. Van Loan, Matrix Computations, 3rd edn. (Johns Hopkins, 1996). ISBN 978-0-8018-5414-9
Z. Zhang, A flexible new technique for camera calibration. Technical Report MSR- TR-98-71 (Microsoft Research, 1998)
R. Hartley, A. Zisserman, Multiple View Geometry in computer vision, 2nd. (Cambridge, 2003)
O. Faugeras, Three-Dimensional Computer Vision: A Geometric Approach (MIT Press, Cambridge, Massachusetts, 1996)
J. Vince, Matrix Transforms for Computer Games and Animation (Springer, 2012)
H.C. Longuet-Higgins, A computer algorithm for reconstructing a scene from two projections. Nature 293, 133–135 (1981)
I. Hartley Richard, In defense of the eight-point algorithm. IEEE Trans. Pattern Recogn. Mach. Intell. 19(6), 580–593 (1997)
Q.-T. Luong, O. Faugeras, The fundamental matrix: theory, algorithms, and stability analysis. Int. J. Comput. Vis. 1(17), 43–76 (1996)
Nistér David, An efficient solution to the five-point relative pose problem. IEEE Trans. Pattern Recogn. Mach. Intell. 26(6), 756–777 (2004)
O. Faugeras, S. Maybank, Motion from point matches : Multiplicity of solutions. Int. J. Comput. Vis. 4, 225–246 (1990)
T.S. Huang, O.D. Faugeras, Some properties of the e matrix in two-view motion estimation. IEEE Trans. Pattern Recogn. Mach. Intell. 11(12), 1310–1312 (1989)
C. Loop, Z. Zhang, Computing rectifying homographies for stereo vision, in IEEE Conference of Computer Vision and Pattern Recognition (1999), vol. 1, pp. 125–131
E. Trucco, A. Verri, Introductory Techniques for 3-D Computer Vision (Prentice Hall, 1998)
R. Mohr, L. Quan, F. Veillon, B. Boufama, Relative 3d reconstruction using multiples uncalibrated images. Technical Report RT 84-I-IMAG LIFIA 12, Lifia-Irimag (1992)
O.D. Faugeras, What can be seen in three dimensions from an uncalibrated stereo rig, in ECCV European Conference on Computer Vision (1992), pp. 563–578
R. Hartley, R. Gupta, T. Chang, Stereo from uncalibrated cameras, in IEEE CVPR Computer Vision and Pattern Recognition (1992), pp. 761–764
R. Mohr, L. Quan, F. Veillon, Relative 3d reconstruction using multiple uncalibrated images. Int. J. Robot. Res. 14(6), 619–632 (1995)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Distante, A., Distante, C. (2020). Camera Calibration and 3D Reconstruction. In: Handbook of Image Processing and Computer Vision. Springer, Cham. https://doi.org/10.1007/978-3-030-42378-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-42378-0_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-42377-3
Online ISBN: 978-3-030-42378-0
eBook Packages: Computer ScienceComputer Science (R0)