A Comprehensive Approach for Camera/LIDAR Frame Alignment

Abstract

This paper deals with the initial phase of any multi-sensor acquisition, the sensor alignment process. We focus on extrinsic calibration of vision-based and line scan LIDAR sensors for automotive application. We investigate two research approaches for the calibration process: analytical and numeric solutions. Additionally, we present a complete implementable tool-chain, to extract the co-features for both types of sensors: line detections for cameras and segmentation process for LIDAR sensors. Moreover, we investigate the impact of the calibration accuracy on sensor fusion performance. Results of experiments using real-world data are presented.

Appendices

APPENDIX A

- The quaternion is generally defined as:

$$\begin{aligned} \bar{q}=q_{4}+q_{1}i+q_{2}j+q_{3}k \end{aligned}$$

(17)

where i, j, and k are hyper-imaginary numbers and the quantity $q_{4}$ is the real or scalar part of the quaternion.

- To convert a 3D vector $\overrightarrow{p}$ to quaternion form we use

$$\begin{aligned} \bar{p}=[\overrightarrow{p}0]^{T} \end{aligned}$$

(18)

-The product $\overrightarrow{p}_{2}= R \overrightarrow{p}_{1}$ where $\overrightarrow{p}_{1}$, $\overrightarrow{p}_{2}$ are vectors, R is the rotation matrix and q its quaternion equivalent, can be written as follow

$$\begin{aligned} \bar{p}_{2}=q\otimes \bar{p}_{1}\otimes q^{-1} \end{aligned}$$

(19)

where $\otimes$ presents quaternion multiplication, $q^{-1}$ is the quaternion inverse defined as $q^{-1}=[-q_{1}\,-q_{2}\,-q_{3}\,\,q_{4}]^{T}$.

-For any quaternions $q_{a}$ and $q_{b}$, the product, $q_{a}\otimes q_{b}$ is defined as

$$\begin{aligned} q_{a}\otimes q_{b}\triangleq \mathcal {L}(q_{a})q_{b}=\mathcal {R}(q_{b})q_{a} \end{aligned}$$

(20)

Where:

$$\begin{aligned} \mathcal {L}(q)=\begin{bmatrix}q_{4}&-q_{3}&q_{2}&q_{1}\\ q_{3}&q_{4}&-q_{1}&q_{2}\\ -q_{2}&q_{1}&q_{4}&q_{3}\\ -q_{1}&-q_{2}&-q_{3}&q_{4} \end{bmatrix}\,\,\,\,\,\,\,\,\,\mathcal {R}(q)=\begin{bmatrix}q_{4}&q_{3}&-q_{2}&q_{1}\\ -q_{3}&q_{4}&q_{1}&q_{2}\\ q_{2}&-q_{1}&q_{4}&q_{3}\\ -q_{1}&-q_{2}&-q_{3}&q_{4} \end{bmatrix} \end{aligned}$$

Also we have following properties

$$\begin{aligned} \begin{aligned}&\mathcal {L}(q_{a})\mathcal {R}(q_{b})=\mathcal {R}(q_{b})\mathcal {L}(q_{a})\\&q_{b}^{T}\mathcal {L}(q_{a})^{T}=q_{b}^{T}\mathcal {R}(q_{a})^{T}\\&\mathcal {L}(q^{-1})=\mathcal {L}(q)^{T}\\&\mathcal {R}(q^{-1})=\mathcal {R}(q)^{T} \end{aligned} \end{aligned}$$

For more details, the interested reader is referred to [29].

APPENDIX B

In this appendix, we present an example to solve a polynomial system by computing the multiplication matrix based on the normal set approach.

Suppose that we have the following polynomial system:

$$\begin{aligned} f_{1} = x^2 y + x + 2 \end{aligned}$$

(21)

$$\begin{aligned} f_{2} = y^2 x+ y+ 6 \end{aligned}$$

(22)

The degrees of the above equations are:

$$\begin{aligned} degree(f_{1}) = 3 \qquad degree(f_{2})=3 \end{aligned}$$

(23)

By using the graded reverse lex ordering [12], the $Gr\ddot{o}bner$ basis of this polynomial system is:

$$\begin{aligned} g_{1} = 3x - y \end{aligned}$$

(24)

$$\begin{aligned} g_{2} = y^3+3y+18 \end{aligned}$$

(25)

Therefore the normal set is:

$$\begin{aligned} NormalSet <= \left\{ 1,y,y^2 \right\} \end{aligned}$$

(26)

Without loss of generality, we have to determine the multiplication matrix in order to solve the polynomial system and find the roots. So, with some algebraic manipulations, the multiplication matrix associated to x is:

$$\begin{aligned} M_{x}= \begin{bmatrix} 0&\dfrac{1}{3}&0 \\ 0&0&\dfrac{1}{3} \\ -6&-1&0 \end{bmatrix} \end{aligned}$$

(27)

and the multiplication matrix associated to y is:

$$\begin{aligned} M_{y}= \begin{bmatrix} 0&1&0 \\ 0&0&1\\ -18&-3&0 \end{bmatrix} \end{aligned}$$

(28)

Finally, the roots can be computed by the eigenvalue decomposition of the multiplication matrix and hence the possible solutions (real and complex) of the polynomial system (21)–(22) are:

(29)