Autonomous Tracking of Vehicle Taillights and Alert Signal Detection by Embedded Smart Cameras

Abstract

An important aspect of collision avoidance and driver assistance systems, as well as autonomous vehicles, is the tracking of vehicle taillights and the detection of alert signals (turns and brakes). In this chapter, we present the design and implementation of a robust and computationally lightweight algorithm for a real-time vision system, capable of detecting and tracking vehicle taillights, recognizing common alert signals using a vehicle-mounted embedded smart camera, and counting the cars passing on both sides of the vehicle. The system is low-power and processes scenes entirely on the microprocessor of an embedded smart camera. In contrast to most existing work that addresses either daytime or nighttime detection, the presented system provides the ability to track vehicle taillights and detect alert signals regardless of lighting conditions. The mobile vision system has been tested in actual traffic scenes and the obtained results demonstrate the performance and lightweight nature of the algorithm.

Appendix

Below are the matrices used for the Kalman filter.

• $A$: Movement Matrix , which represents how the state of the system changes by drawing a relationship between the current state of the system at time step $k$ to the state of the system at the previous time step $k-1$, Eq. (6.7). A $^T$ represents the transposed movement matrix $A$.

  $$\begin{aligned} A = \left[ \begin{array}{cccc} 1 &{} 0 &{} \varDelta t &{} 0 \\ 0 &{} 1 &{} 0 &{} \varDelta t \\ 0 &{} 0 &{} 1 &{} 0 \\ 0 &{} 0 &{} 0 &{} 1 \end{array}\right] \qquad A^T = \left[ \begin{array}{cccc} 1 &{} 0 &{} 0 &{} 0 \\ 0 &{} 1 &{} 0 &{} 0 \\ \varDelta t &{} 0 &{} 1 &{} 0 \\ 0 &{} \varDelta t &{} 0 &{} 1 \end{array}\right] \end{aligned}$$

  (6.7)

• $H$: Measurement Matrix , representing the dependency of the measurement on the system state, Eq. (6.8). H $^T$ represents the transposed movement matrix $H$.

  $$\begin{aligned} H = \left[ \begin{array}{cc} 1 &{} 0 \\ 0 &{} 1 \\ 0 &{} 0 \\ 0 &{} 0 \end{array}\right] \qquad H^T = \left[ \begin{array}{cccc} 1 &{} 0 &{} 0 &{} 0 \\ 0 &{} 1 &{} 0 &{} 0 \end{array}\right] \end{aligned}$$

  (6.8)

• $R$: Measurement Noise Covariance , (constant) Eq. (6.9).

  $$\begin{aligned} R = \left[ \begin{array}{c} 0.2845 \\ 0.0045 \\ 0.0045 \\ 0.2845 \end{array}\right] \end{aligned}$$

  (6.9)

• $Q$: Process Noise Covariance , (constant) Eq. (6.10).

  $$\begin{aligned} Q = \left[ \begin{array}{cccc} 0.01 &{} 0 &{} 0 &{} 0 \\ 0 &{} 0.01 &{} 0 &{} 0 \\ 0 &{} 0 &{} 0.01 &{} 0 \\ 0 &{} 0 &{} 0 &{} 0.01 \end{array}\right] \end{aligned}$$

  (6.10)

• $P_{k-1}$ (initial estimate): Estimation Error Covariance , Eq. (6.11).

  $$\begin{aligned} P_{k-1} \text {(initial estimate)} = \left[ \begin{array}{cccc} 50 &{} 0 &{} 0 &{} 0 \\ 0 &{} 50 &{} 0 &{} 0 \\ 0 &{} 0 &{} 50 &{} 0 \\ 0 &{} 0 &{} 0 &{} 50 \end{array}\right] \end{aligned}$$

  (6.11)

• $I_4$: $4\times 4$ Identity Matrix , Eq. (6.12).

  $$\begin{aligned} I_4 = \left[ \begin{array}{cccc} 1 &{} 0 &{} 0 &{} 0 \\ 0 &{} 1 &{} 0 &{} 0 \\ 0 &{} 0 &{} 1 &{} 0 \\ 0 &{} 0 &{} 0 &{} 1 \end{array}\right] \end{aligned}$$

  (6.12)