Real-time motion detection based on SW/HW-codesign for walking rescue robots

Abstract

In a rescue operation walking robots offer a great deal of flexibility in traversing uneven terrain in an uncontrolled environment. For such a rescue robot, each motion is a potential vital sign and the robot should be sensitive enough to detect such motion, at the same time maintaining high accuracy to avoid false alarms. However, the existing techniques for motion detection have severe limitations in dealing with strong levels of ego-motion on walking robots. This paper proposes an optical flow-based method for the detection of moving objects using a single camera mounted on a hexapod robot. The proposed algorithm estimates and compensates ego-motion to allow for object detection from a continuously moving robot, using a first-order-flow motion model. Our algorithm can deal with strong rotation and translation in 3D, with four degrees of freedom. Two alternative object detection methods using a 2D-histogram based vector clustering and motion-compensated frame differencing, respectively, are examined for the detection of slow- and fast-moving objects. The FPGA implementation with optimized resource utilization using SW/HW codesign can process video frames in real-time at 31 fps. The new algorithm offers a significant improvement in performance over the state-of-the-art, under harsh environment and performs equally well under smooth motion.

Appendix: Estimation of motion model from flow vectors

The computation of the first-order-flow motion model from the optical flow vectors is explained below. The first-order-flow motion model in Eq. 1 can be reversed to form a new Eq. 5 which can be used to compute the motion model (x _c , y _c , D, R) from flow vectors, whose velocity and position are represented by (v _x1, v _y1), [x ₁, y ₁] and (v _x2, v _y2), [x ₂, y ₂] respectively.

$$ \begin{aligned} x_c &=\frac{e_1+e_2+e_3}{v_{x1}(v_{x1}-2v_{x2})+v_{x2}^2+v_{y1}^2-v_{y2}(2v_{y1}-v_{y2})}\\ y_c &=\frac{e_4+e_5+e_6}{v_{x1}(v_{x1}-2v_{x2})+v_{x2}^2+v_{y1}^2-v_{y2}(2v_{y1}-v_{y2})}\\ D &= \frac{(x_1-x_2)(v_{x1}-v_{x2})+(y_1-y_2)(v_{y1}-v_{y2})}{x_1(x_1-2x_2)+x_2^2+y_1(y_1-2y_2)+y_2^2}\\ R &= \frac{(x_1-x_2)(v_{y1}-v_{y2})+(y_2-y_1)(v_{x1}-v_{x2})}{x_1(x_1-2x_2)+x_2^2+y_1(y_1-2y_2)+y_2^2}\\ \end{aligned} $$

(5)

where e ₁, to e ₆ are as shown below:

$$ \begin{aligned} e_1 &= v_{x1} (v_{x1}x_2 - v_{x2}x_1 - v_{x2}x_2 + v_{y2}y_1 - v_{y2}y_2)\\ e_2 &= v_{y1} (v_{y1}x_2 - v_{y2}x_1 - v_{y2}x_2 - v_{x2}y_1 + v_{x2}y_2)\\ e_3 &= x_1 (v_{y2}^2 + v_{x2}^2)\\ e_4 &= v_{x2} (v_{y1}x_1 - v_{y1}x_2 - v_{x1} y_1 + v_{x2}y_1 - v_{x1}y_2)\\ e_5 &= v_{y2} (v_{x1}x_2 - v_{x1}x_1 - v_{y1} y_1 + v_{y2}y_1 - v_{y1} y_2)\\ e_6 &= y_2 (v_{x1}^2 + v_{y1}^2)\\ \end{aligned} $$

(6)