Evaluation

Abstract

In this chapter, the performance of the proposed systems is evaluated in terms of accuracy and runtime. The runtimes were evaluated on a 3GHz single core CPU. As it is hardly possible to acquire accurate ground-truth information for real image data, the accuracy of the developed pose estimation methods was examined thoroughly in simulation. For this purpose, the stereo camera system of ARMAR III was simulated in OpenGL with a baseline of 90mm and a focal length of 860 pixels for images with a resolution of 640×480, which is equal to the value measured by calibration for the used 4mm lenses on ARMAR III.

With this simulation, it is possible to evaluate the theoretically achievable absolute accuracy of the developed methods under optimal conditions. When operating on real image data, the accuracy will differ in the millimeter-range, in particular due to imperfect camera calibration and unsharp images resulting from lenses with a fixed focal length. However, evaluation in simulation allows to examine the strengths and the weaknesses of a method rather than the behavior of a system with a specific hardware embodiment.