Abstract
Current stereo algorithms are capable to calculate accurate (as defined, e.g., by needs in vision-based driver assistance) dense disparity maps in real time. They have become the source of three-dimensional data for several indoor and outdoor applications. However, ground truth-based evaluation of such algorithms has been typically limited to data sets generated indoors in laboratories. In this paper we present a new approach to evaluate stereo algorithms using ground-truth over real world data sets. Ground truth is generated using range measurements acquired with a high-end laser range-finder. For evaluating as many points as possible in a given disparity map, we use two evaluation approaches: A direct comparison for those pixels with available range data, and a confidence measure for the remaining pixels.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Banks, J., Corke, P.: Quantitative evaluation of matching methods and validity measures for stereo vision. Int. J. Robotics Research 20, 512–532 (2001)
Eid, A., Farag, A.: A unified framework for performance evaluation of 3-d reconstruction techniques. In: Proc. Comp. Vision Pattern Recognition Workshop, vol. 3, pp. 33–41 (2004)
Egnal, G., Mintz, M., Wildes, R.: A stereo confidence metric using single view imagery with comparison to five alternative approaches. Image Vision Computing 22, 943–957 (2004)
.enpeda. Group, University of Auckland: EISATS (.enpeda. sequence analysis test site) (2010), http://www.mi.auckland.ac.nz/EISATS
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Comp. Vision 70, 41–54 (2006)
Gherardi, R.: Confidence-based cost modulation for stereo matching. In: Proc. ICPR, pp. 1–4 (2008)
Guan, S., Klette, R., Woo, Y.W.: Belief propagation for stereo analysis of night-vision sequences. In: PSIVT 2009. LNCS, vol. 5414, pp. 932–943 (2009)
Haeusler, R., Klette, R.: Benchmarking stereo data (Not the matching algorithms). In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) Pattern Recognition. LNCS, vol. 6376, pp. 383–392. Springer, Heidelberg (2010)
Hirschmüller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: Proc. CVPR, pp. 807–814 (2005)
Huang, F., Klette, R., Scheibe, K.: Panoramic Imaging: Sensor-Line Cameras and Laser Range-Finders. Wiley, Chichester (2008)
Klette, R., Vaudrey, T., Wiest, J., Haeusler, R., Jiang, R., Morales, S.: Current challenges in vision-based driver assistance. In: Progress in Combinat, pp. 3–32. Image Analysis, Research Publ. Services, Singapore (2010)
Kolmogorov, V., Zabih, R.: Multi-camera scene reconstruction via graph cuts. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 82–96. Springer, Heidelberg (2002)
Lepetit, V., Moreno-Noguer, F., Fua, P.: Accurate non-iterative o(n)solution to the PNP problem. In: Proc. ICCV, pp. 874–885 (2007)
Liu, Z., Klette, R.: Approximate ground truth for stereo and motion analysis on real-world sequences. In: PSIVT 2009. LNCS, vol. 5414, pp. 874–885 (2009)
Morales, S., Klette, R.: A third eye for performance evaluation in stereo sequence analysis. In: Jiang, X., Petkov, N. (eds.) CAIP 2009. LNCS, vol. 5702, pp. 1078–1086. Springer, Heidelberg (2009)
Mordohai, P.: The self-aware matching measure for stereo. In: Proc. ICCV, pp. 1841–1848 (2009)
Murray, D., Little, J.J.: Using real-time stereo vision for mobile robot navigation. Aut. Robots 8, 161–171 (2000)
Ohta, Y., Kanade, T.: Stereo by two-level dynamic programming. In: Proc. Int. Joint Conf. Artificial Int., pp. 1120–1126 (1985)
Reulke, R., Luber, A., Haberjahn, M., Piltz, B.: Validierung von mobilen Stereokamerasystemen in einem 3D-Testfeld. In: Proc. 3D-NordOst (2009)
Satoh, Y., Sakaue, K.: An omnidirectional stereo vision-based smart wheelchair. J. Image Video Processing 2007, 1–11 (2007)
Seelinger, M., Yoder, J.D.: Automatic pallet engagement by a vision guided forklift. In: Proc. IEEE Int. Conf. Robotics Automation, pp. 4068–4073 (2005)
Steingrube, P., Gehrig, S., Franke, U.: Performance evaluation of stereo algorithms for automotive applications. In: Proc. Int. Conf. Comp. Vision Systems, pp. 285–394 (2009)
Szeliski, R.: Prediction error as a quality metric for motion and stereo. In: Proc. ICCV, pp. 781–788 (1999)
Velodyne Lidar Inc.: Velodyne’s HDL-64E S2 user manual, http://www.velodyne.com/lidar/hdlproducts/hdl64e.aspx
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Morales, S., Klette, R. (2011). Ground Truth Evaluation of Stereo Algorithms for Real World Applications. In: Koch, R., Huang, F. (eds) Computer Vision – ACCV 2010 Workshops. ACCV 2010. Lecture Notes in Computer Science, vol 6469. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22819-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-22819-3_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22818-6
Online ISBN: 978-3-642-22819-3
eBook Packages: Computer ScienceComputer Science (R0)