Mid-level Segmentation and Segment Tracking for Long-Range Stereo Analysis

  • Simon Hermann
  • Anko Börner
  • Reinhard Klette
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7087)


This paper presents a novel way of combining dense stereo and motion analysis for the purpose of mid-level scene segmentation and object tracking. The input is video data that addresses long-range stereo analysis, as typical when recording traffic scenes from a mobile platform. The task is to identify shapes of traffic-relevant objects without aiming at object classification at the considered stage. We analyse disparity dynamics in recorded scenes for solving this task. Statistical shape models are generated over subsequent frames. Shape correspondences are established by using a similarity measure based on set theory. The motion of detected shapes (frame to frame) is compensated by using a dense motion field as produced by a real-time optical flow algorithm. Experimental results show the quality of the proposed method which is fairly simple to implement.


Driver Assistance System Obstacle Detection Pedestrian Detection Statistical Shape Model Occupancy Grid 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Badino, H.: A Robust Approach for Ego-Motion Estimation Using a Mobile Stereo Platform. In: Jähne, B., Mester, R., Barth, E., Scharr, H. (eds.) IWCM 2004. LNCS, vol. 3417, pp. 198–208. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Badino, H., Franke, U., Pfeiffer, D.: The Stixel World - A Compact Medium Level Representation of the 3D-World. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 51–60. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Barth, A., Siegemund, J., Meißner, A., Franke, U., Förstner, W.: Probabilistic Multi-Class Scene Flow Segmentation for Traffic Scenes. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) DAGM 2010. LNCS, vol. 6376, pp. 503–512. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  4. 4.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation Based on a Theory for Warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  5. 5.
    enpeda. image sequences analysis test site,
  6. 6.
    Franke, U., Rabe, C., Badino, H., Gehrig, S.: 6D-Vision: Fusion of Stereo and Motion for Robust Environment Perception. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 216–223. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Gómez, G.D.: A global approach to vision-based pedestrian detection for advanced driver assistance systems. PhD thesis, Univ. Autónoma de Barcelona (2010)Google Scholar
  8. 8.
    Haller, I., Pantillie, C., Oniga, F., Nedevschi, S.: Real-time semi-global dense stereo solution with improved sub-pixel accuracy. In: IVS, pp. 369–376 (2010)Google Scholar
  9. 9.
    Hirschmüller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. CVPR 2, 807–814 (2005)Google Scholar
  10. 10.
    Hirschmüller, H., Scharstein, D.: Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans. Pattern Analysis Machine Int. 31, 1582–1599 (2009)CrossRefGoogle Scholar
  11. 11.
    Ivekovic, S., Clark, D.: Multi-Object Stereo Filtering in Disparity Space. In: COGIS (2009)Google Scholar
  12. 12.
    Klette, R., Krüger, N., Vaudrey, T., Pauwels, K., van Hulle, M., Morales, S., Kandil, F., Haeusler, R., Pugeault, N., Rabe, C., Markus, L.: Performance of correspondence algorithms in vision-based driver assistance using an online image sequence database. IEEE Trans. Vehicular Technology (2011)Google Scholar
  13. 13.
    Klette, R., Rosenfeld, A.: Digital Geometry - Geometric Algorithms for Digital Picture Analysis. Morgan Kaufmann, San Francisco (2004)zbMATHGoogle Scholar
  14. 14.
    Labayrade, R., Aubert, D., Tarel, J.-P.: Real time obstacle detection in stereovision on non flat road geometry through ”v-disparity” representation. In: IVS, pp. 646–651 (2002)Google Scholar
  15. 15.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IUW, pp. 121–130 (1981)Google Scholar
  16. 16.
    Oniga, F., Nedevschi, S., Meinecke, M.M.: Occupancy grids detected from dense stereo using an elevation map representation. In: WIT, pp. 133–138 (2009)Google Scholar
  17. 17.
    Petersson, L., Fletcher, L., Zelinsky, A., Barnes, N., Arnell, F.: Towards safer roads by integration of road scene monitoring and vehicle control. Int. J. Robotic Res. 25, 53–72 (2006)CrossRefGoogle Scholar
  18. 18.
    Shimizu, M., Okutomi, M.: An analysis of subpixel estimation error on area-based image matching. In: Proc. Digital Signal Processing, vol. 2, pp. 1239–1242 (2002)Google Scholar
  19. 19.
    Vaudrey, T., Rabe, C., Klette, R., Milburn, J.: Differences between stereo and motion behaviour on synthetic and real-world stereo sequences. In: IVCNZ, pp. 1–6 (2008)Google Scholar
  20. 20.
    Wedel, A., Badino, H., Rabe, C., Loose, H., Franke, U., Cremers, D.: B-spline modeling of road surfaces with an application to free space estimation. In: IVS, pp. 828–833 (2008)Google Scholar
  21. 21.
    Wedel, A., Meißner, A., Rabe, C., Franke, U., Cremers, D.: Detection and Segmentation of Independently Moving Objects from Dense Scene Flow. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 14–27. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  22. 22.
    Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 739–751. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  23. 23.
    Wegener, P.: A technique for counting ones in a binary computer. Comm. ACM 3, 322 (1960)CrossRefGoogle Scholar
  24. 24.
    Yu, Q., Araujo, H., Wang, H.: A Stereovision Method for Obstacle Detection and Tracking in Non-Flat Urban Environments. Autonomous Robots 19, 141–157 (2005)CrossRefGoogle Scholar
  25. 25.
    Zabih, R., Woodfill, J.: Non-Parametric Local Transform for Computing Visual Correspondence. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, pp. 151–158. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  26. 26.
    Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L 1 optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  27. 27.
    Zhao, L., Thorpe, C.: Stereo and neural network-based pedestrian detection. IEEE Trans. Int. Transportation Systems 1, 148–154 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Simon Hermann
    • 1
  • Anko Börner
    • 2
  • Reinhard Klette
    • 1
  1. 1.The .enpeda.. Project, Department of Computer ScienceThe University of AucklandNew Zealand
  2. 2.DLR (German Aerospace Center)Berlin-AdlershofGermany

Personalised recommendations