An Evaluation Approach for Scene Flow with Decoupled Motion and Position

  • Andreas Wedel
  • Tobi Vaudrey
  • Annemarie Meissner
  • Clemens Rabe
  • Thomas Brox
  • Uwe Franke
  • Daniel Cremers
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5604)


This chapter presents a technique for estimating the three-dimensional displacement vector field that describes the motion of each visible scene point. This displacement field consists of the change in image position, also known as optical flow, and the change in disparity and is called the scene flow. The technique presented uses two consecutive image pairs from a stereo sequence. The main contribution is to decouple the disparity (3D position) and the scene flow (optical flow and disparity change) estimation steps. Thus we present a technique to estimate a dense scene flow field using a variational approach from the gray value images and a given stereo disparity map.

Although the two subproblems disparity and scene flow estimation are decoupled, we enforce the scene flow to yield consistent displacement vectors in all four stereo images involved at two time instances. The decoupling strategy has two benefits: Firstly, we are independent in choosing a disparity estimation technique, which can yield either sparse or dense correspondences, and secondly, we can achieve frame rates of 5 fps on standard consumer hardware. This approach is then expanded to real-world displacements, and two metrics are presented that define likelihoods of movement with respect to the background. Furthermore, an evaluation approach is presented to compare scene flow algorithms on long image sequences, using synthetic data as ground truth.


Root Mean Square Disparity Estimation Lead Vehicle Stereo Image Pair Stereo Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Badino, H.: A robust approach for ego-motion estimation using a mobile stereo platform. In: Jähne, B., Mester, R., Barth, E., Scharr, H. (eds.) IWCM 2004. LNCS, vol. 3417, pp. 198–208. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Bruhn, A., Weickert, J., Kohlberger, T., Schnörr, C.: Discontinuity preserving computation of variational optic flow in real-time. In: ScaleSpace 2005, pp. 279–290 (2005)Google Scholar
  4. 4.
    Franke, U., Joos, A.: Real-time stereo vision for urban traffic scene understanding. In: Proc. IEEE Intelligent Vehicles Symposium, Dearborn, pp. 273–278 (2000)Google Scholar
  5. 5.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)Google Scholar
  6. 6.
    Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Pattern Analysis and Machine Intelligence (PAMI) 30(2), 328–341 (2008)CrossRefGoogle Scholar
  7. 7.
    Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  8. 8.
    Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: IEEE Int. Conf. on Computer Vision (ICCV) (October 2007)Google Scholar
  9. 9.
    Isard, M., MacCormick, J.: Dense motion and disparity estimation via loopy belief propagation. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3852, pp. 32–41. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Mahalanobis, P.C.: On the generalised distance in statistics. In: Proc. of the National Institute of Science of India, vol. 12, pp. 49–55 (1936)Google Scholar
  11. 11.
    Mémin, E., Pérez, P.: Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Trans. on Image Processing 7(5), 703–719 (1998)CrossRefGoogle Scholar
  12. 12.
    Min, D., Sohn, K.: Edge-preserving simultaneous joint motion-disparity estimation. In: Proc. Int. Conf. on Pattern Recognition (ICPR), Washington, DC, USA, 2006, pp. 74–77. IEEE Computer Society Press, Los Alamitos (2006)Google Scholar
  13. 13.
    Patras, I., Hendriks, E., Tziritas, G.: A joint motion/disparity estimation method for the construction of stereo interpolated images in stereoscopic image sequences. In: Proc. 3rd Annual Conf. of the Advanced School for Computing and Imaging, Heijen, The Netherlands (1997)Google Scholar
  14. 14.
    Pons, J.-P., Keriven, R., Faugeras, O.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. Int. J. Computer Vision 72(2), 179–193 (2007)CrossRefGoogle Scholar
  15. 15.
    Rabe, C., Franke, U., Gehrig, S.: Fast detection of moving objects in complex scenarios. In: Proc. IEEE Intelligent Vehicles Symposium, June 2007, pp. 398–403 (2007)Google Scholar
  16. 16.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. In: Proc. IEEE Int. Conf. on Computer Vision (ICCV), pp. 7–42 (2002)Google Scholar
  17. 17.
    Stein, F.: Efficient computation of optical flow using the census transform. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 79–86. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  18. 18.
    Tomasi, C., Kanade, T.: Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon University (April 1991)Google Scholar
  19. 19.
    University of Auckland. enpeda. Image Sequence Analysis Test Site (EISATS) (2008),
  20. 20.
    Vaudrey, T., Rabe, C., Klette, R., Milburn, J.: Differences between stereo and motion behaviour on synthetic and real-world stereo sequences. In: Proc. Int. Conf. Image and Vision Computing New Zealand (IVCNZ) (2008)Google Scholar
  21. 21.
    Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) 27(3), 475–480 (2005)CrossRefGoogle Scholar
  22. 22.
    Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for TV-L1 optical flow. In: Cremers, D., Rosenhann, B., Yuille, A., Schmidt, F.R. (eds.) Visual Motion Analysis 2008. LNCS, vol. 5604, pp. 23–45. Springer, Heidelberg (2009)Google Scholar
  23. 23.
    Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 739–751. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  24. 24.
    Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime tv-L 1 optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  25. 25.
    Zhang, Y., Kambhamettu, C.: On 3d scene flow and structure estimation. In: Proc. IEEE Conf. in Computer Vision and Pattern Recognition, vol. 2, p. 778. IEEE Computer Society Press, Los Alamitos (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Andreas Wedel
    • 1
    • 2
  • Tobi Vaudrey
    • 3
  • Annemarie Meissner
    • 1
  • Clemens Rabe
    • 1
  • Thomas Brox
    • 4
  • Uwe Franke
    • 1
  • Daniel Cremers
    • 2
  1. 1.Daimler Group ResearchGermany
  2. 2.University of BonnGermany
  3. 3.University of AucklandNew Zealand
  4. 4.U.C. Berkeley (University of California)USA

Personalised recommendations