Advertisement

Scene Flow

  • Andreas Wedel
  • Daniel Cremers

Abstract

Building upon optical flow and recent developments in stereo matching estimation, we discuss in this chapter how the motion of points in the three-dimensional world can be derived from stereo image sequences. The proposed algorithm takes into account image pairs from two consecutive times and computes both depth and a 3D motion vector associated with each point in the image. We particularly investigate a decoupled approach of depth estimation and variational motion estimation, which has some practical advantages. The variational formulation is quite flexible and can handle both sparse or dense disparity maps. With the depth map being computed on an FPGA, and the scene flow computed on the GPU, the scene flow algorithm currently runs at frame rates of 20 frames per second on QVGA images (320×240 pixels).

Keywords

Optical Flow Motion Estimation Disparity Estimation Stereo Image Pair Optical Flow Field 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 4.
    Badino, H.: A robust approach for ego-motion estimation using a mobile stereo platform. In: Proc. International Workshop on Complex Motion, Günzburg, Germany, pp. 198–208 (2004) Google Scholar
  2. 13.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Proc. European Conference on Computer Vision, Prague, Czech Republic, pp. 25–36 (2004) Google Scholar
  3. 28.
    Costeira, J., Kanande, T.: A multi-body factorization method for motion analysis. In: Proc. International Conference on Computer Vision, pp. 1071–1076 (1995) Google Scholar
  4. 33.
    Franke, U., Joos, A.: Real-time stereo vision for urban traffic scene understanding. In: Proc. IEEE Intelligent Vehicles Symposium, Dearborn, pp. 273–278 (2000) Google Scholar
  5. 34.
    Goldluecke, B., Cremers, D.: Convex relaxation for multilabel problems with product label spaces. In: Proc. European Conference on Computer Vision (2010) Google Scholar
  6. 35.
    Gong, M.: Real-time joint disparity and disparity flow estimation on programmable graphics hardware. Comput. Vis. Image Underst. 113(1), 90–100 (2009) CrossRefGoogle Scholar
  7. 36.
    Gong, M., Yang, Y.-H.: Disparity flow estimation using orthogonal reliability-based dynamic programming. In: Proc. International Conference on Pattern Recognition, pp. 70–73. IEEE Computer Society, Los Alamitos (2006) Google Scholar
  8. 38.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004) MATHCrossRefGoogle Scholar
  9. 40.
    Hirschmüller, H.: Stereo vision in structured environments by consistent semi-global matching. In: Proc. International Conference on Computer Vision and Pattern Recognition, New York, NY, USA, pp. 2386–2393 (2006) Google Scholar
  10. 41.
    Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008) CrossRefGoogle Scholar
  11. 43.
    Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: Online-Proc. International Conference on Computer Vision, Rio de Janeiro, Brazil, October 2007 Google Scholar
  12. 44.
    Isard, M., MacCormick, J.: Dense motion and disparity estimation via loopy belief propagation. In: Proc. Asian Conference on Computer Vision, Hyderabad, India, pp. 32–41 (2006) Google Scholar
  13. 46.
    Kanatani, K., Sugaya, Y.: Multi-stage optimization for multi-body motion segmentation. IEICE Trans. Inf. Syst. E87-D(7), 1935–1942 (2004) Google Scholar
  14. 63.
    Min, D., Sohn, K.: Edge-preserving simultaneous joint motion-disparity estimation. In: Proc. International Conference on Pattern Recognition, Hong Kong, China, pp. 74–77 (2006) Google Scholar
  15. 68.
    Patras, I., Alvertos, N., Tziritas, G.: Joint disparity and motion field estimation in stereoscopic image sequences. In: Proc. International Conference on Pattern Recognition, Vienna, Austria, pp. 359–363 (1996) CrossRefGoogle Scholar
  16. 71.
    Pock, T., Cremers, D., Bischof, H., Chambolle, A.: Global solutions of variational models with convex regularization. SIAM J. Imag. Sci. 3(4), 1122–1145 (2010) MathSciNetMATHCrossRefGoogle Scholar
  17. 72.
    Pons, J.-P., Keriven, R., Faugeras, O.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. Int. J. Comput. Vis. 72(2), 179–193 (2007) CrossRefGoogle Scholar
  18. 74.
    Rabe, C., Franke, U., Gehrig, S.: Fast detection of moving objects in complex scenarios. In: Proc. IEEE Intelligent Vehicles Symposium, Istanbul, Turkey, pp. 398–403 (2007) CrossRefGoogle Scholar
  19. 76.
    Rao, S.R., Tron, R., Vidal, R., Ma, Y.: Motion segmentation via robust subspace separation in the presence of outlying, incomplete, or corrupted trajectories. In: Proc. International Conference on Computer Vision and Pattern Recognition (2008) Google Scholar
  20. 80.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. In: Proc. International Conference on Computer Vision, pp. 7–42. IEEE Computer Society, Los Alamitos (2002) Google Scholar
  21. 85.
    Stein, F.: Efficient computation of optical flow using the Census transform. In: Pattern Recognition (Proc. DAGM), Tübingen, Germany, pp. 79–86 (2004) CrossRefGoogle Scholar
  22. 103.
    Vedula, S., Baker, S., Rander, P., Kanade, R.C.T.: Three-dimensional scene flow. IEEE Trans. Pattern Anal. Mach. Intell. 27(3), 475–480 (2005) CrossRefGoogle Scholar
  23. 108.
    Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: Proc. European Conference on Computer Vision, Marseille, France, pp. 739–751 (2008) Google Scholar
  24. 115.
    Yan, J., Pollefeys, M.: A general framework for motion segmentation: independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: Proc. European Conference on Computer Vision, vol. 3954, pp. 94–106. Springer, Berlin (2006) Google Scholar
  25. 121.
    Zhang, Y., Kambhamettu, C.: On 3d scene flow and structure estimation. In: Proc. International Conference on Computer Vision and Pattern Recognition, Kauai Marriott, Hawaii, pp. 778–785 (2001) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Group ResearchDaimler AGSindelfingenGermany
  2. 2.Department of Computer ScienceTechnical University of MunichGarchingGermany

Personalised recommendations