Dense 3D Motion Field Estimation from a Moving Observer in Real Time

  • Clemens RabeEmail author
  • Uwe Franke
  • Reinhard Koch


In this chapter an approach for estimating the three-dimensional motion fields of real-world scenes is proposed. This approach combines state-of-the-art dense optical flow estimation, including spatial regularization, and dense stereo information using Kalman filters to achieve temporal smoothness and robustness. The result is a dense and accurate reconstruction of the three-dimensional motion field of the observed scene. An efficient parallel implementation using a GPU and an automotive compliant FPGA yields a real-time vision system which is directly applicable in real-world scenarios including driver assistance systems, robotics, and surveillance.


Computer vision Driver assistance Motion estimation 


  1. 1.
    A.A. Argyros, M.I. Lourakis, P.E. Trahanias, S.C. Orphanoudakis, Qualitative detection of 3D motion discontinuities, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’96), vol. 3, pp. 1630–1637, Nov 1996Google Scholar
  2. 2.
    S.S. Beauchemin, J.L. Barron, The computation of optical flow. ACM Comput. Surv. 27, 433–466 (1995)CrossRefGoogle Scholar
  3. 3.
    G.J. Bierman, Factorization Methods for Discrete Sequential Estimation (Academic Press, Inc., New York, 1977)zbMATHGoogle Scholar
  4. 4.
    D.J. Fleet, Y. Weiss, Optical flow estimation, Chapter 15, in Handbook of Mathematical Models in Computer Vision, ed. by N. Paragios, Y. Chen, O. Faugeras (Springer, Berlin, 2006), pp. 239–258Google Scholar
  5. 5.
    U. Franke, C. Rabe, H. Badino, S. Gehrig, 6DVision: fusion of stereo and motion for robust environment perception, in Proceedings of the 27th DAGM Symposium, pp. 216–223, 2005Google Scholar
  6. 6.
    S. Gehrig, F. Eberli, T. Meyer, A real-time low-power stereo vision engine using semi-global matching, in Proceedings of the 7th International Conference on Computer Vision Systems, Liège, Belgium, Oct 2009Google Scholar
  7. 7.
    S.K. Gehrig, C. Rabe, Real-time semi-global matching on the CPU, in Proceedings of the IEEE Workshop on Embedded Computer Vision, 2010Google Scholar
  8. 8.
    S. Heinrich, Fast obstacle detection using flow/depth constraint, in Proceedings of the IEEE Intelligent Vehicles Symposium 2002, vol. 2, pp. 658–665, Jun 2002Google Scholar
  9. 9.
    H. Hirschmüller, Accurate and efficient stereo processing by semi-global matching and mutual information, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, no. 2, San Diego, CA, USA, pp. 807–814, Jun 2005Google Scholar
  10. 10.
    B.K.P. Horn, B.G. Schunck, Determining optical flow. Artif. Intell. 17, 185–203 (1981)CrossRefGoogle Scholar
  11. 11.
    R.E. Kalman, A new approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 82(Series D), 35–45 (1960)CrossRefGoogle Scholar
  12. 12.
    P.J. Kellman, M.K. Kaiser, Extracting object motion during observer motion: combining constraints from optic flow and binocular disparity. J. Opt. Soc. Am. A 12, 623–625 (1995)CrossRefGoogle Scholar
  13. 13.
    M.C. Martin, H. Moravec, Robot evidence grids. Tech. Rep. CMU-RI-TR-96-06, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, Mar 1996Google Scholar
  14. 14.
    S. Mills, Stereo-motion analysis of image sequences, in Proceedings of the first joint Australia and New Zealand conference on Digital Image and Vision Computing: Techniques and Applications, DICTA’97/IVCNZ’97, Dec 1997Google Scholar
  15. 15.
    T. Müller, C. Rabe, J. Rannacher, U. Franke, R. Mester, Illumination robust dense optical flow using census signatures, in Proceedings of the 33th DAGM Symposium, 2011Google Scholar
  16. 16.
    C. Rabe, U. Franke, S. Gehrig, Fast detection of moving objects in complex scenarios, in Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 398–403, Jun 2007Google Scholar
  17. 17.
    C. Rabe, T. Müller, A. Wedel, U. Franke, Dense, robust, and accurate motion field estimation from stereo image sequences in real-time, in Proceedings of the 11th European Conference on Computer Vision, ed. by K. Daniilidis, P. Maragos, N. Paragios. Lecture Notes in Computer Science, vol. 6314 (Springer, Berlin, 2010), pp. 582–595Google Scholar
  18. 18.
    J. Rannacher, Realtime 3D motion estimation on graphics hardware, Bachelor thesis, Heidelberg University, 2009Google Scholar
  19. 19.
    S. Vedula, S. Baker, P. Rander, R. Collins, T. Kanade, Three-dimensional scene flow, in Seventh International Conference on Computer Vision (ICCV’99), vol. 2, pp. 722–729, 1999Google Scholar
  20. 20.
    A.M. Waxman, J.H. Duncan, Binocular image flows: steps toward stereo-motion fusion. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 715–729 (1986)CrossRefGoogle Scholar
  21. 21.
    A. Wedel, T. Pock, C. Zach, H. Bischof, D. Cremers, Statistical and geometrical approaches to visual motion analysis, in An Improved Algorithm for TV-L1 Optical Flow, ed. by D. Cremers, B. Rosenhahn, A.L. Yuille, F.R. Schmidt (Springer, Berlin, 2009), pp. 23–45Google Scholar
  22. 22.
    J. Weickert, A. Bruhn, T. Brox, N. Papenberg, A survey on variational optic flow methods for small displacements, in Mathematical Models for Registration and Applications to Medical Imaging, ed. by O. Scherzer (Springer, New York, 2006)Google Scholar
  23. 23.
    R. Zabih, J. Woodfill, Non-parametric local transforms for computing visual correspondence, in Proceedings of the Third European Conference on Computer Vision, pp. 151–158, May 1994Google Scholar
  24. 24.
    C. Zach, T. Pock, H. Bischof, A duality based approach for realtime TV-L1 optical flow, in Proceedings of the 29th DAGM Symposium on Pattern Recognition, pp. 214–223, 2007Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Research and Technology/Machine PerceptionDaimler AGSindelfingenGermany
  2. 2.Multimedia Information ProcessingChristian-Albrechts-Universität zu KielKielGermany

Personalised recommendations