Optical Flow Estimation

  • Andreas Wedel
  • Daniel Cremers


In this chapter we review the estimation of the two-dimensional apparent motion field of two consecutive images in an image sequence. This apparent motion field is referred to as optical flow field, a two-dimensional vector field on the image plane. Because it is nearly impossible to cover the vast amount of approaches in the literature, in this chapter we set the focus on energy minimization approaches which estimate a dense flow field. The term dense refers to the fact that a flow vector is assigned to every (non-occluded) image pixel. Most dense approaches are based on the variational formulation of the optical flow problem, firstly suggested by Horn and Schunk. Depending on the application, density might be one important property besides accuracy and robustness. In many cases computational speed and real-time capability is a crucial issue. In this chapter we therefore discuss the latest progress in accuracy, robustness and real-time capability of dense optical flow algorithms.


Optical Flow Data Term Flow Vector Pyramid Level Optical Flow Field 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Alvarez, L., Esclarín, J., Lefébure, M., Sánchez, J.: A PDE model for computing the optical flow. In: Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, Gran Canaria, Spain, pp. 1349–1356 (1999) Google Scholar
  2. 2.
    Aubert, G., Deriche, R., Kornprobst, P.: Computing optical flow via variational techniques. SIAM J. Appl. Math. 60(1), 156–182 (1999) MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Aujol, J.F., Gilboa, G., Chan, T.F., Osher, S.J.: Structure-texture image decomposition: modeling, algorithms, and parameter selection. Int. J. Comput. Vis. 67(1) (2006). doi: 10.1007/s11263-006-4331-z
  4. 5.
    Baker, S., Matthews, I.: Lucas-Kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(3), 221–255 (2004) CrossRefGoogle Scholar
  5. 6.
    Baker, S., Roth, S., Scharstein, D., Black, M.J., Lewis, J.P., Szeliski, R.: A database and evaluation methodology for optical flow. In: Online-Proc. International Conference on Computer Vision, Rio de Janeiro, Brazil, October 2007 Google Scholar
  6. 7.
    Barni, M., Cappellini, V., Mecocci, A.: Fast vector median filter based on Euclidean norm approximation. IEEE Signal Process. Lett. 1(6), 92–94 (2004) CrossRefGoogle Scholar
  7. 9.
    Black, M.J., Anandan, P.: A framework for the robust estimation of optical flow. In: Proc. International Conference on Computer Vision, Nice, France, pp. 231–236 (1993) Google Scholar
  8. 11.
    Brox, T.: From pixels to regions: partial differential equations in image analysis. Ph.D. thesis, Faculty of Mathematics and Computer Science, Saarland University, Germany (2005) Google Scholar
  9. 12.
    Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. (2010). doi: 10.1109/TPAMI.2010.143 Google Scholar
  10. 13.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Proc. European Conference on Computer Vision, Prague, Czech Republic, pp. 25–36 (2004) Google Scholar
  11. 18.
    Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schnörr, C.: Variational optic flow computation in real-time. IEEE Trans. Image Process. 14(5), 608–615 (2005) MathSciNetCrossRefGoogle Scholar
  12. 19.
    Bruhn, A., Weickert, J., Kohlberger, T., Schnörr, C.: A multigrid platform for real-time motion computation with discontinuity-preserving variational methods. Int. J. Comput. Vis. 70(3), 257–277 (2006) CrossRefGoogle Scholar
  13. 20.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1), 89–97 (2004) MathSciNetCrossRefGoogle Scholar
  14. 21.
    Chambolle, A.: Total variation minimization and a class of binary MRF models. In: Proc. International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, St. Augustine, FL, USA, pp. 136–152 (2005) CrossRefGoogle Scholar
  15. 22.
    Chambolle, A., Caselles, V., Cremers, D., Novaga, M., Pock, T.: An introduction to total variation for image analysis. In: Theoretical Foundations and Numerical Methods for Sparse Recovery. De Gruyter, Berlin (2010) Google Scholar
  16. 23.
    Chan, T.F., Golub, G.H., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Appl. Math. 20(10), 1964–1977 (1999) MathSciNetMATHGoogle Scholar
  17. 24.
    Cohen, I.: Nonlinear variational method for optical flow computation. In: Scandinavian Conf. on Image Analysis, pp. 523–523 (1993) Google Scholar
  18. 25.
    Coleman, T.F., Hulbert, L.A.: A direct active set algorithm for large sparse quadratic programs with simple bounds. Math. Program., Sers. A, B 45(3), 373–406 (1989) MathSciNetMATHCrossRefGoogle Scholar
  19. 26.
    Cooke, T.: Two applications of graph-cuts to image processing. In: Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia, pp. 498–504 (2008) CrossRefGoogle Scholar
  20. 27.
    Corpetti, T., Memin, E., Perez, P.: Dense estimation of fluid flows. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 365–380 (2002) CrossRefGoogle Scholar
  21. 30.
    Deriche, R., Kornprobst, P., Aubert, G.: Optical flow estimation while preserving its discontinuities: a variational approach. In: Proc. Asian Conference on Computer Vision, Singapore, pp. 290–295 (1995) Google Scholar
  22. 31.
    Devillard, N.: Fast median search: an ANSI C implementation (1998).
  23. 32.
    Felsberg, M.: On the relation between anisotropic diffusion and iterated adaptive filtering. In: Pattern Recognition (Proc. DAGM), Munich, Germany, pp. 436–445 (2008) CrossRefGoogle Scholar
  24. 34.
    Goldluecke, B., Cremers, D.: Convex relaxation for multilabel problems with product label spaces. In: Proc. European Conference on Computer Vision (2010) Google Scholar
  25. 42.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981) CrossRefGoogle Scholar
  26. 47.
    Karush, W.: Minima of functions of several variables with inequalities as side constraints. Ph.D. thesis, Dept. of Mathematics, University of Chicago (1939) Google Scholar
  27. 48.
    Klappstein, J.: Optical-flow based detection of moving objects in traffic scenes. Ph.D. thesis, University of Heidelberg, Heidelberg, Germany (2008) Google Scholar
  28. 53.
    Lempitsky, V., Roth, S., Rother, C.: Fusionflow: discrete-continuous optimization for optical flow estimation. In: Online-Proc. International Conference on Computer Vision and Pattern Recognition, Anchorage, USA, June 2008 Google Scholar
  29. 54.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. of the 7th International Joint Conference on Artificial Intelligence (IJCAI), Vancouver, British Columbia, Canada, pp. 674–679 (1981) Google Scholar
  30. 55.
    Luenberger, D.G.: Linear and Nonlinear Programming. Addison-Wesley, Reading (1984) MATHGoogle Scholar
  31. 58.
    Masoomzadeh-Fard, A., Venetsanopoulos, A.N.: An efficient vector ranking filter for colour image restoration. In: Proc. Canadian Conference on Electrical and Computer Engineering, Vancouver, BC, Canada, pp. 1025–1028 (1993) Google Scholar
  32. 60.
    Mémin, E., Pérez, P.: A multigrid approach for hierarchical motion estimation. In: Proc. International Conference on Computer Vision, Bombay, India, pp. 933–938 (1998) Google Scholar
  33. 64.
    Nagel, H.H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8(5), 565–593 (1986) CrossRefGoogle Scholar
  34. 65.
    Nagel, H.H.: Constraints for the estimation of displacement vector fields from image sequences. In: Proc. Eighth Int. Conf. Artif. Intell., Karlsruhe, Germany, pp. 945–951 (1983) Google Scholar
  35. 66.
    Nagel, H.-H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8(5), 565–593 (1986) CrossRefGoogle Scholar
  36. 67.
    Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optic flow computation with theoretically justified warping. Int. J. Comput. Vis. 67(2), 141–158 (2006) CrossRefGoogle Scholar
  37. 70.
    Pock, T.: Fast total variation for computer vision. Ph.D. thesis, Institute for Computer Graphics and Vision, University of Graz, Graz, Austria (2008) Google Scholar
  38. 73.
    Rabe, C., Volmer, C., Franke, U.: Kalman filter based detection of obstacles and lane boundary. Autonome Mobile Systeme 19(1), 51–58 (2005) Google Scholar
  39. 78.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992) MATHCrossRefGoogle Scholar
  40. 79.
    Ruhnau, P., Schnoerr, C.: Variational estimation of experimental fluid flows with physics-based spatio-temporal regularization. Meas. Sci. Technol. 18, 755–763 (2007) CrossRefGoogle Scholar
  41. 81.
    Schnörr, C.: Segmentation of visual motion by minimizing convex non-quadratic functionals. In: 12th Int. Conf. on Pattern Recognition, Jerusalem, Israel, pp. 661–663 (1994) CrossRefGoogle Scholar
  42. 84.
    Spies, H., Kirchges̈ner, N., Scharr, H., Jähne, B.: Dense structure estimation via regularised optical flow. In: Proc. Vision, Modeling, and Visualization, Saarbrücken, Germany, pp. 57–64 (2000) Google Scholar
  43. 85.
    Stein, F.: Efficient computation of optical flow using the Census transform. In: Pattern Recognition (Proc. DAGM), Tübingen, Germany, pp. 79–86 (2004) CrossRefGoogle Scholar
  44. 86.
    Steinbruecker, F., Pock, T., Cremers, D.: Large displacement optical flow computation without warping. In: IEEE International Conference on Computer Vision (ICCV), Kyoto, Japan (2009) Google Scholar
  45. 87.
    Stewart, E.: Intel Integrated Performance Primitives: How to Optimize Software Applications Using Intel Ipp. Intel Press, Santa Clara (2004) Google Scholar
  46. 90.
    Sun, D., Roth, S., Lewis, J.P., Black, M.J.: Learning optical flow. In: Proc. European Conference on Computer Vision, Marseille, France, pp. 83–91 (2008) Google Scholar
  47. 91.
    Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: Proc. International Conference on Computer Vision and Pattern Recognition, pp. 2432–2439 (2010) Google Scholar
  48. 92.
    Tomasi, C., Kanade, T.: Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon University, April 1991 Google Scholar
  49. 94.
    Trobin, W., Pock, T., Cremers, D., Bischof, H.: Continuous energy minimization via repeated binary fusion. In: European Conference on Computer Vision (ECCV), Marseille, France, October 2008 Google Scholar
  50. 95.
    Trobin, W., Pock, T., Cremers, D., Bischof, H.: An unbiased second-order prior for high-accuracy motion estimation. In: Pattern Recognition (Proc. DAGM), Munich, Germany, pp. 396–405 (2008) CrossRefGoogle Scholar
  51. 96.
    Unger, M., Pock, T., Bischof, H.: Continuous globally optimal image segmentation with local constraints. In: Computer Vision Winter Workshop, February 2008 Google Scholar
  52. 113.
    Weickert, J., Brox, T.: Diffusion and regularization of vector- and matrix-valued images. In: Inverse Problems, Image Analysis and Medical Imaging. Contemporary Mathematics vol. 313, pp. 251–268. AMS, Providence (2002) Google Scholar
  53. 114.
    Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE-based computation of image motion. Int. J. Comput. Vis. 45(3), 245–264 (2001) CrossRefGoogle Scholar
  54. 116.
    Yang, Z., Fox, M.D.: Speckle reduction and structure enhancement by multichannel median boosted anisotropic diffusion. EURASIP J. Appl. Signal Process. 2492–2502 (2004). doi: 10.1155/S1110865704402091
  55. 117.
    Yin, W., Goldfarb, D., Osher, S.: Image cartoon-texture decomposition and feature selection using the total variation regularized L 1 functional. In: Variational, Geometric, and Level Set Methods in Computer Vision, Beijing, China, pp. 73–80 (2005) CrossRefGoogle Scholar
  56. 118.
    Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Proc. European Conference on Computer Vision, Prague, Czech Republic, pp. 151–158 (2004) Google Scholar
  57. 119.
    Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L1 optical flow. In: Pattern Recognition (Proc. DAGM), Heidelberg, Germany, pp. 214–223 (2007) CrossRefGoogle Scholar
  58. 120.
    Zach, C., Gallup, D., Frahm, J.M.: Fast gain-adaptive KLT tracking on the GPU. In: Online-Proc. International Conference on Computer Vision and Pattern Recognition, Anchorage, AK, June 2008 Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Group ResearchDaimler AGSindelfingenGermany
  2. 2.Department of Computer ScienceTechnical University of MunichGarchingGermany

Personalised recommendations