Discrete-Continuous Optimization for Optical Flow Estimation

  • Stefan Roth
  • Victor Lempitsky
  • Carsten Rother
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5604)


The accurate estimation of optical flow is a challenging task, which is often posed as an energy minimization problem. Most top-performing methods approach this using continuous optimization algorithms. In many cases, the employed models are assumed to be convex to ensure tractability of the optimization problem. This is in contrast to the related problem of narrow-baseline stereo matching, where the top-performing methods employ powerful discrete optimization algorithms such as graph cuts and message-passing to optimize highly non-convex energies.

In this chapter, we demonstrate how similar non-convex energies can be formulated and optimized in the context of optical flow estimation using a combination of discrete and continuous techniques. Starting with a set of candidate solutions that are produced by either fast continuous flow estimation algorithms or sparse feature matching, the proposed method iteratively fuses these candidate solutions by the computation of minimum cuts on graphs. The obtained continuous-valued result is then further improved using local gradient descent. Experimentally, we demonstrate that the proposed energy is an accurate model and that the proposed discrete-continuous optimization scheme not only finds lower energy solutions than traditional discrete or continuous optimization techniques, but also leads to very accurate flow estimates.


Discrete Optimization Continuous Optimization Spatial Term Proposal Solution Extended Graph 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artificial Intelligence 17(1-3), 185–203 (1981)CrossRefGoogle Scholar
  2. 2.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI, April 1981, pp. 674–679 (1981)Google Scholar
  3. 3.
    Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optic flow computation with theoretically justified warping. IJCV 67(2), 141–158 (2006)CrossRefGoogle Scholar
  4. 4.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU 63(1), 75–104 (1996)Google Scholar
  5. 5.
    Roth, S., Black, M.J.: On the spatial statistics of optical flow. IJCV 74(1), 33–50 (2007)CrossRefGoogle Scholar
  6. 6.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. In: ICCV 2007 (2007),
  7. 7.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. TPAMI 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  8. 8.
    Sun, J., Zhen, N.N., Shum, H.Y.: Stereo matching using belief propagation. PAMI 25(7), 787–800 (2003)CrossRefGoogle Scholar
  9. 9.
    Meltzer, T., Yanover, C., Weiss, Y.: Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation. In: ICCV 2005, vol. 1, pp. 428–435 (2005)Google Scholar
  10. 10.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. In: CVPR 2004, vol. 1, pp. 261–268 (2004)Google Scholar
  11. 11.
    Glocker, B., Komodakis, N., Paragios, N., Tziritas, G., Navab, N.: Inter and intra-modal deformable registration: Continuous deformations meet efficient optimal linear programming. In: Karssemeijer, N., Lelieveldt, B. (eds.) IPMI 2007. LNCS, vol. 4584, pp. 408–420. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  12. 12.
    Lempitsky, V., Rother, C., Blake, A.: LogCut - Efficient graph cut optimization for Markov random fields. In: ICCV 2007 (2007)Google Scholar
  13. 13.
    Shekhovtsov, A., Kovtun, I., Hlavac, V.: Efficient MRF deformation model for non-rigid image matching. In: CVPR 2007 (2007)Google Scholar
  14. 14.
    Lempitsky, V., Roth, S., Rother, C.: FusionFlow: Discrete-continuous optimization for optical flow estimation. In: CVPR 2008 (2008)Google Scholar
  15. 15.
    Glocker, B., Paragios, N., Komodakis, N., Tziritas, G., Navab, N.: Optical flow estimation with uncertainties through dynamic MRFs. In: CVPR 2008 (2008)Google Scholar
  16. 16.
    Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. IJCV 61(3), 211–231 (2005)CrossRefGoogle Scholar
  17. 17.
    Boros, E., Hammer, P.L., Tavares, G.: Preprocessing of unconstrained quadratic binary optimization. Technical Report RUTCOR RRR (2006)Google Scholar
  18. 18.
    Boros, E., Hammer, P.L.: Pseudo-boolean optimization. Discrete Applied Mathematics 123(1-3), 155–225 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Kolmogorov, V., Rother, C.: Minimizing non-submodular functions with graph cuts — A review. TPAMI 29(7), 1274–1279 (2006)CrossRefGoogle Scholar
  20. 20.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)CrossRefGoogle Scholar
  21. 21.
    Liu, C., Yuen, J., Torralba, A.B., Sivic, J., Freeman, W.T.: SIFT flow: Dense correspondence across different scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 28–42. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  22. 22.
    Brox, T., Bruhn, A., Weickert, J.: Variational motion segmentation with level sets. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 471–483. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  23. 23.
    Mémin, É., Pérez, P.: Hierarchical estimation and segmentation of dense motion fields. IJCV 46(2), 129–155 (2002)zbMATHCrossRefGoogle Scholar
  24. 24.
    Heitz, F., Bouthemy, P.: Multimodal estimation of discontinuous optical flow using Markov random fields. TPAMI 15(12), 1217–1232 (1993)CrossRefGoogle Scholar
  25. 25.
    Konrad, J., Dubois, E.: Multigrid Bayesian estimation of image motion fields using stochastic relaxation. In: ICCV 1988, pp. 354–362 (1988)Google Scholar
  26. 26.
    Lempitsky, V., Rother, C., Roth, S., Blake, A.: Fusion moves for Markov random field optimization. TPAMI (in revision)Google Scholar
  27. 27.
    Woodford, O.J., Torr, P.H.S., Reid, I.D., Fitzgibbon, A.W.: Global stereo reconstruction under second order smoothness priors. In: CVPR 2008 (2008)Google Scholar
  28. 28.
    Trobin, W., Pock, T., Cremers, D., Bischof, H.: Continuous energy minimization via repeated binary fusion. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 677–690. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  29. 29.
    Birchfield, S., Natarjan, B., Tomasi, C.: Correspondence as energy-based segmentation. Image Vision Comp. 25(8), 1329–1340 (2007)CrossRefGoogle Scholar
  30. 30.
    Sun, D., Roth, S., Lewis, J.P., Black, M.J.: Learning optical flow. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 83–97. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  31. 31.
    Li, Y., Huttenlocher, D.P.: Learning for optical flow using stochastic optimization. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 379–391. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  32. 32.
    Greig, D.M., Porteous, B.T., Seheult, A.H.: Exact MAP estimation for binary images. J. Roy. Stat. Soc. B 51(2), 271–279 (1989)Google Scholar
  33. 33.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. TPAMI 26(9), 1124–1137 (2004)CrossRefGoogle Scholar
  34. 34.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? TPAMI 24(2), 147–159 (2004)Google Scholar
  35. 35.
    Rother, C., Kolmogorov, V., Lempitsky, V., Szummer, M.: Optimizing binary MRFs via extended roof duality. In: CVPR 2007 (2007)Google Scholar
  36. 36.
    Ishikawa, H.: Exact optimization for Markov random fields with convex priors. TPAMI 25(10), 1333–1336 (2003)CrossRefGoogle Scholar
  37. 37.
    Rasmussen, C.E.: minimize.m (September 2006),

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Stefan Roth
    • 1
  • Victor Lempitsky
    • 2
  • Carsten Rother
    • 2
  1. 1.Department of Computer ScienceTU DarmstadtDarmstadtGermany
  2. 2.Microsoft Research CambridgeUK

Personalised recommendations