Journal of Mathematical Imaging and Vision

, Volume 50, Issue 3, pp 199–213 | Cite as

Variational-Bayes Optical Flow

  • Giannis Chantas
  • Theodosios Gkamas
  • Christophoros Nikou


The Horn-Schunck (HS) optical flow method is widely employed to initialize many motion estimation algorithms. In this work, a variational Bayesian approach of the HS method is presented, where the motion vectors are considered to be spatially varying Student’s t-distributed unobserved random variables, i.e., the prior is a multivariate Student’s t-distribution, while the only observations available is the temporal and spatial image difference. The proposed model takes into account the residual resulting from the linearization of the brightness constancy constraint by Taylor series approximation, which is also assumed to be a spatially varying Student’s t-distributed observation noise. To infer the model variables and parameters we recur to variational inference methodology leading to an expectation-maximization (EM) framework with update equations analogous to the Horn-Schunck approach. This is accomplished in a principled probabilistic framework where all of the model parameters are estimated automatically from the data. Experimental results show the improvement obtained by the proposed model which may substitute the standard algorithm in the initialization of more sophisticated optical flow schemes.


Optical flow estimation Variational inference Bayesian methodology Student’s t-distribution 


  1. 1.
    Alvarez, L., Weickert, J., Sanchez, J.: Reliable estimation of dense optical flow fields with large displacements. Int. J. Comput. Vis. 39, 41–56 (2000) CrossRefMATHGoogle Scholar
  2. 2.
    Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proceedings of the International Conference of Computer Vision (ICCV), pp. 1–8 (2007) Google Scholar
  3. 3.
    Barron, J., Fleet, D., Beauchemin, S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994) CrossRefGoogle Scholar
  4. 4.
    Beal, M.J.: Variational algorithms for approximate Bayesian inference. Technical report, The Gatsby Computational Neuroscience Unit, University College, London (2003) Google Scholar
  5. 5.
    Beal, M.J.: The variational Bayesian EM algorithm for incomplete data with application to scoring graphical model structures. Bayesian Stat. 7, 453–464 (2003) MathSciNetGoogle Scholar
  6. 6.
    Ben-Ari, R., Sochen, N.: A general framework and new alignment criterion for dense optical flow. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 529–536 (2006) Google Scholar
  7. 7.
    Birchfield, S., Pundlik, S.: Joint tracking of features and edges. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2008) Google Scholar
  8. 8.
    Bishop, C.: Pattern Recognition and Machine Learning. Springer, Berlin (2006) MATHGoogle Scholar
  9. 9.
    Black, M., Anandan, P.: A framework for the robust estimation of optical flow. In: Fourth International Conference on Computer Vision, pp. 231–236 (1993) Google Scholar
  10. 10.
    Black, M., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996) CrossRefGoogle Scholar
  11. 11.
    Black, M., Fleet, D.: Probabilistic detection and tracking of motion boundaries. Int. J. Comput. Vis. 38, 231–245 (2000) CrossRefMATHGoogle Scholar
  12. 12.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Proceedings of the 8th European Conference on Computer Vision (ECCV), vol. 4, pp. 25–36 (2004) Google Scholar
  13. 13.
    Bruhn, A., Weickert, J., Schnorr, C.: Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. Int. J. Comput. Vis. 61(3), 211–231 (2005) CrossRefGoogle Scholar
  14. 14.
    Chantas, G., Galatsanos, N., Likas, A., Saunders, M.: Variational Bayesian image restoration based on a product of t-distributions image prior. IEEE Trans. Image Process. 17(10), 1795–1805 (2008) MathSciNetCrossRefGoogle Scholar
  15. 15.
    Gkamas, T., Chantas, G., Nikou, C.: A probabilistic formulation of the optical flow problem. In: 21st International Conference on Pattern Recognition (ICPR), 11–15 November, Tsukuba, Japan (2012) Google Scholar
  16. 16.
    Glocker, B., Heibel, T.H., Navab, N., Kohli, P., Rother, C.: Triangleflow: optical flow with triangulation-based higher-order likelihoods. In: Proceedings of the 11th European Conference on Computer Vision Conference on Computer Vision: Part III, ECCV’10, pp. 272–285 (2010) Google Scholar
  17. 17.
    Heas, P., Herzet, C., Memin, E.: Robust optic-flow estimation with Bayesian inference of model and hyper-parameters. In: Scale Space and Variational Methods in Computer Vision. LNCS, vol. 6667, pp. 773–785 (2012) CrossRefGoogle Scholar
  18. 18.
    Horn, B., Schunck, B.: Determining optical flow. Artif. Intell. 17, 185–203 (1981) CrossRefGoogle Scholar
  19. 19.
    Krajsek, K., Mester, A.: Bayesian inference and maximum entropy methods in science and engineering, aip conference proceedings. In: Scale Space and Variational Methods in Computer Vision. LNCS, vol. 872, pp. 311–318 (2006) Google Scholar
  20. 20.
    Krajsek, K., Mester, R.: A maximum likelihood estimator for choosing the regularization parameters in global optical flow methods. In: IEEE International Conference on Image Processing, pp. 1081–1084 (2006) Google Scholar
  21. 21.
    Krajsek, K., Mester, A.: Bayesian model selection for optical flow estimation. In: Proceedings of the 29th DAGM Conference on Pattern Recognition, Heidelberg (2007) Google Scholar
  22. 22.
    Lee, K.J., Kwon, D., Yun, I.D., Lee, S.U.: Optical flow estimation with adaptive convolution kernel prior on discrete framework. In: CVPR, pp. 2504–2511 (2010) Google Scholar
  23. 23.
    Liu, C., Rubin, D.B.: ML estimation of the t-distribution using EM and its extensions. Technical report, ECM and ECME, Statistica Sinica (1995) Google Scholar
  24. 24.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intellegence (IJCAI), pp. 674–679 (1981) Google Scholar
  25. 25.
    McCane, B., Novins, K., Crannitch, D., Galvin, B.: On benchmarking optical flow. Comput. Vis. Image Underst. 84(1), 126–143 (2001) CrossRefMATHGoogle Scholar
  26. 26.
    Memin, E., Perez, P.: Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Trans. Image Process. 7(5), 703–719 (1998) CrossRefGoogle Scholar
  27. 27.
    Molina, R.: On the hierarchical Bayesian approach to image restoration. Applications to astronomical images. IEEE Trans. Pattern Anal. Mach. Intell. 16(11), 1122–1128 (1994) CrossRefGoogle Scholar
  28. 28.
    Nagel, H., Enkelman, W.: An investigation of smoothness constraints for estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8(5), 565–593 (1986) CrossRefGoogle Scholar
  29. 29.
    Paige, C., Saunders, M.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12, 617–629 (1975) MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Ren, X.: Local grouping for optical flow. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2008) Google Scholar
  31. 31.
    Roth, S., Black, M.: On the spatial statistics of optical flow. Int. J. Comput. Vis. 74(1), 33–50 (2007) CrossRefGoogle Scholar
  32. 32.
    Sfikas, G., Nikou, C., Galatsanos, N.: Edge preserving spatially varying mixtures for image segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska, USA (2008) Google Scholar
  33. 33.
    Sfikas, G., Nikou, C., Galatsanos, N., Heinrich, C.: Spatially varying mixtures incorporating line processes for image segmentation. J. Math. Imaging Vis. 36, 91–110 (2010) MathSciNetCrossRefGoogle Scholar
  34. 34.
    Sfikas, G., Heinrich, C., Zallat, J., Nikou, C., Galatsanos, N.: Recovery of poliarimetric Stokes images by spatial mixture models. J. Opt. Soc. Am. A 28(3), 465–474 (2011) CrossRefGoogle Scholar
  35. 35.
    Sfikas, G., Nikou, C., Galatsanos, N., Heinrich, C.: Majorization-minimization mixture model determination in image segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2169–2176. Colorado Springs, Colorado (2011) Google Scholar
  36. 36.
    Sun, D., Roth, S., Lewis, J.P., Black, M.J.: Learning optical flow. In: Proceedings of the 10th European Conference on Computer Vision (ECCV): Part III, ECCV ’08, pp. 83–97 (2008) Google Scholar
  37. 37.
    Tzikas, D., Likas, A., Galatsanos, N.: The variational approximation for Bayesian inference. IEEE Signal Process. Mag. 25(6), 131–146 (2008) CrossRefGoogle Scholar
  38. 38.
    Wedel, A., Cremers, D., Pock, T., Bischof, H.: Structure- and motion-adaptive regularization for high accuracy optic flow. In: International Conference on Computer Vision (ICCV), pp. 1663–1668 (2009) Google Scholar
  39. 39.
    Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE based computation of image motion. Int. J. Comput. Vis. 45(3), 245–264 (2001) CrossRefMATHGoogle Scholar
  40. 40.
    Weickert, J., Schnörr, C.: Variational optical flow computation with a spatio-temporal smoothness constraint. J. Math. Imaging Vis. 14(3), 245–255 (2001) CrossRefMATHGoogle Scholar
  41. 41.
    Werlberger, M., Pock, T., Bischof, H.: Motion estimation with non-local total variation regularization. In: CVPR, pp. 2464–2471 (2010) Google Scholar
  42. 42.
    Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1744–1757 (2012) CrossRefGoogle Scholar
  43. 43.
    Zhou, Z., Leahy, R.M., Qi, J.: Approximate maximum likelihood hyperparameter estimation for Gibbs priors. IEEE Trans. Image Process. 6(6), 844–861 (1997) CrossRefGoogle Scholar
  44. 44.
    Zimmer, H., Bruhn, A., Weickert, J.: Optic flow in harmony. Int. J. Comput. Vis. 93(3), 368–388 (2011) MathSciNetCrossRefMATHGoogle Scholar
  45. 45.
    Zitnick, C., Jojic, N., Kang, S.: Consistent segmentation for optical flow estimation. In: Proceedings of the International Conference on Computer Vision (ICCV), vol. 2, pp. 1308–1315 (2005) Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Giannis Chantas
    • 1
  • Theodosios Gkamas
    • 2
  • Christophoros Nikou
    • 3
  1. 1.Information Technologies InstituteCentre of Research and Technology, HELLASThessalonikiGreece
  2. 2.Laboratoire des Sciences de l’Ingenieur, de l’Informatique et de l’Imagerie (ICube), CNRS UMR 7357University of StrasbourgStrasbourgFrance
  3. 3.Department of Computer Science and EngineeringUniversity of IoanninaIoanninaGreece

Personalised recommendations