Duality and the Continuous Graphical Model

  • Alexander Fix
  • Sameer Agarwal
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8691)


Inspired by the Linear Programming based algorithms for discrete MRFs, we show how a corresponding infinite-dimensional dual for continuous-state MRFs can be approximated by a hierarchy of tractable relaxations. This hierarchy of dual programs includes as a special case the methods of Peng et al. [17] and Zach & Kohli [33]. We give approximation bounds for the tightness of our construction, study their relationship to discrete MRFs and give a generic optimization algorithm based on Nesterov’s dual-smoothing method [16].


  1. 1.
    Bach, S.H., Broecheler, M., Getoor, L., O’Leary, D.P.: Scaling MPE inference for constrained continuous markov random fields with consensus optimization. In: Advances in Neural Information Processing Systems, pp. 2663–2671 (2012)Google Scholar
  2. 2.
    Bertsekas, D.: Nonlinear Programming. Athena Scientific (1995)Google Scholar
  3. 3.
    Besag, J., Besag, J.: On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society B, 48–259 (1986)Google Scholar
  4. 4.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  5. 5.
    Carothers, N.L.: A short course on approximation theory (2009)Google Scholar
  6. 6.
    Crandall, D.J., Owens, A., Snavely, N., Huttenlocher, D.P.: SfM with MRFs: Discrete-continuous optimization for large-scale structure from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(12), 2841–2853 (2013)CrossRefGoogle Scholar
  7. 7.
    Dolecki, S., Kurcyusz, S.: On φ-convexity in extremal problems. SIAM Journal on Control and Optimization 16(2), 277–300 (1978)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6(6), 721–741 (1984)CrossRefzbMATHGoogle Scholar
  9. 9.
    Ihler, A., McAllester, D.: Particle belief propagation. In: Artificial Intelligence and Statistics, pp. 256–263 (2009)Google Scholar
  10. 10.
    Ishikawa, H.: Higher-order gradient descent by fusion-move graph cut. In: IEEE International Conference on Computer Vision, pp. 568–574 (2009)Google Scholar
  11. 11.
    Jojic, V., Gould, S., Koller, D.: Fast and smooth: Accelerated dual decomposition for MAP inference. In: International Conference on Machine Learning (2010)Google Scholar
  12. 12.
    Komodakis, N., Tziritas, G.: Approximate labeling via graph cuts based on linear programming. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(8), 1436–1453 (2007)CrossRefGoogle Scholar
  13. 13.
    Lasserre, J.B.: Global optimization with polynomials and the problem of moments. SIAM Journal on Optimization 11, 796–817 (2001)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Lee, J.: A first course in combinatorial optimization, vol. 36. Cambridge University Press (2004)Google Scholar
  15. 15.
    Lucet, Y.: Faster than the fast Legendre transform, the linear-time Legendre transform. Numerical Algorithms 16(2), 171–185 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Nesterov, Y.: Smooth minimization of non-smooth functions. Mathematical Programming 103(1), 127–152 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Peng, J., Hazan, T., Mcallester, D., Urtasun, R.: Convex max-product algorithms for continuous MRFs with applications to protein folding. In: International Conference on Machine Learning (2011)Google Scholar
  18. 18.
    Rockafellar, R.T., Wets, R.J.B., Wets, M.: Variational analysis, vol. 317. Springer (1998)Google Scholar
  19. 19.
    Rockafellar, R.: Conjugate Duality and Optimization. Society for Industrial and Applied Mathematics (1974)Google Scholar
  20. 20.
    Rockafellar, R.: Convex Analysis. Princeton University Press (1997)Google Scholar
  21. 21.
    Savchynskyy, B., Schmidt, S., Schnrr, C.: Efficient MRF energy minimization via adaptive diminishing smoothing. In: Uncertanity in Artificial Intelligence (2012)Google Scholar
  22. 22.
    Shimony, S.E.: Finding MAPs for belief networks is NP-hard. Artificial Intelligence 68(2), 399–410 (1994)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Sigal, L., Bhatia, S., Roth, S., Black, M., Isard, M.: Tracking loose-limbed people. In: IEEE Conference on Computer Vision and Pattern Recognition (2004)Google Scholar
  24. 24.
    Sudderth, E.B., Michael, I.M., Freeman, W.T., Willsky, A.S.: Distributed occlusion reasoning for tracking with nonparametric belief propagation. In: Advances in Neural Information Processing Systems, pp. 1369–1376 (2004)Google Scholar
  25. 25.
    Sun, J., Zheng, N.N., Shum, H.Y.: Stereo matching using belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(7), 787–800 (2003)CrossRefGoogle Scholar
  26. 26.
    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) Vision Algorithms 1999. LNCS, vol. 1883, pp. 298–372. Springer, Heidelberg (2000)Google Scholar
  27. 27.
    Trinh, H., McAllester, D.: Particle-based belief propagation for structure from motion and dense stereo vision with unknown camera constraints. In: Sommer, G., Klette, R. (eds.) RobVis 2008. LNCS, vol. 4931, pp. 16–28. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  28. 28.
    Vazirani, V.V.: Approximation algorithms. Springer (2001)Google Scholar
  29. 29.
    Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1(1-2), 1–305 (2008)zbMATHGoogle Scholar
  30. 30.
    Weiss, Y., Freeman, W.T.: On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs. IEEE Trans. Inf. Theor. 47(2), 736–744 (2006)CrossRefMathSciNetGoogle Scholar
  31. 31.
    Werner, T.: A linear programming approach to max-sum problem: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(7), 1165–1179 (2007)CrossRefGoogle Scholar
  32. 32.
    Yamaguchi, K., Hazan, T., McAllester, D., Urtasun, R.: Continuous Markov random fields for robust stereo estimation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 45–58. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  33. 33.
    Zach, C., Kohli, P.: A convex discrete-continuous approach for Markov random fields. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 386–399. Springer, Heidelberg (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Alexander Fix
    • 1
  • Sameer Agarwal
    • 2
  1. 1.Cornell UniversityUSA
  2. 2.Google Inc.USA

Personalised recommendations