Energy Minimization under Constraints on Label Counts

  • Yongsub Lim
  • Kyomin Jung
  • Pushmeet Kohli
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6312)


Many computer vision problems such as object segmentation or reconstruction can be formulated in terms of labeling a set of pixels or voxels. In certain scenarios, we may know the number of pixels or voxels which can be assigned to a particular label. For instance, in the reconstruction problem, we may know size of the object to be reconstructed. Such label count constraints are extremely powerful and have recently been shown to result in good solutions for many vision problems.

Traditional energy minimization algorithms used in vision cannot handle label count constraints. This paper proposes a novel algorithm for minimizing energy functions under constraints on the number of variables which can be assigned to a particular label. Our algorithm is deterministic in nature and outputs ε-approximate solutions for all possible counts of labels. We also develop a variant of the above algorithm which is much faster, produces solutions under almost all label count constraints, and can be applied to all submodular quadratic pseudoboolean functions. We evaluate the algorithm on the two-label (foreground/back-ground) image segmentation problem and compare its performance with the state-of-the-art parametric maximum flow and max-sum diffusion based algorithms. Experimental results show that our method is practical and is able to generate impressive segmentation results in reasonable time.


Energy Minimization Energy Function Grid Graph Labelling Problem Graph Decomposition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-642-15552-9_39_MOESM1_ESM.blg (0 kb)
Electronic Supplementary Material (1 KB)


  1. 1.
    Chekuri, C., Khanna, S., Naor, J., Zosin, L.: A linear programming formulation and approximation algorithms for the metric labelling problem. SIAM Journal on Discrete Mathematics (2005)Google Scholar
  2. 2.
    Komodakis, N., Tziritas, G., Paragios, N.: Fast, approximately optimal solutions for single and dynamic MRFs. In: CVPR (2007)Google Scholar
  3. 3.
    Kumar, M.P., Koller, D.: MAP estimation of semi-metric MRFs via hierarchical graph cuts. In: UAI (2009)Google Scholar
  4. 4.
    Sontag, D., Meltzer, T., Globerson, A., Jaakkola, T., Weiss, Y.: Tightening LP relaxations for MAP using message passing. In: UAI (2008)Google Scholar
  5. 5.
    Werner, T.: A linear programming approach to max-sum problem: A review. PAMI (2007)Google Scholar
  6. 6.
    Weiss, Y., Yanover, C., Meltzer, T.: MAP estimation, linear programming and belief propagation with convex free energies. In: UAI (2007)Google Scholar
  7. 7.
    Yedidia, J., Freeman, W., Weiss, Y.: Generalized belief propagation. In: NIPS (2001)Google Scholar
  8. 8.
    Wainwright, M., Jaakkola, T., Willsky, A.: MAP estimation via agreement on trees: message-passing and linear programming. IEEE Transactions on Information Theory (2005)Google Scholar
  9. 9.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. PAMI (2001)Google Scholar
  10. 10.
    Kolev, K., Cremers, D.: Integration of multiview stereo and silhouettes via convex functionals on convex domains. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 752–765. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Sinha, S., Pollefeys, M.: Multi-view reconstruction using photo-consistency and exact silhouette constraints: A maximum-flow formulation. In: ICCV (2005)Google Scholar
  12. 12.
    Vogiatzis, G., Torr, P., Cipolla, R.: Multi-view stereo via volumetric graph-cuts. In: CVPR (2005)Google Scholar
  13. 13.
    Boykov, Y., Jolly, M.: Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In: ICCV (2001)Google Scholar
  14. 14.
    Vicente, S., Kolmogorov, V., Rother, C.: Graph cut based image segmentation with connectivity priors. In: CVPR (2008)Google Scholar
  15. 15.
    Naor, J., Schwartz, R.: Balanced metric labeling. In: STOC (2005)Google Scholar
  16. 16.
    Werner, T.: High-arity interactions, polyhedral relaxations, and cutting plane algorithm for soft contraint optimisation (map-mrf). In: CVPR (2008)Google Scholar
  17. 17.
    Woodford, O., Rother, C., Kolmogorov, V.: A global perspective on MAP inference for low-level vision. In: ICCV (2009)Google Scholar
  18. 18.
    Kolmogorov, V., Boykov, Y., Rother, C.: Application of parametric maxflow in computer vision. In: ICCV (2007)Google Scholar
  19. 19.
    Gallo, G., Grigoriadis, M., Tarjan, R.: A fast parametric maximum flow algorithm and applications. SIAM J. on Comput. 18, 30–55 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Garey, M., Johnson, D.S.: Computers and intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman, New York (1979)Google Scholar
  21. 21.
    Goldberg, A., Tarjan, R.: A new approach to the maximum-flow problem. Journal of the Association for Computing Machinery (1988)Google Scholar
  22. 22.
    Kohli, P.: Minimizing dynamic and higher order energy functions using graph cuts (2007)Google Scholar
  23. 23.
    Jung, K., Shah, D.: Local algorithms for approximate inference in minor-excluded graphs. In: NIPS (2007)Google Scholar
  24. 24.
    Blake, A., Rother, C., Brown, M., Pérez, P., Torr, P.: Interactive image segmentation using an adaptive GMMRF model. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 428–441. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  25. 25.
    Rhemann, C., Rother, C., Rav-Acha, A., Sharp, T.: High resolution matting via interactive trimap segmentation. In: CVPR (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Yongsub Lim
    • 1
  • Kyomin Jung
    • 1
  • Pushmeet Kohli
    • 2
  1. 1.Korea Advanced Institute of Science and TechnologyDaejeonKorea
  2. 2.Microsoft ResearchCambridgeUnited Kingdom

Personalised recommendations