Multiclass Image Labeling with Semidefinite Programming

  • Jens Keuchel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3952)


We propose a semidefinite relaxation technique for multiclass image labeling problems. In this context, we consider labeling as a special case of supervised classification with a predefined number of classes and known but arbitrary dissimilarities between each image element and each class. Using Markov random fields to model pairwise relationships, this leads to a global energy minimization problem. In order to handle its combinatorial complexity, we apply Lagrangian relaxation to derive a semidefinite program, which has several advantageous properties over alternative methods like graph cuts. In particular, there are no restrictions concerning the form of the pairwise interactions, which e.g. allows us to incorporate a basic shape concept into the energy function. Based on the solution matrix of our convex relaxation, a suboptimal solution of the original labeling problem can be easily computed. Statistical ground-truth experiments and several examples of multiclass image labeling and restoration problems show that high quality solutions are obtained with this technique.


Lagrangian Relaxation Image Element Quadratic Assignment Problem Image Label Label Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Winkler, G.: Image Analysis, Random Fields and Dynamic Monte Carlo Methods. Appl. of Mathematics, vol. 27. Springer, Heidelberg (1995)CrossRefzbMATHGoogle Scholar
  2. 2.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. PAMI 6, 721–741 (1984)CrossRefzbMATHGoogle Scholar
  3. 3.
    Besag, J.: On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society, Series B 48, 259–302 (1986)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Li, S.Z.: Markov Random Field Modeling in Image Analysis. Springer, Heidelberg (2001)CrossRefzbMATHGoogle Scholar
  5. 5.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. PAMI 23, 1222–1239 (2001)CrossRefGoogle Scholar
  6. 6.
    Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge (1987)Google Scholar
  7. 7.
    Hofmann, T., Buhmann, J.: Pairwise data clustering by deterministic annealing. IEEE Trans. PAMI 19, 1–14 (1997)CrossRefGoogle Scholar
  8. 8.
    Kleinberg, J., Tardos, E.: Approximation algorithms for classification problems with pairwise relationships: Metric labeling and Markov random fields. Journal of the ACM 49, 616–639 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Keuchel, J., Schnörr, C., Schellewald, C., Cremers, D.: Binary partitioning, perceptual grouping, and restoration with semidefinite programming. IEEE Trans. PAMI 25, 1364–1379 (2003)CrossRefzbMATHGoogle Scholar
  10. 10.
    Zhao, Q., Karisch, S.E., Rendl, F., Wolkowicz, H.: Semidefinite programming relaxations for the quadratic assignment problem. J. Combinat. Optim. 2, 71–109 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Wolkowicz, H., Anjos, M.F.: Semidefinite programming for discrete optimization and matrix completion problems. Discr. Appl. Math. 123, 513–577 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Laurent, M., Rendl, F.: Semidefinite programming and integer programming. In: Aardal, K., Nemhauser, G.L., Weismantel, R. (eds.) Discrete Optimization. Handbooks in Op. Res. and Man. Sci, vol. 12, Elsevier, Amsterdam (2005)CrossRefGoogle Scholar
  13. 13.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Trans. PAMI 26, 147–159 (2004)CrossRefzbMATHGoogle Scholar
  14. 14.
    Rother, C., Kumar, S., Kolmogorov, V., Blake, A.: Digital tapestry. In: Proceedings CVPR, vol. (1), pp. 589–596 (2005)Google Scholar
  15. 15.
    Raj, A., Zabih, R.: A graph cut algorithm for generalized image deconvolution. In: Proc. 10th Int. Conf. Computer Vision (ICCV), pp. 1048–1054 (2005)Google Scholar
  16. 16.
    Wolkowicz, H., Saigal, R., Vandenberghe, L. (eds.): Handbook of Semidefinite Programming. International Series in Operations Research & Management Science, vol. 27. Kluwer Acad. Publ., Boston (2000)zbMATHGoogle Scholar
  17. 17.
    Çela, E.: The Quadratic Assignment Problem: Theory and Algorithms. Kluwer Academic Publishers, Dordrecht (1998)CrossRefzbMATHGoogle Scholar
  18. 18.
    Brixius, N., Anstreicher, K.: Solving quadratic assignment problems using convex quadratic programming relaxations. Optim. Methods Software 16, 49–68 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Alizadeh, F.: Interior point methods in semidefinite programming with applications to combinatorial optimization. SIAM J. Optimization 5, 13–51 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Kočvara, M., Stingl, M.: PENNON - a code for convex nonlinear and semidefinite programming. Optimization Methods and Software 18, 317–333 (2003)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Jens Keuchel
    • 1
  1. 1.Institute of Computational ScienceETH ZurichZurichSwitzerland

Personalised recommendations