Sublabel-Accurate Convex Relaxation with Total Generalized Variation Regularization

  • Michael StreckeEmail author
  • Bastian Goldluecke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11269)


We propose a novel idea to introduce regularization based on second order total generalized variation (\(\text {TGV}\)) into optimization frameworks based on functional lifting. The proposed formulation extends a recent sublabel-accurate relaxation for multi-label problems and thus allows for accurate solutions using only a small number of labels, significantly improving over previous approaches towards lifting the total generalized variation. Moreover, even recent sublabel accurate methods exhibit staircasing artifacts when used in conjunction with common first order regularizers such as the total variation (\(\text {TV}\)). This becomes very obvious for example when computing derivatives of disparity maps computed with these methods to obtain normals, which immediately reveals their local flatness and yields inaccurate normal maps. We show that our approach is effective in reducing these artifacts, obtaining disparity maps with a smooth normal field in a single optimization pass.



This work was supported by the ERC Starting Grant “Light Field Imaging and Analysis” (LIA 336978, FP7-2014) and the SFB Transregio 161 “Quantitative Methods for Visual Computing”.


  1. 1.
    Alberti, G., Bouchitté, G., Dal Maso, G.: The calibration method for the Mumford-Shah functional and free-discontinuity problems. Calc. Var. Partial Differ. Equ. 16(3), 299–333 (2003)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge (1987)Google Scholar
  3. 3.
    Bouchitté, G.: Recent convexity arguments in the calculus of variations. In: Lecture Notes from the 3rd International Summer School on the Calculus of Variations, Pisa (1998)Google Scholar
  4. 4.
    Boykov, Y., Veksler, O., Zabih, R.: Markov random fields with efficient approximations. In: Proceedings of International Conference on Computer Vision and Pattern Recognition, Santa Barbara, California, pp. 648–655 (1998)Google Scholar
  5. 5.
    Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chambolle, A.: Convex representation for lower semicontinuous envelopes of functionals in \(L^1\). J. Convex Anal. 8(1), 149–170 (2001)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Cremers, D., Strekalovskiy, E.: Total cyclic variation and generalizations. J. Math. Imaging Vis. 47(3), 258–277 (2012)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Ferstl, D., Reinbacher, C., Ranftl, R., Rüther, M., Bischof, H.: Image guided depth upsampling using anisotropic total generalized variation. In: Proceedings of International Conference on Computer Vision, pp. 993–1000. IEEE (2013)Google Scholar
  9. 9.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)CrossRefGoogle Scholar
  10. 10.
    Goldluecke, B., Strekalovskiy, E., Cremers, D.: Tight convex relaxations for vector-valued labeling. SIAM J. Imaging Sci. 6(3), 1626–1664 (2013)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Honauer, K., Johannsen, O., Kondermann, D., Goldluecke, B.: A dataset and evaluation methodology for depth estimation on 4D light fields. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10113, pp. 19–34. Springer, Cham (2017). Scholar
  12. 12.
    Ishikawa, H.: Exact optimization for Markov random fields with convex priors. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1333–1336 (2003)CrossRefGoogle Scholar
  13. 13.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)CrossRefGoogle Scholar
  14. 14.
    Lellmann, J., Becker, F., Schnörr, C.: Convex optimization for multi-class image labeling with a novel family of total variation based regularizers. In: IEEE International Conference on Computer Vision (ICCV) (2009)Google Scholar
  15. 15.
    Moellenhoff, T., Laude, E., Moeller, M., Lellmann, J., Cremers, D.: Sublabel-accurate relaxation of nonconvex energies. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  16. 16.
    Möllenhoff, T., Cremers, D.: Sublabel-accurate discretization of nonconvex free-discontinuity problems. In: International Conference on Computer Vision (ICCV), Venice, Italy, October 2017Google Scholar
  17. 17.
    Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42, 577–685 (1989)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Pock, T., Cremers, D., Bischof, H., Chambolle, A.: Global solutions of variational models with convex regularization. SIAM J. Imaging Sci. 3, 1122–1145 (2010)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Ranftl, R., Gehrig, S., Pock, T., Bischof, H.: Pushing the limits of stereo using variational stereo estimation. In: IEEE Intelligent Vehicles Symposium (2012)Google Scholar
  20. 20.
    Ranftl, R., Pock, T., Bischof, H.: Minimizing TGV-based variational models with non-convex data terms. In: Kuijper, A., Bredies, K., Pock, T., Bischof, H. (eds.) SSVM 2013. LNCS, vol. 7893, pp. 282–293. Springer, Heidelberg (2013). Scholar
  21. 21.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60, 259–268 (1992)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Strecke, M., Alperovich, A., Goldluecke, B.: Accurate depth and normal maps from occlusion-aware focal stack symmetry. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2017)Google Scholar
  23. 23.
    Zach, C., Häne, C., Pollefeys, M.: What is optimized in convex relaxations for multilabel problems: connecting discrete and continuously inspired map inference. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 157–170 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of KonstanzKonstanzGermany

Personalised recommendations