Skip to main content
SpringerLink
Log in

Augmented Lagrangian Method for Total Variation Based Image Restoration and Segmentation Over Triangulated Surfaces

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Recently total variation (TV) regularization has been proven very successful in image restoration and segmentation. In image restoration, TV based models offer a good edge preservation property. In image segmentation, TV (or vectorial TV) helps to obtain convex formulations of the problems and thus provides global minimizations. Due to these advantages, TV based models have been extended to image restoration and data segmentation on manifolds. However, TV based restoration and segmentation models are difficult to solve, due to the nonlinearity and non-differentiability of the TV term. Inspired by the success of operator splitting and the augmented Lagrangian method (ALM) in 2D planar image processing, we extend the method to TV and vectorial TV based image restoration and segmentation on triangulated surfaces, which are widely used in computer graphics and computer vision. In particular, we will focus on the following problems. First, several Hilbert spaces will be given to describe TV and vectorial TV based variational models in the discrete setting. Second, we present ALM applied to TV and vectorial TV image restoration on mesh surfaces, leading to efficient algorithms for both gray and color image restoration. Third, we discuss ALM for vectorial TV based multi-region image segmentation, which also works for both gray and color images. The proposed method benefits from fast solvers for sparse linear systems and closed form solutions to subproblems. Experiments on both gray and color images demonstrate the efficiency of our algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MATH  Google Scholar 

  2. Sapiro, G., Ringach, D.: Anisotropic diffusion of multivalued images with applications to color filtering. IEEE Trans. Image Process. 5(11), 1582–1586 (1996)

    Article  Google Scholar 

  3. Blomgren, P., Chan, T.: Color tv: total variation methods for restoration of vector-valued images. IEEE Trans. Image Process. 7(3), 304–309 (1998)

    Article  Google Scholar 

  4. Chan, T., Kang, S., Shen, J.: Total variation denoising and enhancement of color images based on the CB and HSV color models. J. Vis. Commun. Image Rep. 12, 422–435 (2001)

    Article  Google Scholar 

  5. Bresson, X., Chan, T.: Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Problems and Imaging 2(4), 455–484 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  6. Chan, T., Golub, G., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Sci. Comput. 20, 1964–1977 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  7. Carter, J.: Dual methods for total variation based image restoration. Ph.D. thesis (2001)

  8. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vision 20, 89–97 (2004)

    Article  MathSciNet  Google Scholar 

  9. Zhu, M., Wright, S., Chan, T.: Duality-based algorithms for total variation image restoration. Tech. Rep. 08-33 (2008)

  10. Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1, 248–272 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  11. Yang, J., Yin, W., Zhang, Y., Wang, Y.: A fast algorithm for edge-preserving variational multichannel image restoration. Tech. Rep. 08-50 (2008)

  12. Huang, Y., Ng, M., Wen, Y.: A fast total variation minimization method for image restoration. SIAM Multiscale Model. Simul. 7, 774–795 (2009)

    Article  MathSciNet  Google Scholar 

  13. Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for compressend sensing and related problems. SIAM J. Imaging Sci. 1, 143–168 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  14. Goldstein, T., Osher, S.: The split bregman method for l1 regularized problems. SIAM J. Imaging Sci. 2, 323–343 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  15. Tai, X.C., Wu, C.: Augmented lagrangian method, dual methods and split bregman iteration for rof model. In: Proc. Scale Space and Variational Methods in Computer Vision, Second International Conference (SSVM) 2009, pp. 502–513 (2009)

    Chapter  Google Scholar 

  16. Weiss, P., Blanc-Fraud, L., Aubert, G.: Efficient schemes for total variation minimization under constraints in image processing. SIAM J. Sci. Comput. 31, 2047–2080 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  17. Wu, C., Tai, X.C.: Augmented lagrangian method dual methods, and split bregman iteration for rof, vectorial tv, and high order models. SIAM J. Imaging Sci. 3(3), 300–339 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  18. Wu, C., Zhang, J., Tai, X.C.: Augmented lagrangian method for total variation restoration with non-quadratic fidelity. Inverse Problems and Imaging 5(1), 237–261 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  19. Zhang, X., Burger, M., Osher, S.: A unified primal-dual algorithm framework based on bregman iteration. J. Sci. Comput. 46, 20–46 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  20. Michailovich, O.: An iterative shrinkage approach to total-variation image restoration. IEEE Trans. Image Process. (2011)

  21. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2, 183–202 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  22. Chan, R., Chan, T., Shen, L., Shen, Z.: Wavelet algorithms for high-resolution image reconstruction. SIAM J. Sci. Comput. 24, 1408–1432 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  23. Daubechies, I., Defrise, M., Mol, C.D.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm. Pure and Appl. Math. 57, 1413–1457 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  24. Figueiredo, M., Nowak, R.: An em algorithm for wavelet-based image restoration. IEEE Trans. Image Process. 12, 906–916 (2003)

    Article  MathSciNet  Google Scholar 

  25. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int’l. J. Comput. Vis. 1, 321–331 (1988)

    Article  Google Scholar 

  26. Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int’l J. Comput. Vision 22, 61–79 (1997)

    Article  MATH  Google Scholar 

  27. Goldenberg, R., Kimmel, R., Rivlin, E., Rudzsky, M.: Fast geodesic active contours. IEEE Trans. Image Process. 10, 1467–1475 (2001)

    Article  MathSciNet  Google Scholar 

  28. Chan, T., Vese, L.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)

    Article  MATH  Google Scholar 

  29. Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Springer, Berlin (2002)

    Google Scholar 

  30. Lie, J., Lysaker, M., Tai, X.: A variant of the level set method and applications in image segmentation. Math. Comp. 75, 1155–1174 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  31. Vese, L., Chan, T.: A multiphase level set framework for image segmentation using the mumford-shah model. Int’l J. Comput. Vision 50(3), 271–293 (2002)

    Article  MATH  Google Scholar 

  32. Cremers, D., Tischhauser, F., Weickert, J., Schnorr, C.: Diffusion snakes: introducing statistical shape knowledge into the mumfordc̈shah functional. Int’l J. Computer Vision 50(3), 295–313 (2002)

    Article  MATH  Google Scholar 

  33. Lie, J., Lysaker, M., Tai, X.: A binary level set model and some applications to mumford-shah image segmentation. IEEE Trans. Image Process. 15, 1171–1181 (2006)

    Article  Google Scholar 

  34. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math. 42, 577–685 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  35. Appleton, B., Talbot, H.: Globally minimal surfaces by continuous maximal flows. IEEE Trans. Pattern Anal. Mach. Intell. (1) 28, 106–118 (2006)

    Article  Google Scholar 

  36. Nikolova, M., Esedoglu, S., Chan, T.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66(5), 1632–1648 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  37. Bresson, X., Esedoglu, S., Vandergheynst, P., Thiran, J., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imaging Vision 28(2), 151–167 (2007)

    Article  MathSciNet  Google Scholar 

  38. Zach, C., Gallup, D., Frahm, J.M., Niethammer, M.: Fast global labeling for real-time stereo using multiple plane sweeps. In: Proc. Vision, Modeling and Visualization Workshop (VMV) (2008)

    Google Scholar 

  39. Pock, T., Schoenemann, T., Graber, G., Bischof, H., Cremers, D.: A convex formulation of continuous multi-label problems. In: Proc. European Conference on Computer Vision (ECCV 2008), pp. III, pp. 792–805 (2008)

    Google Scholar 

  40. Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the mumford-shah functional. In: Proc. ICCV (2009)

    Google Scholar 

  41. Pock, T., Chambolle, A., Cremers, D., Bischof, H.: A convex relaxation approach for computing minimal partitions. In: Proc. CVPR (2009)

    Google Scholar 

  42. Lellmann, J., Kappes, J., Yuan, J., Becker, F., Schnörr, C.: Convex multi-class image labeling by simplex-constrained total variation. In: Proc. Second International Conference on Scale Space and Variational Methods in Computer Vision (SSVM 2009), pp. 150–162. Springer, Berlin (2009)

    Chapter  Google Scholar 

  43. Bae, E., Yuan, J., Tai, X.: Global minimization for continuous multiphase partitioning problems using a dual approach. Tech. rep. (2009). URL: ftp://ftp.math.ucla.edu/pub/camreport/cam09-75.pdf

  44. Brown, E., Chan, T., Bresson, X.: A convex approach for multi-phase piecewise constant mumford-shah image segmentation. Tech. rep. (2009). URL: ftp://ftp.math.ucla.edu/pub/camreport/cam09-66.pdf

  45. Goldstein, T., Bresson, X., Osher, S.: Geometric applications of the split bregman method: segmentation and surface reconstruction. J. Sci. Comput. 45, 272–293 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  46. Brown, E., Chan, T., Bresson, X.: A convex relaxation method for a class of vector-valued minimization problems with applications to mumford-shah segmentation. Tech. rep. (2010). URL: ftp://ftp.math.ucla.edu/pub/camreport/cam10-43.pdf

  47. Brown, E., Chan, T., Bresson, X.: Globally convex chan-vese image segmentation. Tech. rep. (2010). URL: ftp://ftp.math.ucla.edu/pub/camreport/cam10-44.pdf

  48. Lellmann, J., Becker, F., Schnörr, C.: Convex optimization for multi-class image labeling with a novel family of total variation based regularizers. In: Proc. IEEE International Conference on Computer Vision (ICCV), pp. 646–653 (2009)

    Google Scholar 

  49. Lellmann, J., Schnoerr, C.: Continuous multiclass labeling approaches and algorithms. Tech. rep., Univ. of Heidelberg (2010). URL: http://www.ub.uni-heidelberg.de/archiv/10460/

  50. Wu, C., Deng, J., Chen, F.: Diffusion equations over arbitrary triangulated surfaces for filtering and texture applications. IEEE Trans. Visual. Comput. Graph. 14(3), 666–679 (2008)

    Article  Google Scholar 

  51. Wu, C., Deng, J., Chen, F., Tai, X.: Scale-space analysis of discrete filtering over arbitrary triangulated surfaces. SIAM J. Imaging Sci. 2(2), 670–709 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  52. Delaunoy, A., Fundana, K., Prados, E., Heyden, A.: Convex multi-region segmentation on manifolds. In: Proc. 12th IEEE International Conference on Computer Vision (ICCV), pp. 662–669 (2009)

    Chapter  Google Scholar 

  53. Lai, R., Chan, T.: A framework for intrinsic image processing on surfaces. Tech. Rep. 10-25 (2010). URL: ftp://ftp.math.ucla.edu/pub/camreport/cam10-25.pdf

  54. Hirani, A.: Discrete exterior calculus. Ph.D. thesis, California Institute of Technology (2003)

  55. Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of rn. J. Optim. Theory Appl. 50(1), 195–200 (1986)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, NUS, Block S17, 10, Lower Kent Ridge Road, Singapore, 119076, Singapore

    Chunlin Wu

  2. School of Computer Engineering, Nanyang Technological University, Singapore, Singapore

    Juyong Zhang

  3. MAS, SPMS, Nanyang Technological University, Singapore, Singapore

    Yuping Duan & Xue-Cheng Tai

  4. University of Bergen, Bergen, Norway

    Xue-Cheng Tai

Authors
  1. Chunlin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juyong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuping Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xue-Cheng Tai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunlin Wu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wu, C., Zhang, J., Duan, Y. et al. Augmented Lagrangian Method for Total Variation Based Image Restoration and Segmentation Over Triangulated Surfaces. J Sci Comput 50, 145–166 (2012). https://doi.org/10.1007/s10915-011-9477-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-011-9477-3

Keywords

Search

Navigation