Journal of Scientific Computing

, Volume 76, Issue 1, pp 243–274 | Cite as

Convergence Analysis of Primal–Dual Based Methods for Total Variation Minimization with Finite Element Approximation

  • WenYi Tian
  • Xiaoming Yuan


We consider a minimization model with total variational regularization, which can be reformulated as a saddle-point problem and then be efficiently solved by the primal–dual method. We utilize the consistent finite element method to discretize the saddle-point reformulation; thus possible jumps of the solution can be captured over some adaptive meshes and a generic domain can be easily treated. Our emphasis is analyzing the convergence of a more general primal–dual scheme with a combination factor for the discretized model. We establish the global convergence and derive the worst-case convergence rate measured by the iteration complexity for this general primal–dual scheme. This analysis is new in the finite element context for the minimization model with total variational regularization under discussion. Furthermore, we propose a prediction–correction scheme based on the general primal–dual scheme, which can significantly relax the step size for the discretization in the time direction. Its global convergence and the worst-case convergence rate are also established. Some preliminary numerical results are reported to verify the rationale of considering the general primal–dual scheme and the primal–dual-based prediction–correction scheme.


Total variation minimization Saddle-point problem Finite element method Primal–dual method Convergence rate 


  1. 1.
    Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Probl. 10(6), 12171229 (1994)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Allen, S.M., Cahn, J.W.: A microscopic theory for antiphase boundary motion and its application to antiphase domain coarsening. Acta Metall. 27(6), 1085–1095 (1979)CrossRefGoogle Scholar
  3. 3.
    Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems. Clarendon Press, Oxford (2000)MATHGoogle Scholar
  4. 4.
    Andreu, F., Ballester, C., Caselles, V., Mazón, J.M.: Minimizing total variation flow. Differ. Integral Equ. 14(3), 321–360 (2001)MathSciNetMATHGoogle Scholar
  5. 5.
    Arrow, K.J., Hurwicz, L., Uzawa, H.: Studies in Linear and Non-linear Programming. Stanford University Press, Stanford (1958)MATHGoogle Scholar
  6. 6.
    Attouch, H., Buttazzo, G., Michaille, G.: Variational Analysis in Sobolev and BV Spaces: Applications to PDEs and Optimization. SIAM/MPS, Philadelphia (2006)CrossRefMATHGoogle Scholar
  7. 7.
    Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd edn. Springer, New York (2006)MATHGoogle Scholar
  8. 8.
    Bartels, S.: Total variation minimization with finite elements: convergence and iterative solution. SIAM J. Numer. Anal. 50(3), 1162–1180 (2012)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Bartels, S.: Broken Sobolev space iteration for total variation regularized minimization problems. IMA J. Numer. Anal. 36(2), 493–502 (2016)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Bartels, S., Nochetto, R.H., Salgado, A.J.: Discrete total variation flows without regularization. SIAM J. Numer. Anal. 52(1), 363–385 (2014)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Bellettini, G., Caselles, V., Novaga, M.: The total variation flow in \(\mathbb{R}^n\). J. Differ. Equ. 184(2), 475–525 (2002)CrossRefMATHGoogle Scholar
  12. 12.
    Blum, E., Oettli, W.: Mathematische Optimierung. Springer, Berlin (1975)CrossRefMATHGoogle Scholar
  13. 13.
    Boţ, R.I., Csetnek, E.R.: On the convergence rate of a forward-backward type primal-dual splitting algorithm for convex optimization problems. Optimization 64(1), 5–23 (2015)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Bramble, J.H., Pasciak, J.E., Vassilev, A.T.: Analysis of the inexact Uzawa algorithm for saddle point problems. SIAM J. Numer. Anal. 34(3), 1072–1092 (1997)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Brenner, S.C., Scott, L.R.: The Mathematical Theory of Finite Element Methods, 3rd edn. Springer, New York (2008)CrossRefMATHGoogle Scholar
  16. 16.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004)MathSciNetMATHGoogle Scholar
  17. 17.
    Chambolle, A., Lions, P.-L.: Image recovery via total variation minimization and related problems. Numer. Math. 76(2), 167–188 (1997)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Chan, R.H., Chen, K.: A multilevel algorithm for simultaneously denoising and deblurring images. SIAM J. Sci. Comput. 32(2), 1043–1063 (2010)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Chan, T.F., Mulet, P.: On the convergence of the lagged diffusivity fixed point method in total variation image restoration. SIAM J. Numer. Anal. 36(2), 354–367 (1999)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Chan, T.F., Tai, X.-C.: Identification of discontinuous coefficients in elliptic problems using total variation regularization. SIAM J. Sci. Comput. 25(3), 881–904 (2003)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Chan, T.F., Golub, G.H., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Sci. Comput. 20(6), 1964–1977 (1999)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Chavent, G., Kunisch, K.: Regularization of linear least squares problems by total bounded variation. ESAIM Control Optim. Calc. Var. 2, 359–376 (1997)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Chen, Z., Zou, J.: An augmented Lagrangian method for identifying discontinuous parameters in elliptic systems. SIAM J. Control Optim. 37(3), 892–910 (1999)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Dobson, D., Scherzer, O.: Analysis of regularized total variation penalty methods for denoising. Inverse Probl. 12(5), 601–617 (1996)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Dobson, D.C., Santosa, F.: An image-enhancement technique for electrical impedance tomography. Inverse Probl. 10(2), 317–334 (1994)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Dobson, D.C., Santosa, F.: Recovery of blocky images from noisy and blurred data. SIAM J. Appl. Math. 56(4), 1181–1198 (1996)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Dobson, D.C., Vogel, C.R.: Convergence of an iterative method for total variation denoising. SIAM J. Numer. Anal. 34(5), 1779–1791 (1997)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Elliott, C.M., Smitheman, S.A.: Numerical analysis of the TV regularization and \({H}^{-1}\) fidelity model for decomposing an image into cartoon plus texture. IMA J. Numer. Anal. 29(3), 651–689 (2008)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Elman, H.C., Golub, G.H.: Inexact and preconditioned Uzawa algorithms for saddle point problems. SIAM J. Numer. Anal. 31(6), 1645–1661 (1994)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Esser, E., Zhang, X., Chan, T.F.: A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM J. Imaging Sci. 3(4), 1015–1046 (2010)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. I. Springer, New York (2003)MATHGoogle Scholar
  33. 33.
    Feng, X.B., Prohl, A.: Analysis of total variation flow and its finite element approximations. ESAIM Math. Model. Numer. Anal. 37(3), 533–556 (2003)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Feng, X., von Oehsen, M., Prohl, A.: Rate of convergence of regularization procedures and finite element approximations for the total variation flow. Numer. Math. 100(3), 441–456 (2005)MathSciNetCrossRefMATHGoogle Scholar
  35. 35.
    Fortin, M., Glowinski, R.: Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary Value Problems. North-Holland Publishing Co., Amsterdam (1983)MATHGoogle Scholar
  36. 36.
    Glowinski, R.: Numerical Methods for Nonlinear Variational Problems. Springer, New York (1984)CrossRefMATHGoogle Scholar
  37. 37.
    He, B.S., Yuan, X.M.: Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective. SIAM J. Imaging Sci. 5(1), 119–149 (2012)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    He, B.S., Yuan, X.M.: On the \(O(1/n)\) convergence rate of the Douglas–Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    He, B.S., Liu, H., Wang, Z., Yuan, X.M.: A strictly contractive Peaceman–Rachford splitting method for convex programming. SIAM J. Optim. 24(3), 1011–1040 (2014)MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Keung, Y.L., Zou, J.: Numerical identifications of parameters in parabolic systems. Inverse Probl. 14(1), 83–100 (1998)MathSciNetCrossRefMATHGoogle Scholar
  41. 41.
    Li, B., Sun, W.: Linearized FE approximations to a nonlinear gradient flow. SIAM J. Numer. Anal. 52(6), 2623–2646 (2014)MathSciNetCrossRefMATHGoogle Scholar
  42. 42.
    Marquina, A., Osher, S.: Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM J. Sci. Comput. 22(2), 387–405 (2000)MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    Nemirovsky, A.S., Yudin, D.B.: Problem Complexity and Method Efficiency in Optimization. Wiley, New York (1983)Google Scholar
  44. 44.
    Nesterov, Y.: Gradient methods for minimizing composite functions. Math. Program. 140(1), 125–161 (2013)MathSciNetCrossRefMATHGoogle Scholar
  45. 45.
    Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate \(O(1/k^{2})\). Dokl. Akad. Nauk SSSR 269(3), 543–547 (1983). (in Russian. Translated in Soviet Math. Dokl., 27 (1983), pp. 372–376.)MathSciNetGoogle Scholar
  46. 46.
    Ng, M.K., Qi, L., Yang, Y.-F., Huang, Y.-M.: On semismooth Newton-methods for total variation minimization. J. Math. Imaging Vis. 27(3), 265–276 (2007)MathSciNetCrossRefGoogle Scholar
  47. 47.
    Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variation-based image restoration. Multiscale Model. Simul. 4(2), 460–489 (2005)MathSciNetCrossRefMATHGoogle Scholar
  48. 48.
    Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1762–1769 (2011)Google Scholar
  49. 49.
    Queck, W.: The convergence factor of preconditioned algorithms of the Arrow–Hurwicz type. SIAM J. Numer. Anal. 26(4), 1016–1030 (1989)MathSciNetCrossRefMATHGoogle Scholar
  50. 50.
    Ring, W.: Structural properties of solutions to total variation regularization problems. ESAIM Math. Model. Numer. Anal. 34(04), 799–810 (2000)MathSciNetCrossRefMATHGoogle Scholar
  51. 51.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)MathSciNetCrossRefMATHGoogle Scholar
  52. 52.
    Sapiro, G., Caselles, V.: Histogram modification via differential equations. J. Differ. Equ. 135(2), 238–268 (1997)MathSciNetCrossRefMATHGoogle Scholar
  53. 53.
    Strong, D., Chan, T.: Edge-preserving and scale-dependent properties of total variation regularization. Inverse Probl. 19(6), 165–187 (2003)MathSciNetCrossRefMATHGoogle Scholar
  54. 54.
    Vogel, C.R., Oman, M.E.: Iterative methods for total variation denoising. SIAM J. Sci. Comput. 17(1), 227–238 (1996)MathSciNetCrossRefMATHGoogle Scholar
  55. 55.
    Zhang, J., Chen, K., Yu, B.: An iterative Lagrange multiplier method for constrained total-variation-based image denoising. SIAM J. Numer. Anal. 50(3), 983–1003 (2012)MathSciNetCrossRefMATHGoogle Scholar
  56. 56.
    Zhang, X., Burger, M., Osher, S.: A unified primal-dual algorithm framework based on Bregman iteration. J. Sci. Comput. 46(1), 20–46 (2011)MathSciNetCrossRefMATHGoogle Scholar
  57. 57.
    Zhu, M., Chan, T.F.: An efficient primal-dual hybrid gradient algorithm for total variation image restoration. CAM Report 08-34, UCLA, Los Angeles, CA (2008)Google Scholar
  58. 58.
    Ziemer, W.P.: Weakly Differentiable Functions: Sobolev Spaces and Functions of Bounded Variation. Springer, New York (1989)CrossRefMATHGoogle Scholar
  59. 59.
    Zulehner, W.: Analysis of iterative methods for saddle point problems: a unified approach. Math. Comput. 71(238), 479–505 (2002)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Center for Applied MathematicsTianjin UniversityTianjinChina
  2. 2.Department of MathematicsHong Kong Baptist UniversityHong KongChina
  3. 3.Department of MathematicsThe University of Hong KongHong KongChina

Personalised recommendations