Numerische Mathematik

, Volume 141, Issue 1, pp 215–252 | Cite as

Higher order convergence rates for Bregman iterated variational regularization of inverse problems

  • Benjamin SprungEmail author
  • Thorsten Hohage


We study the convergence of variationally regularized solutions to linear ill-posed operator equations in Banach spaces as the noise in the right hand side tends to 0. The rate of this convergence is determined by abstract smoothness conditions on the solution called source conditions. For non-quadratic data fidelity or penalty terms such source conditions are often formulated in the form of variational inequalities. Such variational source conditions (VSCs) as well as other formulations of such conditions in Banach spaces have the disadvantage of yielding only low-order convergence rates. A first step towards higher order VSCs has been taken by Grasmair (J Inverse Ill-Posed Probl 21(3):379–394, 2013. who obtained convergence rates up to the saturation of Tikhonov regularization. For even higher order convergence rates, iterated versions of variational regularization have to be considered. In this paper we introduce VSCs of arbitrarily high order which lead to optimal convergence rates in Hilbert spaces. For Bregman iterated variational regularization in Banach spaces with general data fidelity and penalty terms, we derive convergence rates under third order VSC. These results are further discussed for entropy regularization with elliptic pseudodifferential operators where the VSCs are interpreted in terms of Besov spaces and the optimality of the rates can be demonstrated. Our theoretical results are confirmed in numerical experiments.

Mathematics Subject Classification

65J20 65J22 



We would like to thank two anonymous referees for their comments, which helped to improve the paper. Financial support by Deutsche Forschungsgemeinschaft through Grant CRC 755, Project C09, and RTG 2088 is gratefully acknowledged.


  1. 1.
    Albani, V., Elbau, P., de Hoop, M.V., Scherzer, O.: Optimal convergence rates results for linear inverse problems in Hilbert spaces. Numer. Funct. Anal. Optim. 37(5), 521–540 (2016). MathSciNetzbMATHGoogle Scholar
  2. 2.
    Andreev, R., Elbau, P., de Hoop, M.V., Qiu, L., Scherzer, O.: Generalized convergence rates results for linear inverse problems in Hilbert spaces. Numer. Funct. Anal. Optim. 36(5), 549–566 (2015). MathSciNetzbMATHGoogle Scholar
  3. 3.
    Borwein, J.M., Lewis, A.S.: Convergence of best entropy estimates. SIAM J. Optim. 1(2), 191–205 (1991). MathSciNetzbMATHGoogle Scholar
  4. 4.
    Burger, M., Osher, S.: Convergence rates of convex variational regularization. Inverse Prob. 20(5), 1411–1421 (2004)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Burger, M., Resmerita, E., He, L.: Error estimation for Bregman iterations and inverse scale space methods in image restoration. Computing 81(2–3), 109–135 (2007). MathSciNetzbMATHGoogle Scholar
  6. 6.
    Censor, Y., Zenios, S.A.: Proximal minimization algorithm with \(D\)-functions. J. Optim. Theory Appl. 73(3), 451–464 (1992). MathSciNetzbMATHGoogle Scholar
  7. 7.
    Chen, G., Teboulle, M.: Convergence analysis of a proximal-like minimization algorithm using Bregman functions. SIAM J. Optim. 3(3), 538–543 (1993). MathSciNetzbMATHGoogle Scholar
  8. 8.
    Cioranescu, I.: Geometry of Banach Spaces, Duality Mappings and Nonlinear Problems, Mathematics and its Applications, vol. 62. Kluwer Academic Publishers, Dordrecht (1990)zbMATHGoogle Scholar
  9. 9.
    DeVore, R.A., Popov, V.A.: Interpolation of Besov spaces. Trans. Am. Math. Soc. 305(1), 397–414 (1988). MathSciNetzbMATHGoogle Scholar
  10. 10.
    Eckstein, J.: Nonlinear proximal point algorithms using Bregman functions, with applications to convex programming. Math. Oper. Res. 18(1), 202–226 (1993). MathSciNetzbMATHGoogle Scholar
  11. 11.
    Eggermont, P.P.B.: Maximum entropy regularization for Fredholm integral equations of the first kind. SIAM J. Math. Anal. 24, 1557–1576 (1993)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Ekeland, I., Temam, R.: Convex Analysis and Variational Problems. North-Holland Publishing Company, Amsterdam (1976)zbMATHGoogle Scholar
  13. 13.
    Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems, Mathematics and its Applications, vol. 375. Kluwer Academic Publishers Group, Dordrecht (1996). zbMATHGoogle Scholar
  14. 14.
    Engl, H.W., Landl, G.: Convergence rates for maximum entropy regularization. SIAM J. Numer. Anal. 30, 1509–1536 (1993)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Flemming, J.: Generalized Tikhonov Regularization and Modern Convergence Rate Theory in Banach Spaces. Shaker Verlag, Aachen (2012)zbMATHGoogle Scholar
  16. 16.
    Frick, K., Grasmair, M.: Regularization of linear ill-posed problems by the augmented Lagrangian method and variational inequalities. Inverse Probl. 28(10), 104005 (2012). MathSciNetzbMATHGoogle Scholar
  17. 17.
    Frick, K., Lorenz, D.A., Resmerita, E.: Morozov’s principle for the augmented Lagrangian method applied to linear inverse problems. Multiscale Model. Simul. 9(4), 1528–1548 (2011). MathSciNetzbMATHGoogle Scholar
  18. 18.
    Frick, K., Scherzer, O.: Regularization of ill-posed linear equations by the non-stationary augmented Lagrangian method. J. Integral Equ. Appl. 22(2), 217–257 (2010). MathSciNetzbMATHGoogle Scholar
  19. 19.
    Godefroy, G.: Renormings in Banach spaces. In: Jonson, J.L.W.B. (ed.) Handbook of the Geometry of Banach Spaces, vol. 1, pp. 781–835. Elsevier Science, Amsterdam (2001)Google Scholar
  20. 20.
    Grasmair, M.: Generalized Bregman distances and convergence rates for non-convex regularization methods. Inverse Probl. 26, 115014 (2010). MathSciNetzbMATHGoogle Scholar
  21. 21.
    Grasmair, M.: Variational inequalities and higher order convergence rates for Tikhonov regularisation on Banach spaces. J. Inverse Ill-Posed Probl. 21(3), 379–394 (2013). MathSciNetzbMATHGoogle Scholar
  22. 22.
    Groetsch, C.W.: The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind. Pitman, Boston (1984)zbMATHGoogle Scholar
  23. 23.
    Hofmann, B., Kaltenbacher, B., Pöschl, C., Scherzer, O.: A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators. Inverse Prob. 23(3), 987–1010 (2007). MathSciNetzbMATHGoogle Scholar
  24. 24.
    Hofmann, B., Yamamoto, M.: On the interplay of source conditions and variational inequalities for nonlinear ill-posed problems. Appl. Anal. 89(11), 1705–1727 (2010). MathSciNetzbMATHGoogle Scholar
  25. 25.
    Hohage, T., Weidling, F.: Verification of a variational source condition for acoustic inverse medium scattering problems. Inverse Probl. 31(7), 075006 (2015). MathSciNetzbMATHGoogle Scholar
  26. 26.
    Hohage, T., Weidling, F.: Characterizations of variational source conditions, converse results, and maxisets of spectral regularization methods. SIAM J. Numer. Anal. 55(2), 598–620 (2017). MathSciNetzbMATHGoogle Scholar
  27. 27.
    Hohage, T., Werner, F.: Convergence rates for inverse problems with impulsive noise. SIAM J. Numer. Anal. 52(3), 1203–1221 (2014)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Hohage, T., Werner, F.: Inverse problems with Poisson data: statistical regularization theory, applications and algorithms. Inverse Probl. 32, 093001 (2016). MathSciNetzbMATHGoogle Scholar
  29. 29.
    Johnsen, J.: Pointwise multiplication of Besov and Triebel–Lizorkin spaces. Math. Nach. 175, 85–133 (1995)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Kaltenbacher, B., Hofmann, B.: Convergence rates for the iteratively regularized Gauss–Newton method in Banach spaces. Inverse Probl. 26(3), 035007 (2010). MathSciNetzbMATHGoogle Scholar
  31. 31.
    Kateb, D.: Fonctions qui opèrent sur les espaces de Besov. Proc. Am. Math. Soc. 128(3), 735–743 (2000). MathSciNetzbMATHGoogle Scholar
  32. 32.
    Lindenstrauss, J., Tzafriri, L.: Classical Banach Spaces II: Function Spaces (Ergebnisse Der Mathematik Und Ihrer Granzgebiete), vol. 97. Springer, Berlin (1979)zbMATHGoogle Scholar
  33. 33.
    Neubauer, A.: On converse and saturation results for Tikhonov regularization of linear ill-posed problems. SIAM J. Numer. Anal. 34(2), 517–527 (1997). MathSciNetzbMATHGoogle Scholar
  34. 34.
    Neubauer, A., Hein, T., Hofmann, B., Kindermann, S., Tautenhahn, U.: Improved and extended results for enhanced convergence rates of Tikhonov regularization in Banach spaces. Appl. Anal. 89(11), 1729–1743 (2010). MathSciNetzbMATHGoogle Scholar
  35. 35.
    Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variation-based image restoration. Multiscale Model. Simul. 4(2), 460–489 (2005). MathSciNetzbMATHGoogle Scholar
  36. 36.
    Resmerita, E.: Regularization of ill-posed problems in Banach spaces: convergence rates. Inverse Probl. 21(4), 1303–1314 (2005)MathSciNetzbMATHGoogle Scholar
  37. 37.
    Resmerita, E., Scherzer, O.: Error estimates for non-quadratic regularization and the relation to enhancement. Inverse Probl. 22(3), 801–814 (2006)MathSciNetzbMATHGoogle Scholar
  38. 38.
    Scherzer, O., Grasmair, M., Grossauer, H., Haltmeier, M., Lenzen, F.: Variational Methods in Imaging: 167 (Applied Mathematical Sciences). Springer, New York (2008)zbMATHGoogle Scholar
  39. 39.
    Skilling, J. (ed.): Maximum Entropy and Bayesian Methods. Kluwer, Dordrecht (1989)zbMATHGoogle Scholar
  40. 40.
    Triebel, H.: Interpolation Theory, Function Spaces, Differential Operators, North-Holland Mathematical Library, vol. 18. North-Holland Publishing Co., Amsterdam (1978)Google Scholar
  41. 41.
    Triebel, H.: Theory of Function Spaces II. Birkhäuser, Basel (1992)zbMATHGoogle Scholar
  42. 42.
    Xu, Z.B., Roach, G.: Characteristic inequalities of uniformly convex and uniformly smooth Banach spaces. J. Math. Anal. Appl. 157(1), 189–210 (1991). MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.GöttingenGermany

Personalised recommendations