Regularization Methods for Ill-Posed Problems

  • Jin Cheng
  • Bernd Hofmann
Reference work entry


In this chapter, we outline the mathematical theory of direct regularization methods for in general nonlinear and ill-posed inverse problems. One focus is on Tikhonov regularization in Hilbert spaces with quadratic misfit and penalty terms. Moreover, recent results of an extension of the theory to Banach spaces are presented concerning the variational regularization with convex penalty term. Five examples of parameter identification problems in integral and differential equations are given in order to show how to apply the theory of this chapter to specific inverse and ill-posed problems.


Inverse Problem Variational Inequality Regularization Parameter Penalty Term Tikhonov Regularization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References and Further Reading

  1. 1.
    Acar R, Vogel CR (1994) Analysis of bounded variation penalty methods for ill-posed problems. Inverse Probl 10(6):1217–1229MathSciNetMATHCrossRefGoogle Scholar
  2. 2.
    Ammari H (2008) An introduction to mathematics of emerging biomedical imaging. Springer, BerlinMATHGoogle Scholar
  3. 3.
    Bakushinsky A, Goncharsky A (1994) Ill-posed problems: theory and applications. Kluwer, DordrechtCrossRefGoogle Scholar
  4. 4.
    Bakushinsky AB, Kokurin MYu (2004) Iterative methods for approximate solution of inverse problems. Springer, DordrechtMATHGoogle Scholar
  5. 5.
    Banks HT, Kunisch K (1989) Estimation techniques for distributed parameter systems. Birkhäuser, BostonMATHCrossRefGoogle Scholar
  6. 6.
    Baumeister J (1987) Stable solution of inverse problems. Vieweg, BraunschweigMATHGoogle Scholar
  7. 7.
    Beretta E, Vessella S (1999) Stable determination of boundaries from Cauchy data. SIAM J Math Anal 30:220–232MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Bertero M, Boccacci P (1998) Introduction to inverse problems in imaging. Institute of Physics Publishing, BristolMATHCrossRefGoogle Scholar
  9. 9.
    Bonesky T, Kazimierski K, Maass P, Schöpfe F, Schuster T (2008) Minimization of Tikhonov functionals in Banach spaces. Abstr Appl Anal Art 192679:1–19CrossRefGoogle Scholar
  10. 10.
    Bredies K, Lorenz DA (2009) Regularization with non-convex separable constraints. Inverse Probl 25(8):1–14, 085011Google Scholar
  11. 11.
    Bukhgeim AL, Cheng J, Yamamoto M (1999) Stability for an inverse boundary problem of determining a part of a boundary. Inverse Probl 15:1021–1032MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Burger M, Osher S (2004) Convergence rates of convex variational regularization. Inverse Probl 20(5):1411–1421MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    Burger M, Resmerita E, He L (2007) Error estimation for Bregman iterations and inverse scale space methods in image restoration. Computing 81(2–3):109–135MathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    Chavent G (2009) Nonlinear least squares for inverse problems. Springer, DordrechtMATHGoogle Scholar
  15. 15.
    Chavent G, Kunisch K (1996) On weakly nonlinear inverse problems. SIAM J Appl Math 56(2): 542–572MathSciNetMATHCrossRefGoogle Scholar
  16. 16.
    Chavent G, Kunisch K (1998) State space regularization: geometric theory. Appl Math Opt 37(3):243–267MathSciNetMATHCrossRefGoogle Scholar
  17. 17.
    Cheng J, Nakamura G (2001) Stability for the inverse potential problem by finite measurements on the boundary. Inverse Probl 17:273–280MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    Cheng J, Yamamoto M (2000) Conditional stabilizing estimation for an integral equation of first kind with analytic kernel. J Integral Equat Appl 12:39–61MathSciNetMATHCrossRefGoogle Scholar
  19. 19.
    Cheng J, Yamamoto M (2000) One new strategy for a priori choice of regularizing parameters in Tikhonov’s regularization. Inverse Probl 16(4):L31–L38MathSciNetMATHCrossRefGoogle Scholar
  20. 20.
    Colton D, Kress R (1992) Inverse acoustic and electromagnetic scattering theory. Springer, BerlinMATHGoogle Scholar
  21. 21.
    Engl HW, Hanke M, Neubauer A (1996/2000) Regularization of inverse problems. Kluwer, DordrechtGoogle Scholar
  22. 22.
    Engl HW, Isakov V (1992) On the identifiability of steel reinforcement bars in concrete from magnetostatic measurements. Eur J Appl Math 3:255–262MathSciNetCrossRefGoogle Scholar
  23. 23.
    Engl HW, Kunisch K, Neubauer A (1989) Convergence rates for Tikhonov regularisation of non-linear ill-posed problems. Inverse Probl 5(4): 523–540MathSciNetMATHCrossRefGoogle Scholar
  24. 24.
    Engl HW, Zou J (2000) A new approach to convergence rate analysis of Tikhonov regularization for parameter identification in heat conduction. Inverse Probl 16(6):1907–1923MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    Favaro P, Soatto S (2007) 3-D shape estimation and image restoration. Exploiting defocus and motion blur. Springer, LondonMATHGoogle Scholar
  26. 26.
    Fischer B, Modersitzki J (2008) Ill-posed medicine – an introduction to image registration. Inverse Probl 24(3): 1–16, 034008Google Scholar
  27. 27.
    Flemming J, Hofmann B (2010) A new approach to source conditions in regularization with general residual term. Numer Funct Anal Optimiz 31(3):254–284MathSciNetMATHCrossRefGoogle Scholar
  28. 28.
    Grasmair M, Haltmeier M, Scherzer O (2008) Sparse regularization with \({\mathcal{l}}_{\mathrm{q}}\) penalty term. Inverse Probl 24(5):1–13, 055020Google Scholar
  29. 29.
    Gorenflo R, Hofmann B (1994) On autoconvolution and regularization. Inverse Probl 10(2): 353–373MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Groetsch CW (1984) The theory of Tikhonov regularization for Fredholm integral equations of the first kind. Pitman, BostonGoogle Scholar
  31. 31.
    Hansen PC (1998) Rank-deficient and discrete ill-posed poblems. SIAM, PhiladelphiaCrossRefGoogle Scholar
  32. 32.
    Hein T (2009) Tikhonov regularization in Banach spaces – improved convergence rates results. Inverse Probl 25(3):1–18, 035002Google Scholar
  33. 33.
    Hein T, Hofmann B (2009) Approximate source conditions for nonlinear ill-posed problems – chances and limitations. Inverse Probl 25(3):1–16, 035003Google Scholar
  34. 34.
    Hofmann B, Kaltenbacher B, Pöschl C, Scherzer O (2007) A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators. Inverse Probl 23(3):987–1010MATHCrossRefGoogle Scholar
  35. 35.
    Hofmann B, Mathé P (2007) Analysis of profile functions for general linear regularization methods. SIAM J Num Anal 45(3):1122–1141MATHCrossRefGoogle Scholar
  36. 36.
    Hofmann B, Mathé P, Pereverzev SV (2007) Regularization by projection: approximation theoretic aspects and distance functions. J Inv Ill-Posed Problems 15(5):527–545MATHCrossRefGoogle Scholar
  37. 37.
    Hofmann B, Scherzer O (1994) Factors influencing the ill-posedness of nonlinear problems. Inverse Probl 10(6):1277–1297MathSciNetMATHCrossRefGoogle Scholar
  38. 38.
    Hofmann B, Yamamoto M (2010) On the interplay of source conditions and variational inequalities for nonlinear ill-posed problems. Appl Anal 89:doi 10.1080/00036810903208148Google Scholar
  39. 39.
    Imanuvilov OYu, Yamamoto M (2001) Global uniqueness and stability in determining coefficients of wave equations. Comm Partial Diff Equat 26:1409–1425MathSciNetMATHCrossRefGoogle Scholar
  40. 40.
    Isakov V (2006) Inverse problems for partial differential equations. Springer, New YorkMATHGoogle Scholar
  41. 41.
    Ito K, Kunisch K (1992) On the choice of the regularization parameter in nonlinear inverse problems. SIAM J Optimiz 2(3):376–404MathSciNetMATHCrossRefGoogle Scholar
  42. 42.
    Janno J, Wolfersdorf Lv (2005) A general class of autoconvolution equations of the third kind. Z Anal Anwend 24(3):523–543MATHCrossRefGoogle Scholar
  43. 43.
    Jin B, Zou J (2009) Aug mented Tikhonov regularization. Inverse Probl 25(2):1–25, 025001Google Scholar
  44. 44.
    Kaipio J, Somersalo E (2005) Statistical and computational inverse problems. Springer, New YorkMATHGoogle Scholar
  45. 45.
    Kaltenbacher B (2008) A note on logarithmic convergence rates for nonlinear Tikhonov regularization. J Inv Ill-Posed Probl 16(1):79–88MathSciNetMATHCrossRefGoogle Scholar
  46. 46.
    Kaltenbacher B, Neubauer A, Scherzer O (2008) Iterative regularization methods for nonlinear ill-posed problems. Walter de Gruyter, BerlinMATHCrossRefGoogle Scholar
  47. 47.
    Kirsch A (1996) An introduction to the mathematical theory of inverse problems. Springer, New YorkMATHCrossRefGoogle Scholar
  48. 48.
    Klann E, Kuhn M, Lorenz DA, Maass P, Thiele H (2007) Shrinkage versus deconvolution. Inverse Probl 23(5):2231–2248MathSciNetMATHCrossRefGoogle Scholar
  49. 49.
    Kress R (1989) Linear integral equations. Springer, BerlinMATHCrossRefGoogle Scholar
  50. 50.
    Lattès R, Lions J-L (1969) The method of quasi-reversibility. applications to partial differential equations. Modern analytic and computational methods in science and mathematics, No. 18. American Elsevier Publishing, New YorkGoogle Scholar
  51. 51.
    Louis AK (1989) Inverse und schlecht gestellte Probleme. Teubner, StuttgartMATHGoogle Scholar
  52. 52.
    Liu F, Nashed MZ (1998) Regularization of nonlinear ill-posed variational inequalities and convergence rates. Set-Valued Anal 6(4):313–344MathSciNetMATHCrossRefGoogle Scholar
  53. 53.
    Lu S, Pereverzev SV, Ramlau R (2007) An analysis of Tikhonov regularization for nonlinear ill-posed problems under a general smoothness assumption. Inverse Probl 23(1):217–230MathSciNetMATHCrossRefGoogle Scholar
  54. 54.
    Mahale P, Nair MT (2007) Tikhonov regularization of nonlinear ill-posed equations under general source conditions. J Inv Ill-Posed Probl 15(8): 813–829MathSciNetMATHCrossRefGoogle Scholar
  55. 55.
    Mathé P, Pereverzev SV (2003) Geometry of linear ill-posed problems in variable Hilbert scales. Inverse Probl 19(3):789–803MATHCrossRefGoogle Scholar
  56. 56.
    Modersitzki J (2009) FAIR. Flexible Algorithms for Image Registration. SIAM, PhiladelphiaMATHGoogle Scholar
  57. 57.
    Morozov VA (1984) Methods for solving incorrectly posed problems. Springer, New YorkCrossRefGoogle Scholar
  58. 58.
    Natterer F (2007) Imaging and inverse problems of partial differential equations. Jahresber Dtsch Math-Ver 109(1):31–48MathSciNetMATHGoogle Scholar
  59. 59.
    Natterer F, Wübbeling F (2001) Mathematical methods in image reconstruction. SIAM, PhiladelphiaMATHCrossRefGoogle Scholar
  60. 60.
    Neubauer A (1989) Tikhonov regularization for nonlinear ill-posed problems: optimal convergence rate and finite dimensional approximation. Inverse Probl 5(4):541–558MathSciNetMATHCrossRefGoogle Scholar
  61. 61.
    Neubauer A (2009) On enhanced convergence rates for Tikhonov regularization of nonlinear ill-posed problems in Banach spaces. Inverse Probl 25(6):1–10, 065009Google Scholar
  62. 62.
    Neubauer A, Hein T, Hofmann B, Kindermann S, Tautenhahn U (2010) Improved and extended results for enhanced convergence rates of Tikkhonov regularization in Banach spaces. Appl Anal 89 (DOI: 10.1080/00036810903517597)Google Scholar
  63. 63.
    Pöschl C (2008) Tikhonov regularization with general residual term. Ph.D thesis, University of Innsbruck, AustriaGoogle Scholar
  64. 64.
    Ramlau R (2003) TIGRA – an iterative algorithm for regularizing nonlinear ill-posed problems. Inverse Probl 19(2):433–465MathSciNetMATHCrossRefGoogle Scholar
  65. 65.
    Resmerita E, Scherzer O (2006) Error estimates for non-quadratic regularization and the relation to enhancement. Inverse Probl 22(3):801–814MathSciNetMATHCrossRefGoogle Scholar
  66. 66.
    Rondi L (2000) Uniqueness and stability for the determination of boundary defects by electrostatic measurements. Proc Roy Soc Edinburgh Sect A 130:1119–1151MathSciNetMATHCrossRefGoogle Scholar
  67. 67.
    Scherzer O, Engl HW, Kunisch K (1993) Optimal a posteriori parameter choice for Tikhonov regularization for solving nonlinear ill-posed problems. SIAM J Numer Anal 30(6):1796–1838MathSciNetMATHCrossRefGoogle Scholar
  68. 68.
    Scherzer O (ed) (2006) Mathematical models for registration and applications to medical imaging. Mathematics in industry 10. The European Consortium for Mathematics in Industry. Springer, BerlinGoogle Scholar
  69. 69.
    Scherzer O, Grasmair M, Grossauer H, Haltmeiner M, Lenzen F (2009) Variational methods in imaging. Springer, New YorkMATHGoogle Scholar
  70. 70.
    Seidman TI, Vogel CR (1989) Well posedness and convergence of some regularization methods for nonlinear ill posed problems. Inverse Probl 5(2):227–238MathSciNetMATHCrossRefGoogle Scholar
  71. 71.
    Tautenhahn U (1997) On a general regularization scheme for nonlinear ill-posed problems. Inverse Probl 13(5):1427–1437MathSciNetMATHCrossRefGoogle Scholar
  72. 72.
    Tautenhahn U (2002) On the method of Lavrentiev regularization for nonlinear ill-posed problems. Inverse Probl 18(1):191–207MathSciNetMATHCrossRefGoogle Scholar
  73. 73.
    Tautenhahn U, Jin Q (2003) Tikhonov regularization and a posteriori rules for solving nonlinear ill-posed problems. Inverse Probl 19(1): 1–21MathSciNetMATHCrossRefGoogle Scholar
  74. 74.
    Tikhonov AN, Arsenin VY (1977) Solutions of ill-posed problems. Wiley, New YorkMATHGoogle Scholar
  75. 75.
    Tikhonov AN, Goncharsky AV, Stepanov VV, Yagola AG (1995) Numerical methods for the solution of ill-posed problems. Kluwer, DordrechtMATHGoogle Scholar
  76. 76.
    Tikhonov AN, Leonov AS, Yagola AG (1998) Nonlinear ill-posed problems, vols 1 and 2. Series Applied mathematics and mathematical computation, vol 14. Chapman & Hall, LondonGoogle Scholar
  77. 77.
    Vasin VV (2006) Some tendencies in the Tikhonov regularization of ill-posed problems. J Inv Ill-Posed Problems 14(8):813–840MathSciNetMATHCrossRefGoogle Scholar
  78. 78.
    Vogel C (2002) Computational methods for inverse problems. SIAM, PhiladelphiaMATHCrossRefGoogle Scholar
  79. 79.
    Yamamoto M (1996) On ill-posedness and a Tikhonov regularization for a multidimensional inverse hyperbolic problem. J Math Kyoto Univ 36:825–856MathSciNetMATHGoogle Scholar
  80. 80.
    Zarzer CA (2009) On Tikhonov regularization with non-convex sparsity constraints. Inverse Probl 25(2):1–13, 025006Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Jin Cheng
    • 1
  • Bernd Hofmann
    • 2
  1. 1.Fudan UniversityShanghaiChina
  2. 2.Chemnitz University of TechnologyChemnitzGermany

Personalised recommendations