BIT Numerical Mathematics

, Volume 46, Issue 1, pp 41–59 | Cite as

Exploiting Residual Information in the Parameter Choice for Discrete Ill-Posed Problems

  • P. C. Hansen
  • M. E. Kilmer
  • R. H. Kjeldsen


Most algorithms for choosing the regularization parameter in a discrete ill-posed problem are based on the norm of the residual vector. In this work we propose a different approach, where we seek to use all the information available in the residual vector. We present important relations between the residual components and the amount of information that is available in the noisy data, and we show how to use statistical tools and fast Fourier transforms to extract this information efficiently. This approach leads to a computationally inexpensive parameter-choice rule based on the normalized cumulative periodogram, which is particularly suited for large-scale problems.

Key words

regularization discrete ill-posed problems parameter-choice method SVD analysis Fourier analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. Boggess and F. Narcowich, A First Course in Wavelets with Fourier Analysis, Prentice Hall, New Jersey, 2001.Google Scholar
  2. 2.
    V. Faber, A. Manteuffel, A. B. White Jr., and G. M. Wing, Asymptotic behavior of singular values and functions of certain convolution operators, Comput. Math. Appl., 12A (1986), pp. 733–747.Google Scholar
  3. 3.
    W. A. Fuller, Introduction to Statistical Time Series, 2nd edn., Wiley, New York, 1996.Google Scholar
  4. 4.
    R. R. Goldberg, Methods of Real Analysis, 2nd edn., Wiley, New York, 1976.Google Scholar
  5. 5.
    P. C. Hansen, Computation of the singular value expansion, Computing, 40 (1988), pp. 185–199.Google Scholar
  6. 6.
    P. C. Hansen, Analysis of discrete ill-posed problems by means of the L-curve, SIAM Rev., 34 (1992), pp. 561–580.Google Scholar
  7. 7.
    P. C. Hansen, Regularization Tools: A Matlab package for analysis and solution of discrete ill-posed problems, Numer. Algorithms, 6 (1994), pp. 1–35.Google Scholar
  8. 8.
    P. C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems, SIAM, Philadelphia, 1998.Google Scholar
  9. 9.
    P. C. Hansen, Deconvolution and regularization with Toeplitz matrices, Numer. Algorithms, 29 (2002), pp. 323–378.Google Scholar
  10. 10.
    M. E. Kilmer and D. P. O’Leary, Choosing regularization parameters in iterative methods for ill-posed problems, SIAM J. Matrix Anal. Appl., 22 (2001), pp. 1204–1221.Google Scholar
  11. 11.
    R. Kress, Linear Integral Equations, 2nd edn., Springer, Heidelberg, 1999.Google Scholar
  12. 12.
    J. G. Nagy, K. Palmer and L. Perrone, Iterative methods for image deblurring: A Matlab object-oriented approach, Numer. Algorithms, 36 (2004), pp. 73–93.Google Scholar
  13. 13.
    D. P. O’Leary, Near-optimal parameters for Tikhonov and other regularization methods, SIAM J. Sci. Comput., 23 (2001), pp. 1161–1171.Google Scholar
  14. 14.
    B. W. Rust, Truncating the Singular Value Decomposition for Ill-Posed Problems, Report NISTIR 6131, Mathematical and Computational Sciences Division, NIST, 1998.Google Scholar
  15. 15.
    A. M. Urnamov, A. V. Gribok, H. Bozdogan, J. W. Hines, and R. E. Uhrig, Information complexity-based regularization parameter selection for solution of ill conditioned inverse problems, Inverse Probl., 18 (2000), pp. L1–L9.Google Scholar
  16. 16.
    G. Wahba, Spline Models for Observational Data, SIAM, Philadelphia, 1990.Google Scholar

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  1. 1.Informatics and Mathematical ModellingTechnical University of DenmarkKgs. LyngbyDenmark
  2. 2.Department of MathematicsTufts UniversityMedfordUSA

Personalised recommendations