Science China Mathematics

, Volume 60, Issue 4, pp 701–718 | Cite as

Some results on the regularization of LSQR for large-scale discrete ill-posed problems

  • Yi Huang
  • ZhongXiao Jia


LSQR, a Lanczos bidiagonalization based Krylov subspace iterative method, and its mathematically equivalent conjugate gradient for least squares problems (CGLS) applied to normal equations system, are commonly used for large-scale discrete ill-posed problems. It is well known that LSQR and CGLS have regularizing effects, where the number of iterations plays the role of the regularization parameter. However, it has long been unknown whether the regularizing effects are good enough to find best possible regularized solutions. Here a best possible regularized solution means that it is at least as accurate as the best regularized solution obtained by the truncated singular value decomposition (TSVD) method. We establish bounds for the distance between the k-dimensional Krylov subspace and the k-dimensional dominant right singular space. They show that the Krylov subspace captures the dominant right singular space better for severely and moderately ill-posed problems than for mildly ill-posed problems. Our general conclusions are that LSQR has better regularizing effects for the first two kinds of problems than for the third kind, and a hybrid LSQR with additional regularization is generally needed for mildly ill-posed problems. Exploiting the established bounds, we derive an estimate for the accuracy of the rank k approximation generated by Lanczos bidiagonalization. Numerical experiments illustrate that the regularizing effects of LSQR are good enough to compute best possible regularized solutions for severely and moderately ill-posed problems, stronger than our theory predicts, but they are not for mildly ill-posed problems and additional regularization is needed.


ill-posed problem regularization Lanczos bidiagonalization LSQR CGLS hybrid 


65F22 65J20 15A18 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



This work was supported by National Basic Research Program of China (Grant No. 2011CB302400) and National Natural Science Foundation of China (Grant No. 11371219). The authors thank the referees for their valuable suggestions and comments, which made the authors think over numerous issues deeply and improve the presentation of the paper considerably.


  1. 1.
    Bauer F, Lukas M A. Comparing parameter choice for regularization of ill-posed problems. Math Comput Simul, 2011, 81: 1795–1841MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Bazán F S V, Borges L S. GKB-FP: An algorithm for large-scale discrete ill-posed problems. BIT Numer Math, 2010, 50: 481–507MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Bazán F S V, Cunha M C C, Borges L S. Extension of GKB-FP algorithm to large-scale general-form Tikhonov regularization. Numer Linear Algebra Appl, 2014, 21: 316–339MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Bergou E H, Gratton S, Tshimanga J. The exact condition number of the truncated singular value solution of a linear ill-posed problem. SIAM J Matrix Anal Appl, 2014, 35: 1073–1085MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Björck Å. A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT Numer Math, 1988, 28: 659–670MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Björck Å. Numerical Methods for Least Squares Problems. Philadelphia: SIAM, 1996CrossRefMATHGoogle Scholar
  7. 7.
    Calvetti D, Golub G H, Reichel L. Estimation of the L-curve via Lanczos bidiagonalization. BIT Numer Math, 1999, 39: 603–619MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Calvetti D, Hansen P C, Reichel L. L-curve curvature bounds via Lanczos bidiagonalization. Electron Trans Numer Anal, 2002, 14: 20–35MathSciNetMATHGoogle Scholar
  9. 9.
    Calvetti D, Lewis B, Reichel L. On the choice of subspace for iterative methods for linear ill-posed problems. Int J Appl Math Comput Sci, 2001, 11: 1069–1092MathSciNetMATHGoogle Scholar
  10. 10.
    Calvetti D, Lewis B, Reichel L. GMRES, L-curve, and discrete ill-posed problems. BIT Numer Math, 2002, 42: 44–65MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Calvetti D, Morigi S, Reichel L, et al. Tikhonov regularization and the L-curve for large discrete ill-posed problems. J Comput Appl Math, 2000, 123: 423–446MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Calvetti D, Reichel L. Tikhonov regularization of large linear problems. BIT Numer Math, 2003, 43: 263–283MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Chung J, Nagy J G, O’Leary D P. A weighted GCV method for Lanczos hybrid regularization. Electron Trans Numer Anal, 2008, 28: 149–167MathSciNetMATHGoogle Scholar
  14. 14.
    Engl H W, Hanke M, Neubauer A. Regularization of Inverse Problems. Dordrecht: Kluwer, 1996CrossRefMATHGoogle Scholar
  15. 15.
    Gazzola S. Regularization techniques based on Krylov methods for ill-posed linear systems. PhD Thesis. Padua: University of Padua, 2014Google Scholar
  16. 16.
    Gazzola S, Novati P. Multi-parameter Arnoldi-Tikhonov methods. Electron Trans Numer Anal, 2013, 40: 452–475MathSciNetMATHGoogle Scholar
  17. 17.
    Gazzola S, Novati P, Russo M R. On Krylov projection methods and Tikhonov regularization. Electron Trans Numer Anal, 2015, 44: 83–123MathSciNetMATHGoogle Scholar
  18. 18.
    Hanke M. Conjugate Gradient Type Methods for Ill-Posed Problems. Essex: Longman, 1995MATHGoogle Scholar
  19. 19.
    Hanke M. On Lanczos based methods for the regularization of discrete ill-posed problems. BIT Numer Math, 2001, 41: 1008–1018MathSciNetCrossRefGoogle Scholar
  20. 20.
    Hansen P C. Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion. Philadelphia: SIAM, 1998CrossRefGoogle Scholar
  21. 21.
    Hansen P C. Regularization tools version 4.0 for Matlab 7.3. Numer Algor, 2007, 46: 189–194MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Hansen P C. Discrete Inverse Problems: Insight and Algorithms. Philadelphia: SIAM, 2010CrossRefMATHGoogle Scholar
  23. 23.
    Hnetynková I, Plešinger M, Strakoš Z. The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data. BIT Numer Math, 2009, 49: 669–696MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Hofmann B. Regularization for Applied Inverse and Ill-posed Problems. Stuttgart: Teubner, 1986CrossRefMATHGoogle Scholar
  25. 25.
    Jensen T K, Hansen P C. Iterative regularization with minimum-residual methods. BIT Numer Math, 2007, 47: 103–120MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Jia Z, Niu D. An implicitly restarted refined bidiagonalization Lanczos method for computing a partial singular value decomposition. SIAM J Matrix Anal Appl, 2003, 25: 246–265MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Kilmer M E, O’Leary D P. Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J Matrix Anal Appl, 2001, 22: 1204–1221MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Lewis B, Reichel L. Arnoldi-Tikhonov regularization methods. J Comput Appl Math, 2009, 226: 92–102MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Li Z C, Huang H T, Wei Y M. Ill-conditioning of the truncated singular value decomposition, Tikhonov regularization and their applications to numerical partial differential equations. Numer Linear Algebra Appl, 2011, 18: 205–221MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Neuman A, Reichel L, Sadok H. Algorithms for range restricted iterative methods for linear dicrete ill-posed problems. Numer Algor, 2012, 59: 325–331CrossRefMATHGoogle Scholar
  31. 31.
    Novati P, Russo M R. A GCV based Arnoldi-Tikhonov regularization method. BIT Numer Math, 2014, 54: 501–521MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    O’Leary D P, Simmons J A. A bidiagonalization-regularization procedure for large scale discretizations of ill-posed problems. SIAM J Sci Statist Comput, 1981, 2: 474–489MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Paige C C, Saunders M A. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans Math Soft, 1982, 8: 43–71MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Parlett B N. The Symmetric Eigenvalue Problem. Philedaphia: SIAM, 1998CrossRefMATHGoogle Scholar
  35. 35.
    Reichel L, Rodriguez G. Old and new parameter choice rules for discrete ill-posed problems. Numer Algor, 2013, 63: 65–87MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Stewart G W. Matrix Algorithms, vol. II: Eigensystems. Philadelphia: SIAM, 2001CrossRefMATHGoogle Scholar
  37. 37.
    Vogel C R. Computational Methods for Inverse Problems. Philadelphia: SIAM, 2002CrossRefMATHGoogle Scholar

Copyright information

© Science China Press and Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Department of Mathematical SciencesTsinghua UniversityBeijingChina

Personalised recommendations