LSQR, a Lanczos bidiagonalization based Krylov subspace iterative method, and its mathematically equivalent conjugate gradient for least squares problems (CGLS) applied to normal equations system, are commonly used for large-scale discrete ill-posed problems. It is well known that LSQR and CGLS have regularizing effects, where the number of iterations plays the role of the regularization parameter. However, it has long been unknown whether the regularizing effects are good enough to find best possible regularized solutions. Here a best possible regularized solution means that it is at least as accurate as the best regularized solution obtained by the truncated singular value decomposition (TSVD) method. We establish bounds for the distance between the k-dimensional Krylov subspace and the k-dimensional dominant right singular space. They show that the Krylov subspace captures the dominant right singular space better for severely and moderately ill-posed problems than for mildly ill-posed problems. Our general conclusions are that LSQR has better regularizing effects for the first two kinds of problems than for the third kind, and a hybrid LSQR with additional regularization is generally needed for mildly ill-posed problems. Exploiting the established bounds, we derive an estimate for the accuracy of the rank k approximation generated by Lanczos bidiagonalization. Numerical experiments illustrate that the regularizing effects of LSQR are good enough to compute best possible regularized solutions for severely and moderately ill-posed problems, stronger than our theory predicts, but they are not for mildly ill-posed problems and additional regularization is needed.
ill-posed problem regularization Lanczos bidiagonalization LSQR CGLS hybrid
65F22 65J20 15A18
This is a preview of subscription content, log in to check access.
This work was supported by National Basic Research Program of China (Grant No. 2011CB302400) and National Natural Science Foundation of China (Grant No. 11371219). The authors thank the referees for their valuable suggestions and comments, which made the authors think over numerous issues deeply and improve the presentation of the paper considerably.
Bazán F S V, Cunha M C C, Borges L S. Extension of GKB-FP algorithm to large-scale general-form Tikhonov regularization. Numer Linear Algebra Appl, 2014, 21: 316–339MathSciNetCrossRefMATHGoogle Scholar
Bergou E H, Gratton S, Tshimanga J. The exact condition number of the truncated singular value solution of a linear ill-posed problem. SIAM J Matrix Anal Appl, 2014, 35: 1073–1085MathSciNetCrossRefMATHGoogle Scholar
Hnetynková I, Plešinger M, Strakoš Z. The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data. BIT Numer Math, 2009, 49: 669–696MathSciNetCrossRefMATHGoogle Scholar
Li Z C, Huang H T, Wei Y M. Ill-conditioning of the truncated singular value decomposition, Tikhonov regularization and their applications to numerical partial differential equations. Numer Linear Algebra Appl, 2011, 18: 205–221MathSciNetCrossRefMATHGoogle Scholar
Neuman A, Reichel L, Sadok H. Algorithms for range restricted iterative methods for linear dicrete ill-posed problems. Numer Algor, 2012, 59: 325–331CrossRefMATHGoogle Scholar
O’Leary D P, Simmons J A. A bidiagonalization-regularization procedure for large scale discretizations of ill-posed problems. SIAM J Sci Statist Comput, 1981, 2: 474–489MathSciNetCrossRefMATHGoogle Scholar