Preconditioned Conjugate Gradients for Givens Plane Rotations

Abstract

Assume, as in the previous four chapters, that A ∈ R ^mxn and b ∈ R ^mx1 are given and that m ≥ n as well as rank(A)=n are satisfied. Assume also that an approximation to x = A^↑b = (A^TA)⁻¹A^Tb is to be calculated. In this chapter it will be shown that this problem can be transformed into an equivalent problem, which is a system of linear algebraic equations Cy=d whose coefficient matrix C is symmetric and positive definite. Moreover, C can be written as C = D⁻¹(R^T)⁻¹A^TAR⁻¹D^-1, where D is a diagonal matrix and R is an upper triangular matrix. The conjugate gradient (CG) algorithm can be applied in the solution of the system of linear algebraic equations Cy=d (y=DRx, d=D⁻¹(R^T)⁻¹A^Tb). If D=R=I, then the CG algorithm is in fact applied to the system of normal equations A^TAx = A^Tb and the speed of convergence could be very slow (see also Chapter 3). If D and R are obtained by some kind of orthogonalization with some matrix Q ∈ R ^mxn with orthonormal columns (see Chapter 12) and if the calculations are performed without rounding errors, then C=I and, thus, the CG algorithm converges in one iteration only. Even if the orthogonalization is carried out with rounding errors, the matrix C is normally close to the identity matrix I and the CG algorithm is quickly convergent. However, the orthogonal decomposition is an expensive process (both in regard to storage and in regard to computing time). Therefore it may be profitable to calculate an incomplete orthogonal decomposition. This is achieved in the same way as in the previous chapter: by using a special drop-tolerance parameter T (see also Chapter 3, Chapter 5 and Chapter 11). All elements that are smaller (in absolute value) than the drop-tolerance T are removed. Many numerical examples will be given to illustrate that the CG algorithm applied to the system of linear algebraic equations Cy=d and used with incomplete factors D and R is very efficient for some classes of problems. Matrix C is never calculated explicitly. The whole computational work is carried out by the use of the matrices A, D, and R only.