Iterative Solution of Linear Variational Problems in Hilbert Spaces: Some Conjugate Gradients Success Stories

Abstract

Conjugate gradient was introduced in the early 50s (see refs. [1], [2]) as an iterative method for the solution of finite dimensional linear systems associated to matrices with special properties such as symmetry and positive definiteness. If round-off errors are not taken into account, the conjugate gradient method enjoys the finite termination property, namely: for any initialization, conjugate gradient algorithms will converge in d iterations at most, assuming that the matrix of the linear system, one is trying to solve, is a d × d one. Actually, for large problems (let say as soon as d > 10²) the finite termination property has no practical interest, unlike an estimate of the speed of convergence involving the square-root of the condition number. In fact conjugate gradient algorithms can also be applied (theoretically, at least to the solution of linear problems in Hilbert spaces and by the late 60s a solid foundation for the convergence properties of such algorithms was in place (see [3] and the references therein. Such development was most important, since it opened the door to the efficient iterative solution of optimal control problems and of large classes of linear partial differential equations via an appropriate variational formulation in a well-chosen Hilbert space. Examples of such applications can be found in, e.g., [4]–[7], and of course, in various parts of this volume.