Abstract
The conjugate gradient method was published by Hestenes and Stiefel in 1952, as a direct method for solving linear systems. Today its main use is as an iterative method for solving large sparse linear systems. On a test problem we show that it performs as well as the SOR method with optimal acceleration parameter, and we do not have to estimate any such parameter. However the conjugate gradient method is restricted to positive definite systems. We also consider a mathematical formulation of the preconditioned conjugate gradient method . It is used to speed up convergence of the conjugate gradient method. We only give one example of a possible preconditioner.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
It is this property that has given the method its name.
- 2.
Hint: use induction on k to show that \({\boldsymbol {p}}_k={\boldsymbol {r}}_k+\sum _{j=0}^{k-1}a_{k,j}{\boldsymbol {r}}_j\) for some constants a k,j.
- 3.
Hint: The conjugate gradient method for Ay = r 0 can be written y k+1 := y k + γ kq k, \(\gamma _k:=\frac {{\boldsymbol {s}}_{k}^T{\boldsymbol {s}}_{k}}{{\boldsymbol {q}}_{k}^T{\boldsymbol {A}}{\boldsymbol {q}}_{k}}\), s k+1 := s k − γ kAq k, q k+1 := s k+1 + δ kq k, \(\delta _k:= \frac {{\boldsymbol {s}}_{k+1}^T{\boldsymbol {s}}_{k+1}}{{\boldsymbol {s}}_{k}^T{\boldsymbol {s}}_{k}}\). Show that y k = x k −x 0, s k = r k, and q k = p k, for k = 0, 1, 2….
- 4.
This system known as the normal equations appears in linear least squares problems and was considered in this context in Chap. 9.
- 5.
Hint: Show that \({\boldsymbol {A}}^{-1}({\boldsymbol {r}}_{k+1}-{\boldsymbol {r}}_j)\in {\mathcal {W}}_{k+1}\).
References
O. Axelsson, Iterative Solution Methods (Cambridge University Press, 1994)
M. Hestenes, E. Stiefel, Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 29, 409–439 (1952)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Lyche, T. (2020). The Conjugate Gradient Method. In: Numerical Linear Algebra and Matrix Factorizations. Texts in Computational Science and Engineering, vol 22. Springer, Cham. https://doi.org/10.1007/978-3-030-36468-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-36468-7_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36467-0
Online ISBN: 978-3-030-36468-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)