Error Analysis of Triangular Linear Systems

Abstract

The vast majority of the occurrences of condition numbers in the study of linear systems of equations involve the normwise condition number κ(A). Almost invariably, the use of κ(A) is enough to provide a satisfying explanation of the phenomena observed in practice.

The case of triangular systems of linear equations provides, in contrast, an example in which κ(A) turns out to be inadequate. Practitioners long observed that triangular systems of equations are generally solved to high accuracy in spite of being, in general, ill-conditioned. Thus, for instance, J.H. Wilkinson wrote: “In practice one almost invariably finds that if L is ill-conditioned, so that ∥L∥∥L ⁻¹∥≫1, then the computed solution of Lx=b (or the computed inverse) is far more accurate than [what forward stability analysis] would suggest.”

A first goal in this chapter is to give a precise meaning to the feeling that triangular matrices are, in general, ill-conditioned. We prove that if $L\in \mathbb {R}^{n\times n}$ is a lower-triangular matrix whose entries are independent standard Gaussian random variables, then $\mathbb {E}(\log \kappa(L))=\varOmega(n)$. This yields an expected loss of precision satisfying

$$\mathbb {E}\bigl(\mathsf {LoP}\bigl(L^{-1}b\bigr)\bigr)= \mathcal {O}(n). $$

Were the loss of precision in the solution of triangular systems to conform to this bound, we would not be able to accurately find these solutions. The reason we actually do find them can be briefly stated. The error analysis of triangular systems reveals that we may use a componentwise condition number Cw(L,b) instead of the normwise condition number. The second goal of this chapter is to prove that when L and $b\in \mathbb {R}^{n}$ are drawn from standard Gaussian distributions, then we have $\mathbb {E}(\log \mathsf {Cw}(L,b))=\mathcal {O}(\log n)$. This bound, together with some backward error analysis, yields bounds for $\mathbb {E}(\mathsf {LoP}(L^{-1}b))$ that are much smaller than the one above, as well as closer to the loss of precision observed in practice.