Matrix and Vector Norms

Abstract

In the last chapter we stressed one property of a set of vectors — that of linear independence. We now look at another property possessed by both vectors and matrices, that of ‘size’ or ‘magnitude’. We often want to be able to say that one vector is, in some sense, ‘bigger’ than another. In particular, if we have a sequence of vectors, it is sometimes useful to know that these vectors are getting ‘smaller’. This would certainly be the case if each vector were an ‘error vector’, that is, the difference between a particular vector and some approximation to it. As an example we may have constructed some algorithm for solving the linear equations Ax = b, by generating a séquence of vectors {x _i }, where x _i is the ith approximation to the solution. Clearly if the algorithm is to be at all effective the differences between x _i and A⁻¹ b should get rapidly ‘smaller’ as i increases. But what do we mean by the ‘magnitude’ of a vector? How do we compare two vectors for size? Of the three vectors displayed below, which is the ‘biggest’?