Singular Value Decomposition and Least Squares Solutions

Abstract

Let A be a real m×n matrix with m≧n. It is well known (cf. [4]) that

$$A = U\sum {V^T}$$

((1))

where

$${U^T}U = {V^T}V = V{V^T} = {I_n}{\text{ and }}\sum {\text{ = diag(}}{\sigma _{\text{1}}}{\text{,}} \ldots {\text{,}}{\sigma _n}{\text{)}}{\text{.}}$$

The matrix U consists of n orthonormalized eigenvectors associated with the n largest eigenvalues of AA ^T, and the matrix V consists of the orthonormalized eigenvectors of A ^T A. The diagonal elements of ∑ are the non-negative square roots of the eigenvalues of A ^T A; they are called singular values. We shall assume that

$${\sigma _1} \geqq {\sigma _2} \geqq \cdots \geqq {\sigma _n} \geqq 0.$$

Thus if rank(A)=r, σ _r+1 = σ _r+2=⋯=σ _n = 0. The decomposition (1) is called the singular value decomposition (SVD).

Prepublished in Numer. Math. 14, 403–420 (1970).

The work of this author was in part supported by the National Science Foundation and Office of Naval Research.