Abstract
There exists a very powerful set of techniques for dealing with sets of equations or matrices that are either singular or numerically very close to singular. In many cases where Gaussian elimination and triangle decomposition fail to give satisfactory results, this set of techniques, known as singular value decomposition (SVD) will solve it, in the sense of giving you a useful numerical answer. In this chapter, the reader will see that SVD of any matrix is always realizable. Applications of SVD [and a related tool known as polar decomposition (PD), or canonical factorization, generalizing a complex number’s factorization by its modulus and phase factor] are not restricted to numerical analysis; other application examples are presented in this chapter. More advanced readers who wish to familiarize themselves with infinite-dimensional versions of SVD and PD and their various applications may consult the guide to the literature that is provided.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Lower- and Upper-triangular.
- 2.
Here we do not distinguish between the operators and their matrix representations with respect to the standard coordinate orthobasis.
- 3.
- 4.
The complexified linear operator has the same matrix in a ℂ-basis of the complexified space as a source operator in the same basis, considered as a ℝ-basis of a source space.
- 5.
In fact, these operators generate the group SO(n) (see section P10.27**, in the “One-Parameter Group of Linear Transformations” problem group below).
- 6.
Note that the operator D ∘ A *∘ D −1 is completely determined by A and does not depend on a scalar product that we used to define it!
- 7.
Some advanced readers may object, saying that the adjoint operators act in the adjoint spaces (spaces of linear functionals), W *: ℝm* → ℝn*, to be exact (W * y *)x = y *(Wx), so use of inner products is not necessary [as for instance, “orthogonality” between x∈ℝn and x *∈ℝn* means that x *(x) = 0, and so on]. However, we use an equivalent approach because an inner product in a vector space defines the isomorphism of this space onto its adjoint x \( \mapsto \) x * such that x *(x′) = \( \langle x,x\prime \rangle \), ∀x′.
- 8.
The unitary spectral theorem states that a unitary n × n-matrix is diagonalizable in a proper orthonormal basis in ℂn and that its eigenvalues are located on the unit circle in ℂ. In turn, the orthogonal spectral theorem claims that an orthogonal matrix has a block-diagonal form in an appropriate orthonormal basis, with one- or two-dimensional blocks being, respectively, ±1 and rotations as \( \left( {\begin{array}{llll}{\cos \theta } & {-\sin \theta } \\{\sin \theta } & {\cos \theta } \\ \end{array}} \right) \). Note that a pair of “−1” blocks is equivalent to a two-dimensional block, which is a rotation by 180°, so that orthogonal matrices that preserve (invert) orientation have spectral representations with no (resp. one) “−1” block.
The orthogonal spectral theorem is reduced to the unitary one using complexification. In turn, the unitary spectral theorem may be proved by the following general method. Let a group G act on a vector space L over a field F by linear operators; that is, there is a group homomorphism (referred to as a linear representation, or just representation for short) ρ: G → GL(L), so that the action g: L → L (g∈G) is equated to the linear operator ρ(g). A subspace K ⊆ L is called a G- invariant subspace, or G -subspace, when gK ⊆ K, ∀g∈G. The representation is called irreducible if space L does not have proper (not equal to ether {0} or itself) G -subspaces. The following claim is evident.
Proposition 1 A finite-dimensional representation (dim L < ∞) has a G-subspace K, irreducible with respect to the representation g \( \mapsto \) ρ(g) | K .
For two representations ρ i : G → GL(L i ) (i = 1,2) a G-operator is a linear operator A: L 1 → L 2, which makes the diagrams \( \begin{array}{llll} {A:} & {{L_1}} & \to & {{L_2}} \\ {} & {\downarrow g} & {} & {\downarrow g} \\ {A:} & {{L_1}} & \to & {{L_2}} \\ \end{array} \) commutative, ∀g∈G.
Lemma 1 Kernels and images of G-operators are G-subspaces. (Why?)
An invertible G-operator is also called an equivalence of the two representations.
Lemma 2 If a basic field F is algebraically closed, then an equivalence of irreducible finite-dimensional representations is defined uniquely, within proportionality.
Indeed, let A: L 1 → L 2 and B: L 1 → L 2 be equivalences. A polynomial χ(λ): = det (A − λB) has a root λ 0∈F. A G -operator A − λ 0 B is not invertible; therefore, it is zero, A = λ 0 B. (Why?) QED.
Corollary If a basic field F is algebraically closed, then irreducible finite-dimensional representations of Abelian groups are one-dimensional.
Indeed, consider the representation ρ: G → GL(L). The operators ρ(g) (g∈G) are (auto-) equivalences (why?), so they must be proportional to id L , that is, be homothetic transformations. Hence, all subspaces of L are invariant, and therefore dim L = 1. QED.
So far, we have enumerated common results for all representations. A special result of unitary representations [operators ρ(g), ∀g∈G are unitary] is as follows.
Proposition 2 For a unitary representation, the orthogonal complements to G - subspaces are G - subspaces.
(Why?) Considering the self-representation [ρ(g) = g, ∀g∈G] of the cyclic group G generated by a unitary operator U, and using this proposition in combination with the previous one and the preceding corollary, we will be able to prove the spectral theorem for U by induction on the spatial dimension.
- 9.
Taking into account that a subspace im t W coincides with its closure, due to the finite dimensionality.
- 10.
Which holds due to the finite dimensionality also.
- 11.
A similar proof of the equality of row and column ranks is valid for the matrices over any field and is generalized for wider classes of rings. (Which ones?) A different proof is discussed in the problem group “A Property of Orthogonal Matrices” (section E8.11).
- 12.
In the most frequently encountered case, n = 3, the space of Hooke’s operators has a dimension of 21.
- 13.
In the elasticity theory, λ and μ are referred to as Lamé constants.
- 14.
In the elasticity theory, \( \mathcal{F } \) is referred to as the Helmholtz potential (or free) energy.
- 15.
An explicit determination of coefficients of proportionality is unnecessary; to prove their distinctness from zero, one can proceed as follows. A map P: (ω 1,…, ω n ) \( \mapsto \) (∑ ω i ,…, ∑ ω i n) is a composition of Vieta’s map V: (ω 1,…, ω n ) \( \mapsto \) (σ 1,…, σ n ) and a map S: (σ 1,…, σ n ) \( \mapsto \) (∑ ω i ,…, ∑ ω i n). The Jacobi matrix S * is polynomial and triangular, with constant diagonal elements equal to the desired coefficients. The Jacobi matrix P * is a composition of matrices V * and S *. P * is nondegenerate for (and only for; why?) distinct ω 1,…, ω n , and hence S * is nondegenerate for distinct ω 1,…, ω n ; but since the diagonal elements of S * are constant, they are distinct from zero. QED. (We leave it to the reader to fill in the details.) Also, we have determined the critical set of Vieta’s map (the set of all critical, or singular, points of V, i.e., the points that V * degenerates). (We leave it to the reader to formulate this result. A different proof of this result uses the explicit form of the Jacobian of Vieta’s maps, \( \det {V_{*}}=\det \left( {{\sigma_{i-1 }}({\omega_1}, \ldots, \widehat{{{\omega_j}}}, \ldots, {\omega_n})} \right)=\prod\limits_{i<j } {({\omega_i}-{\omega_j})} \), that can be derived using section S1.4 from the “Jacobi Identities and Related Combinatorial Formulas” problem group. In addition, readers familiar with elements of complex analysis can prove the foregoing result using the implicit function theorem’s inversion, which states that a bijective holomorphic map \( \mathcal{U} \) → \( \mathcal{V} \), where \( \mathcal{U} \), \( \mathcal{V} \) are open sets in ℂn, is biholomorphic (⇔ nonsingular), and the facts about polynomials with complex coefficients – that a polynomial of degree n has exactly n complex roots and all roots of the polynomial with the leading coefficient equal to one will remain bounded when all coefficients remain bounded. Furthermore, a point (ω 1,…, ω n ) with distinct complex coordinates ω i is in an open polydisk \( \mathcal{U} \) = \( \mathcal{D} \) 1 × …× \( \mathcal{D} \) n , where \( \mathcal{D} \) i are disjoint open disks in ℂ with centers ω i , respectively. \( \mathcal{U} \) intersects at most once with any orbit of the group of all permutations of coordinates (the symmetric group) S n ; hence, the restriction \( V{|_u} \)is injective. Next, V(\( \mathcal{W} \)) are open for open \( \mathcal{W} \): if for ω∈\( \mathcal{W} \) the point σ = (σ 1,…, σ n ) = V(ω) does not possess a neighborhood in ℂn every point of which has a preimage in \( \mathcal{W} \), then we can find a convergent sequence ω (ν) → ω′ so that the points σ (ν) = V(ω (ν)) do not have preimages in \( \mathcal{W} \) and σ (ν) → σ as ν → ∞; but since Aω′ = ω for an appropriate transformation A∈S n , Aω (ν) → ω, and so Aω (ν)∈\( \mathcal{W} \) for all sufficiently large ν; however – contrary to the assumption – Aω (ν) (along with ω (ν)) are preimages of σ (ν). Thus, \( \mathcal{U} \) is bijectively mapped onto an open set \( \mathcal{V} \) = V(\( \mathcal{U} \)). We leave it to interested reader to fill in the details.)
- 16.
In the elasticity theory, R and C are related to the shear tensor and the compression tensor, respectively.
- 17.
In this connection, μ is also called the shear modulus and K the bulk elastic modulus.
- 18.
In the elasticity theory, these parameters are called, respectively, Young’s modulus and the Poisson coefficient.
- 19.
In elasticity-theoretic considerations, λ, μ, K, and Ε have the dimensionality of pressure (energy per unit volume) and σ is dimensionless.
- 20.
Values of σ close to (n − 1)−1 (occurring for μ/K ≪ 1) are characteristic for rubber and materials with a similar elastic behavior.
- 21.
The positiveness of σ is usual for natural materials; a material having σ < 0 would thicken under the axial extension!
- 22.
The adjoint space is defined as a space of all continuous linear functionals, which for a finite-dimensional space is the same as the space of all linear functionals.
References
Bochner, S., Martin, W.T.: Several Complex Variables. Princeton University Press, Princeton (1948)
Dieudonné, J.: Foundation of Modern Analysis. Academic, New York/London (1960)
Dunford, N., Schwartz, J.T.: Linear Operators. Part II. Spectral Theory. Self Adjoint Operators on Hilbert Space. Interscience, New York/London (1963)
Gelbaum, B.R., Olmsted, J.M.H.: Counterexamples in Analysis. Holden-Day, San Francisco/London/Amsterdam (1964)
Gelfand, I.M.: Лекции по линейной алгебре, 4th edn. “Наука” Press, Moscow (1971). [English transl. Lectures on Linear Algebra (Dover Books on Mathematics). Dover (1989)]
Kostrikin, A.I., Manin, Yu.I.: Линейная алгебра и геометрия. МГУ Press, Moscow (1980). [English transl. Linear Algebra and Geometry (Algebra, Logic and Applications). Taylor & Francis Press (1997)]
Landau, L.D., Lifshitz, E.M.: Теоретическая физика, Т.7: Теория упругости. “Наука” Press, Moscow (1987). [English transl. Theory of Elasticity, 3rd edn. Pergamon Press, Oxford/New York (1986)]
Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in Fortran 77. The Art of Scientific Computing, 2nd edn. Cambridge University Press, Cambridge/New York/Melbourne (1992)
Reed, M., Simon, B.: Methods of Modern Mathematical Physics. V.1: Functional Analysis. Academic, New York/London (1972)
Riesz, F. & Sz.-Nagy, B.: Leçons d’analyse fonctionnnelle. Sixtème edition, Budapest (1972). [English transl. Functional Analysis. Dover (1990)]
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Roytvarf, A.A. (2013). Polar and Singular Value Decomposition Theorems. In: Thinking in Problems. Birkhäuser, Boston. https://doi.org/10.1007/978-0-8176-8406-8_6
Download citation
DOI: https://doi.org/10.1007/978-0-8176-8406-8_6
Published:
Publisher Name: Birkhäuser, Boston
Print ISBN: 978-0-8176-8405-1
Online ISBN: 978-0-8176-8406-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)