# Convergence on successive over-relaxed iterative methods for non-Hermitian positive definite linear systems

- 761 Downloads

**Part of the following topical collections:**

## Abstract

Some convergence conditions on successive over-relaxed (SOR) iterative method and symmetric SOR (SSOR) iterative method are proposed for non-Hermitian positive definite linear systems. Some examples are given to demonstrate the results obtained.

## Keywords

SOR iterative method SSOR iterative method non-Hermitian positive definite linear system convergence## MSC

15A06 15A15 15A48## 1 Introduction

*n*linear equations

*A*is a large sparse non-Hermitian positive definite matrix, that is, its Hermitian part \(H=(A+A^{*})/2\) is Hermitian positive definite, where \(A^{*}\) denotes the conjugate transpose of a matrix

*A*. In order to solve system (1) by iterative methods, usually, efficient splittings of the coefficient matrix

*A*are required. For example, the classic Jacobi and Gauss-Seidel iterations [1, 2, 3] split the matrix

*A*into its diagonal and off-diagonal parts. Recently, considerable interest appears in the work on the Hermitian and skew-Hermitian splitting (HSS) method for this system introduced by Bai et al. [4] and some generalized HSS methods such as the NSS method [5], PSS method [6], PHSS method [7, 8], and LHSS method [9], and several significant theoretical results are proposed. Meanwhile, these methods and theoretical results are applied to this linear system directly or indirectly (as a preconditioner); see [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]. It is shown in [3, 15, 16] that the successive over-relaxed (SOR) iterative method and symmetric SOR (SSOR) iterative method for Hermitian positive definite linear systems are convergent. But, is the same true for these iterative methods for non-Hermitian positive definite linear systems? In this paper, we mainly study the convergence of the SOR iterative and SSOR iterative method for non-Hermitian positive definite linear systems and propose some convergence conditions.

## 2 Main results

The main theoretical results in this paper are given to propose some convergence conditions on the SOR and SSOR iterative methods. For convenience of presentation, in Section 2.1, we give some classic iterative methods for linear systems.

### 2.1 SOR iterative methods

*A*in (1) into

*L*is a lower triangular matrix, and

*U*is a strictly upper triangular matrix. Then,

*I*is the identity matrix. Without loss of generality, in (2), we can assume that \(D=I\). Decomposition (2) becomes

### 2.2 Convergence results for SOR iterative method

Throughout the paper, we denote the conjugate transpose of a vector \(x\in\mathbb{C}\) and a matrix \(A\in\mathbb{C}^{n\times n}\), the spectrum of *A*, and the set of all eigenvectors of *A* by \(x^{*}\) and \(A^{*}\), \(\sigma(A)=\{\lambda\in\mathbb{C}: \operatorname{det}(\lambda I-A)=0\}\), and \(\vartheta(A)=\{x\in\mathbb{C}^{n}: Ax=\lambda x, \lambda\in\sigma(A)\} \), respectively. Let \(\rho(A)=\max_{\lambda\in\sigma(A)\vert \lambda \vert }\) be the spectral radius of *A*, and \(\vartheta_{\max}(A)=\{x\in\vartheta (A): Ax=\lambda x, \vert \lambda \vert =\rho(A)\}\). The following lemmas will be used in this paper.

### Lemma 1

*Let* \(A=M-N\in\mathbb{C}^{n\times n}\) *with* *M* *nonsingular and* \(F=M^{-1}N\). *Then* \(\rho(F)<1\) *if and only if* \(H-F^{*}HF\) *is Hermitian positive definite on* \(\vartheta(F)\) *for any* \(n\times n\) *Hermitian positive definite matrix* *H*.

### Proof

*λ*be any eigenvalue of the matrix

*F*, and \(x\in\vartheta (F)\) be a corresponding eigenvector with \(x\neq0\), that is, \(Fx=\lambda x\). Then, for all \(x\in\vartheta(F)\), we have

From Lemma 1 we have the following conclusion.

### Lemma 2

*Let* \(A=M-N\in\mathbb{C}^{n\times n}\) *with* *M* *nonsingular and* \(F=M^{-1}N\). *Then* \(\rho(F)<1\) *if and only if* \(M^{*}M-N^{*}N\) *is Hermitian positive definite on* \(\vartheta(F)\).

In what follows, we propose some convergence conditions on SOR and SSOR iterative methods for non-Hermitian positive definite linear systems.

### Theorem 1

*Let*\(A=I-L-U\in\mathbb{C}^{n\times n}\)

*be positive definite with*\(H=(A^{*}+A)/2\),

*and*\(L_{\omega}\)

*be defined in*(6).

*Then the SOR iterative method converges to the unique solution of*(1)

*for any choice of the initial guess*\(x_{0}\),

*that is*, \(\rho(L_{\omega})<1\)

*if and only if*\(0<\omega<1/\eta\)

*if*\(\eta>0\)

*or*\(1/\eta<\omega\)

*if*\(\eta <0\)

*or*\(0<\omega\)

*if*\(\eta=0\),

*where*

*for any*\(x\in\vartheta_{\max}(L_{\omega})\).

### Proof

*λ*is an eigenvalue of \(L_{\alpha}\) with \(\vert \lambda \vert =\rho(L_{\alpha})\) and \(x\in\vartheta_{\max }(L_{\omega})\) is its corresponding eigenvector. Then, \(M^{-1}Nx=\lambda x\), \(Nx=\lambda M x\), and consequently \(x^{*}N^{*}Nx= \vert \lambda \vert ^{2} x^{*}M^{*}Mx\). Thus,

*A*is positive definite, \(H=(A+A^{*})/2\) is Hermitian positive definite, that is, \(x^{*}Hx>0\) for any \(x\neq0\), \(x\in\mathbb{C}^{n}\). As a consequence, \(x^{*}Hx>0\) for any \(x\in\vartheta_{\max}(L_{\omega})\). Thus, \(\rho(L_{\omega})<1\) if and only if \(1/\omega>\eta\). Again, since \(1/\omega>\eta\) holds if and only if \(0<\omega<1/\eta\) if \(\eta>0\) or \(1/\eta<\omega\) if \(\eta<0\) or \(0<\omega\) if \(\eta=0\), this completes the proof. □

### Theorem 2

*Let*\(A=I-L-U\in\mathbb{C}^{n\times n}\)

*be positive definite with*\(H=(A^{*}+A)/2\),

*and*\(L_{\omega}\)

*be defined in*(6).

*Then the SOR iterative method converges to the unique solution of*(1)

*for any choice of the initial guess*\(x_{0}\),

*that is*, \(\rho(L_{\omega})<1\)

*if and only if*\(0<\omega<1/(1-\tau)\)

*if*\(\tau<1\)

*or*\(\omega>1/(1-\eta)\)

*if*\(\tau>1\)

*or*\(\omega>0\)

*if*\(\tau=1\),

*where*

*for any*\(x\in\vartheta_{\max}(L_{\omega})\).

### Proof

### Theorem 3

*Let*\(A-L-U\in\mathbb{C}^{n\times n}\)

*be positive definite with*\(H=(A^{*}+A)/2\),

*and*\(L_{\omega}\)

*be defined in*(6).

*Suppose that*

*A*

*satisfies one of the two conditions*: (i) \(\Vert T\Vert _{2}\leq1\)

*and*\(0<\omega<1\); (ii) \(\Vert T\Vert _{2}>1\)

*and*\(0<\omega<\omega_{0}\).

*Then the SOR iterative method converges to the unique solution of*(1)

*for any choice of the initial guess*\(x_{0}\),

*that is*, \(\rho (L_{\omega})<1\),

*where*

### Proof

(ii) If \(\Vert T\Vert _{2}>1\) and \(0<\omega<\omega_{0}\), then by the same method we can prove that \(\rho(L_{\omega})\leq \Vert M^{-1}N\Vert _{2}<1\), that is, the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\). This completes the proof. □

### Theorem 4

*Let*\(A=I-L-U\in\mathbb{C}^{n\times n}\)

*be positive definite with*\(H=(A^{*}+A)/2\),

*and*\(L_{\omega}\)

*be defined in*(6).

*Suppose that*

*A*

*satisfies one of the two conditions*: (i) \(\sigma _{\min}(F)\leq1\)

*and*\(0<\omega<1\); (ii) \(\sigma_{\min}(F)>1\)

*and*\(0<\omega<\omega_{1}\).

*Then the SOR iterative method converges to the unique solution of*(1)

*for any choice of the initial guess*\(x_{0}\),

*that is*, \(\rho(L_{\omega})<1\),

*where*\(\sigma_{\min}(F)\)

*denotes the minimal singular value of the matrix*

*F*,

*and*

### Proof

(ii) If \(\sigma_{\min}(F)>1\) and \(0<\omega<\omega_{0}\), then by the same method we can prove that \(\rho(L_{\omega})\leq \Vert M^{-1}N\Vert _{2}<1\), that is, the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\). This proof is completed. □

### Remark 1

- (1)
It follows from Theorem 3 that whether the forward Gauss-Seidel method converges or not, there always exists a positive constant \(\omega_{0}\) such that, for \(0<\omega<\omega_{0}\), the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\).

- (2)
Theorem 4 shows that whether the backward Gauss-Seidel method converges or not, there always exists a positive constant \(\omega _{1}\) such that, for \(0<\omega<\omega_{1}\) the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\).

### 2.3 Convergence results for SSOR iterative method

In what follows, convergence results for the SSOR iterative method are established for non-Hermitian positive definite linear systems.

### Theorem 5

*Let* \(A=I-L-U\in\mathbb{C}^{n\times n}\) *with* \(A_{L}=I-L-L^{*}\) *and* \(A_{U}=I-U-U^{*}\) *both Hermitian positive semidefinite*. *Then for* \(0<\omega<2\), *the SSOR iterative method converges to the unique solution of* (1) *for any choice of the initial guess* \(x_{0}\), *that is*, \(\rho(S_{\omega})<1\) *for* \(0<\omega<2\), *where* \(S_{\omega}\) *is defined in* (8).

### Proof

*y*be an eigenvector corresponding to the eigenvalue

*μ*of \(S_{\omega}\) with \(\vert \mu \vert =\rho(S_{\omega})\). Then

We know from (5) and (8) that the SSOR method changes into SGS method when \(\omega=1\). Thus, it follows from Theorem 5 that the following conclusion is true.

### Theorem 6

*Let* \(A=I-L-U\in\mathbb{C}^{n\times n}\) *with* \(A_{L}=I-L-L^{*}\) *and* \(A_{U}=I-U-U^{*}\) *both Hermitian positive semidefinite*. *Then the SGS iterative method converges to the unique solution of* (1) *for any choice of the initial guess* \(x_{0}\).

## 3 Numerical examples

Two numerical examples are given in this section to demonstrate that the convergence results are true and the SOR iterative and SSOR iterative methods are very effective.

### Example 1

*A*of the linear system (1) is given as

*A*is positive definite. By Matlab 7.0 computations we get Table 1.

**The comparison results on the spectral radius of the SOR iterative matrix of** **A****for different** **ω**

| 0.01 | 0.02 | 0.0281 | 0.05 | 0.1 | 0.15 | 0.20 | 0.2292 |

\(\rho(L_{\omega})\) | 0.9929 | 0.9858 | 0.9801 | 0.9645 | 0.9290 | 0.8936 | 0.8585 | 0.8382 |

| 0.25 | 0.30 | 0.42 | 0.52 | 0.6 | 0.72 | 0.82 | 0.87 |

\(\rho(L_{\omega})\) | 0.8239 | 0.7901 | 0.7155 | 0.6679 | 0.6512 | 0.7078 | 0.8769 | 1.0007 |

Table 1 shows that for the matrix *A*, the SOR iterative method is convergent for \(0<\omega<0.87\). By direct computations, \(\Vert T\Vert _{2}=4.0258>1\) when \(\omega_{0}=0.0281\). Thus, Theorem 3 shows that the SOR iterative method converges for the matrix *A* when \(0<\omega <0.0281\). The same is shown in Theorem 4 for \(0<\omega<0.2292\). It follows from Table 1 that Theorems 3 and 4 are both true. However, both intervals given by the theorems are very narrow. Sometimes these intervals fail to include the optimal value such that \(\rho(L_{\omega})\) reaches minimization.

### Example 2

*A*of the linear system (1) is given as

**The comparison results on the spectral radius of the SSOR iterative matrix of** **B****for different** **ω**

| 0.125 | 0.250 | 0.375 | 0.500 | 0.625 | 0.750 | 0.875 | 0.1 |

\(\rho(S_{\omega})\) | 0.76945 | 0.56558 | 0.39654 | 0.26311 | 0.16501 | 0.10674 | 0.27623 | 0.26139 |

| 1.125 | 1.250 | 1.375 | 1.5000 | 1.625 | 1.750 | 1.875 | 1.925 |

\(\rho(S_{\omega})\) | 0.15931 | 0.18860 | 0.26281 | 0.36777 | 0.49742 | 0.64802 | 0.81648 | 0.88823 |

Table 2 shows that the spectral radius \(\rho(S_{\omega})\) gradually decreases to 0.10674 with *ω* increasing from 0.125 to 0.750, whereas \(\rho(S_{\omega})\) gradually increases from 0.15931 to 0.88823 with *ω* increasing from 1.125 to 1.925. However, when *ω* increases from 0.750 to 1.125, \(\rho(S_{\omega})\) gradually increases from 0.10674 to 0.27623 and gradually decreases from 0.27623 to 0.15931. Therefore, the optimal value of *ω* should be \(\omega _{opt}\in(0.625,1.250)\) such that the SSOR iterative method converges faster to the unique solution of (1) for any choice of the initial guess \(x_{0}\).

## 4 Conclusions

In this paper, we study the convergence of the SOR and SSOR iterative methods for non-Hermitian positive definite linear systems. we propose some necessary and sufficient conditions for the convergence of the SOR iterative method. But, these conditions are only theoretically significant and are difficult to apply to practical computations. In what follows, two conditions are presented such that there always exists a positive constant \(\omega_{0}\) (\(\omega_{1}\)) such that, for \(0<\omega<\omega_{0}\) (\(0<\omega<\omega_{1}\)), the SOR iterative method converges for linear system (1) whether the forward or backward Gauss-Seidel method converges or not.

It is more important that a practical condition for both \(A_{L}=I-L-L^{*}\) and \(A_{U}=I-U-U^{*}\) to be Hermitian positive semidefinite is proposed such that both the SSOR iterative method for any \(\omega\in(0,2)\) and the SGS iterative method converge to the unique solution of (1) for any choice of the initial guess \(x_{0}\).

## Notes

### Acknowledgements

The work was supported by the National Natural Science Foundations of China (11201362 and 11271297), the Natural Science Foundation of Shaanxi Province (2016JM1009) and Yunnan NSF Grant (2011FZ190).

## References

- 1.Golub, GH, Van Loan, CF: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996) MATHGoogle Scholar
- 2.Saad, Y: Iterative Methods for Sparse Linear Systems. PWS Publishing Company, Boston (1996) MATHGoogle Scholar
- 3.Young, DM: Iterative Solution of Large Linear Systems. Academic Press, New York (1971) MATHGoogle Scholar
- 4.Bai, ZZ, Golub, GH, Ng, MK: Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems. SIAM J. Matrix Anal. Appl.
**24**(3), 603-626 (2003) MathSciNetCrossRefMATHGoogle Scholar - 5.Bai, ZZ, Golub, GH, Ng, MK: On successive-overrelaxation acceleration of the Hermitian and skew-Hermitian splitting iterations. Numer. Linear Algebra Appl.
**17**, 319-335 (2007) MathSciNetCrossRefMATHGoogle Scholar - 6.Bai, ZZ, Golub, GH, Lu, LZ, Yin, JF: Block triangular and skew-Hermitian splitting methods for positive-definite linear systems. SIAM J. Sci. Comput.
**26**, 844-863 (2005) MathSciNetCrossRefMATHGoogle Scholar - 7.Bai, ZZ, Golub, GH: Accelerated Hermitian and skew-Hermitian splitting iteration methods for saddle-point problems. IMA J. Numer. Anal.
**27**, 1-23 (2007) MathSciNetCrossRefMATHGoogle Scholar - 8.Bai, ZZ, Golub, GH, Pan, JY: Preconditioned Hermitian and skew-Hermitian splitting methods for non-Hermitian positive semidefinite linear systems. Numer. Math.
**98**, 1-32 (2004) MathSciNetCrossRefMATHGoogle Scholar - 9.Li, L, Huang, TZ, Liu, XP: Modified Hermitian and skew-Hermitian splitting methods for non-Hermitian positive-definite linear systems. Numer. Linear Algebra Appl.
**14**, 217-235 (2007) MathSciNetCrossRefMATHGoogle Scholar - 10.Bai, ZZ, Golub, GH, Ng, MK: On inexact Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems. Linear Algebra Appl.
**428**, 413-440 (2008) MathSciNetCrossRefMATHGoogle Scholar - 11.Wang, L, Bai, ZZ: Convergence conditions for splitting iteration methods for non-Hermitian linear systems. Linear Algebra Appl.
**428**, 453-468 (2008) MathSciNetCrossRefMATHGoogle Scholar - 12.Benzi, M, Gander, M, Golub, GH: Optimization of the Hermitian and skew-Hermitian splitting iteration for saddle-point problems. BIT Numer. Math.
**43**, 881-900 (2003) MathSciNetCrossRefMATHGoogle Scholar - 13.Benzi, M, Golub, GH: A preconditioner for generalized saddle point problems. SIAM J. Matrix Anal. Appl.
**26**, 20-41 (2004) MathSciNetCrossRefMATHGoogle Scholar - 14.Bertaccini, D, Golub, GH, Capizzano, SS, Tablino Possio, C: Preconditioned HSS method for the solution of non-Hermitian positive definite linear systems and applications to the discrete convection-diffusion equation. Numer. Math.
**99**, 441-484 (2005) MathSciNetCrossRefMATHGoogle Scholar - 15.Varga, RS: Matrix Iterative Analysis, 2nd edn. Springer, Berlin (2000) CrossRefMATHGoogle Scholar
- 16.Demmel, JW: Applied Numerical Linear Algebra. SIAM, Philadelphia (1997) CrossRefMATHGoogle Scholar
- 17.Zhang, CY, Luo, SH, Xu, CX, Jiang, HY: Schur complements of generally diagonally dominant matrices and criterion for irreducibility of matrices. Electron. J. Linear Algebra
**18**, 69-87 (2009) MathSciNetMATHGoogle Scholar - 18.Kolotilina, LY: Nonsingularity/singularity criteria for non-strictly block diagonally dominant matrices. Linear Algebra Appl.
**359**, 133-159 (2003) MathSciNetCrossRefMATHGoogle Scholar

## Copyright information

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.