# The alternate direction iterative methods for generalized saddle point systems

- 33 Downloads

## Abstract

The paper studies two splitting forms of generalized saddle point matrix to derive two alternate direction iterative schemes for generalized saddle point systems. Some convergence results are established for these two alternate direction iterative methods. Meanwhile, a numerical example is given to show that the proposed alternate direction iterative methods are much more effective and efficient than the existing one.

## Keywords

Alternate direction iterative method Generalized saddle point system Convergence## MSC

65F10 15A15 15F10## 1 Introduction

This class of linear systems arises in many scientific and engineering applications such as a mixed finite element approximation of elliptic partial differential equations, optimization, optimal control, structural analysis and electrical networks; see [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11].

$A=\left[\begin{array}{cc}{A}_{1}& 0\\ 0& {A}_{2}\end{array}\right]$, \(B=[B_{1}, B_{2}]\), \(A_{i}\in R^{n_{i}\times n_{i}}\) for \(i=1,2\) and \(n_{1}+n_{2}=n\), and \(B_{i}\in R^{m\times n_{i}}\) for \(i=1,2\);

\(A_{i}\) is positive definite (i.e., it has positive definite symmetric part \(H_{i}=(A_{i}+A^{T}_{i})/{2}\)) for \(i=1,2\);

\(\operatorname{rank}(B)=m\).

*αI*is replaced by two nonnegative diagonal matrices \(\mathscr{D}_{1}\) and \(\mathscr{D}_{2}\) to form a new alternate direction iterative scheme; another is to propose a new splitting of \(\mathscr{A}\), i.e.,

The paper is organized as follows. Two alternate direction iterative schemes are proposed in Sect. 2. The main convergence results of these two schemes are given in Sect. 3. In Sect. 4, a numerical examples is presented to demonstrate the proposed methods are very effective and efficient in this paper. A conclusion is given in Sect. 5.

## 2 The ADI methods

*Given an initial guess*\(x^{(0)}\),

*for*\(k=0,1,2,\ldots \) ,

*until*\(\{x^{(k)}\}\)

*converges*,

*compute*

*where*\(\mathscr{D}_{1}\)

*and*\(\mathscr{D}_{2}\)

*are defined in*(8).

## 3 The convergence of the ADI methods

In this section, some convergence results on the ADI methods will be established. First, the following lemmas will used in this section.

### Lemma 1

*Let*\(A=M-N\in C^{n\times n}\)*with**A**and**M**nonsingular and let*\(T=NM^{-1}\). *Then*\(A-TAT^{*}=(I-T)(AA^{-*}M^{*}+N)(I-T^{*})\).

The proof is similar to the proof of Lemma 5.30 in [1].

### Lemma 2

*Let*\(A\in R^{n\times n}\)*be symmetric and positive definite*. *If*\(A=M-N\)*with**M**nonsingular is a splitting such that*\(M+N\)*has a nonnegative definite symmetric part*, *then*\(\|T\|_{A}=\|A^{-1/2}TA^{1/2}\|_{2}\leq 1\), *where*\(T=NM^{-1}\).

### Proof

### Lemma 3

*Let*\(\mathscr{A}_{i}\), \(\mathscr{B}_{i}\)

*and*\(\mathscr{D}_{i}\)

*be defined in*(4)

*and*(8)

*for*\(i=1,2\).

*If*\({A}_{i}\)

*has positive definite symmetric part*\(H_{i}\)

*and*\(0<\alpha \leq 2\lambda _{\mathrm{min}}(H_{i})\)

*with*\(\lambda _{\mathrm{min}}(H _{i})\)

*the smallest eigenvalue of*\(H_{i}\),

*then*

*where*\(j=2\)

*if*\(i=1\)

*and*\(j=1\)

*if*\(i=2\).

### Proof

*I*is the \((n_{1}+n_{2}+m)\times (n_{1}+n_{2}+m)\) identity matrix, and

### Theorem 1

*Consider problem* (1) *and assume that*\(\mathscr{A}\)*satisfies the assumptions above*. *Then*\(\mathscr{A}\)*is nonsingular*. *Further*, *if*\(0<\alpha \leq 2\delta \)*with*\(\delta =\min \{ \lambda _{\mathrm{min}}(H_{1}),\lambda _{\mathrm{min}}(H_{2})\}\), *then*\(\|\hat{\mathscr{L}}\|_{2}\leq 1\)*and*\(\|\hat{\mathscr{T}}\|_{2}\leq 1\).

### Proof

### Theorem 2

*Consider problem* (1) *and assume that*\(\mathscr{A}\)*satisfies the assumptions above*. *If*\(0<\alpha \leq 2\delta \)*with*\(\delta =\min \{\lambda _{\mathrm{min}}(H_{1}),\lambda _{\mathrm{min}}(H _{2})\}\), *then the iterations* (10) *and* (11) *are convergent*; *that is*, \(\rho (\mathscr{L})<1\)*and*\(\rho (\mathscr{T})<1\).

### Proof

*λ*is an eigenvalue of \(\hat{\mathscr{{L}}}(\alpha )\) satisfying \(|\lambda |=\rho ( \hat{\mathscr{{L}}})\) and

*x*is the corresponding eigenvector with \(\|x\|_{2}=1\) (note that it must have \(x\neq 0\)). Then \(\hat{\mathscr{{L}}}x=\lambda x\) and consequently,

In what follows we will prove by contradiction that \(u=kv\) and \(u^{*}u\cdot v^{*}v=1\) do not hold simultaneously.

*x*is the eigenvector of \((\mathscr{D}_{2}-\mathscr{A}_{1})(\mathscr{D}_{1}+\mathscr{A}_{1})^{-1}( \mathscr{D}_{1}+\mathscr{A}^{*}_{1})^{-1}(\mathscr{D}_{2}-\mathscr{A} ^{*}_{1})\) and \((\mathscr{D}_{2}+\mathscr{A}^{*}_{2})^{-1}( \mathscr{D}_{1}-\mathscr{A}^{*}_{2})(\mathscr{D}_{1}-\mathscr{A}_{2})( \mathscr{D}_{2}+\mathscr{A}_{2})^{-1}\) corresponding to their having the same eigenvalue, 1, i.e.,

*E*,

*F*and

*G*denote nonzero matrices, the former equation in (31) can be written as

*λ*is an eigenvalue of \(\hat{\mathscr{L}}\) which is similar to \(\mathscr{L}(\alpha )\). Thus \(\hat{\mathscr{L}}\) and \(\mathscr{L}=[( \mathscr{D}_{1}+\mathscr{A}_{1})(\mathscr{D}_{2}+\mathscr{A}_{2})]^{-1}[( \mathscr{D}_{2}-\mathscr{A}_{1})(\mathscr{D}_{1} -\mathscr{A}_{2})]\) have the same eigenvalue, 1. Let

*w*be the eigenvector of \(\mathscr{L}\) corresponding to the eigenvalue 1 (note that necessarily \(w\neq 0\)). One has

*w*is an eigenvector of \(\mathscr{L}(\alpha )\). Thus, \(k\neq 1\) and \(1-k\neq 0\). From the third equation in (42), one has

*t*is an integer. Then

*x*is an eigenvector of \(\hat{\mathscr{L}}(\alpha )\) with \(\|x\|_{2}=1\). By the proof above, it is easy to see that \(u=kv\) and \(u^{*}u\cdot v^{*}v=1\) do not hold simultaneously. Therefore, \(\rho [\mathscr{L}(\alpha )]=| \lambda |<1\) and consequently, the iteration (10) converges.

By the same method, we can obtain \(\rho (\mathscr{T})<1\). Therefore, iterations (10) and (11) are both convergent. This completes the proof. □

## 4 A numerical example

A numerical example is given in this section to show that the proposed alternate direction iterative methods are very effective.

### Example 1

Consider problem (1) and assume that \(\mathscr{A}\) is shown in (2), where \(A_{1}=A_{2}=\operatorname{tri}(1,1,-1)\in R^{n \times n}\), \(B_{1}=B_{2}=I_{n}\in R^{n\times n}\), an \(n\times n\) identity matrix and \(b=(1,1,\ldots ,1)^{T}\in R^{2n}\).

We conduct numerical experiments to compare the performance of the three alternate direction iterative schemes (5), (10) and (11) for the problem (1). The former scheme (5) written as Algorithm 1 (A1) was proposed denoted by Benzi et al. in [12, 13], while the latter schemes (10) and (11) written by Algorithm 2 (A2) and Algorithm 3 (A3) are proposed in this paper. These three algorithms were coded in Matlab, and all computations were performed on a HP dx7408 PC (Intel core E4500 CPU, 2.2 GHz, 1 GB RAM) with Matlab 7.9 (R2009b).

Performance of A1, A2, and A3 with different *n*

Algorithm |
| RE |
| Time (s) |
---|---|---|---|---|

A1 | 500 | 9.97e − 07 | 263 | 1.05 |

1000 | 9.56e − 07 | 260 | 10.38 | |

1500 | 9.47e − 07 | 261 | 34.12 | |

A2 | 500 | 8.99e − 07 | 17 | 0.69 |

1000 | 9.03e − 07 | 17 | 5.13 | |

1500 | 9.17e − 07 | 17 | 23.77 | |

A3 | 500 | 9.29e − 07 | 126 | 0.63 |

1000 | 9.69e − 07 | 126 | 4.37 | |

1500 | 9.90e − 07 | 125 | 12.18 |

From Table 1, we can make the following observations. (i) A2 (i.e., Algorithm 2) generally has much smaller iteration number than A1 and A3 (Algorithm 1 and Algorithm 3) when \(n=500\), \(n=1000\) and \(n=1500\); (ii) A3 has much less computing time than A2 and A1. Thus, both A2 and A3 are generally superior to A1 in terms of iteration number and computing time. Therefore, the proposed methods are more effective and efficient than the existing method.

Figure 1 shows that RE generated by A3 quickly converges to 0 with the iteration number increasing when \(n=1000\). Therefore, A2 is superior to A1 and A3 in terms of iteration number.

## 5 Conclusions

In this paper we propose two alternate direction iterative methods for generalized saddle-point systems based on two splitting forms of generalized saddle-point matrix, and then establish some convergence theorems for these two iterative methods. Finally, we present a numerical example to demonstrate that the proposed alternate direction iterative methods are superior to the existing one.

## Notes

### Acknowledgements

The authors would like to thank the anonymous referees for their valuable comments and suggestions, which actually stimulated this work.

### Availability of data and materials

Not applicable.

### Authors’ contributions

All authors contributed equally to this work. All authors read and approved the final manuscript.

### Funding

The work was supported by the National Natural Science Foundations of China (11601409, 11201362), the Natural Science Foundation of Shaanxi Province of China (2016JM1009), the Natural Science Foundation of Department of Shaanxi Province of China (2017JK0344), the Key Projects of Social Science Planning of Gansu Province (ZD007) and 2018 Strategic Research Projects of the Scientific Research Projects of Institutions of Higher Learning of Gansu Province (2018f-20).

### Competing interests

The authors declare that they have no competing interests.

## References

- 1.Berman, A., Plemmons, R.J.: Nonnegative Matrices in the Mathematical Sciences. Academic Press, New York (1979) zbMATHGoogle Scholar
- 2.Bai, Z.-Z., Golub, G.H., Ng, M.K.: On successive-overrelaxation acceleration of the Hermitian and skew-Hermitian splitting iterations. Numer. Linear Algebra Appl.
**17**, 319–335 (2007) MathSciNetCrossRefGoogle Scholar - 3.Bai, Z.-Z., Golub, G.H.: Accelerated Hermitian and skew-Hermitian splitting iteration methods for saddle-point problems. IMA J. Numer. Anal.
**27**, 1–23 (2007) MathSciNetCrossRefGoogle Scholar - 4.Bai, Z.-Z., Golub, G.H., Lu, L.-Z., Yin, J.-F.: Block triangular and skew-Hermitian splitting methods for positive-definite linear systems. SIAM J. Sci. Comput.
**26**, 844–863 (2005) MathSciNetCrossRefGoogle Scholar - 5.Bai, Z.-Z., Golub, G.H., Pan, J.-Y.: Preconditioned Hermitian and skew-Hermitian splitting methods for non-Hermitian positive semidefinite linear systems. Numer. Math.
**98**, 1–32 (2004) MathSciNetCrossRefGoogle Scholar - 6.Bai, Z.-Z., Golub, G.H., Ng, M.K.: On inexact Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems. Linear Algebra Appl.
**428**, 413–440 (2008) MathSciNetCrossRefGoogle Scholar - 7.Li, L., Huang, T.-Z., Liu, X.-P.: Modified Hermitian and skew-Hermitian splitting methods for non-Hermitian positive-definite linear systems. Numer. Linear Algebra Appl.
**14**, 217–235 (2007) MathSciNetCrossRefGoogle Scholar - 8.Benzi, M., Szyld, D.B.: Existence and uniqueness of splittings for stationary iterative methods with applications to alternating methods. Numer. Math.
**76**, 309–321 (1997) MathSciNetCrossRefGoogle Scholar - 9.Benzi, M., Gander, M., Golub, G.H.: Optimization of the Hermitian and skew-Hermitian splitting iteration for saddle-point problems BIT Numer. Math.
**43**, 881–900 (2003) MathSciNetCrossRefGoogle Scholar - 10.Benzi, M., Golub, G.H.: A preconditioner for generalized saddle point problems. SIAM J. Matrix Anal. Appl.
**26**, 20–41 (2004) MathSciNetCrossRefGoogle Scholar - 11.Benzi, M., Golub, G.H., Liesen, J.: Numerical solution of saddle point problems. Acta Numer.
**14**, 1–137 (2005) MathSciNetCrossRefGoogle Scholar - 12.Benzi, M., Ng, M., Niu, Q., Wang, Z.: A relaxed dimensional factorization preconditioner for the incompressible Navier–Stokes equations. J. Comput. Phys.
**230**, 6185–6202 (2011) MathSciNetCrossRefGoogle Scholar - 13.Benzi, M., Guo, X.-P.: A dimensional split preconditioner for Stokes and linearized Navier–Stokes equations. Appl. Numer. Math.
**61**, 66–76 (2011) MathSciNetCrossRefGoogle Scholar

## Copyright information

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.