Abstract
The rational Krylov subspace method (RKSM) and the low-rank alternating directions implicit (LR-ADI) iteration are established numerical tools for computing low-rank solution factors of large-scale Lyapunov equations. In order to generate the basis vectors for the RKSM, or extend the low-rank factors within the LR-ADI method, the repeated solution to a shifted linear system of equations is necessary. For very large systems this solve is usually implemented using iterative methods, leading to inexact solves within this inner iteration (and therefore to “inexact methods”). We will show that one can terminate this inner iteration before full precision has been reached and still obtain very good accuracy in the final solution to the Lyapunov equation. In particular, for both the RKSM and the LR-ADI method we derive theory for a relaxation strategy (e.g. increasing the solve tolerance of the inner iteration, as the outer iteration proceeds) within the iterative methods for solving the large linear systems. These theoretical choices involve unknown quantities, therefore practical criteria for relaxing the solution tolerance within the inner linear system are then provided. The theory is supported by several numerical examples, which show that the total amount of work for solving Lyapunov equations can be reduced significantly.
Similar content being viewed by others
References
Ahmad, M.I., Szyld, D.B., van Gijzen, M.B.: Preconditioned multishift BiCG for \(H_{2}\)-optimal model reduction. SIAM J. Matrix Anal. Appl. 38(2), 401–424 (2017)
Antoulas, A.C.: Approximation of Large-Scale Dynamical Systems. Advances in Design and Control, vol. 6. SIAM, Philadelphia, PA (2005)
Baker, J., Embree, M., Sabino, J.: Fast singular value decay for Lyapunov solutions with nonnormal coefficients. SIAM J. Matrix Anal. Appl. 36(2), 656–668 (2015)
Bartels, R.H., Stewart, G.W.: Solution of the matrix equation \({AX}+{XB}={C}\): algorithm 432. Commun. ACM 15, 820–826 (1972)
Beckermann, B., Townsend, A.: On the singular values of matrices with displacement structure. SIAM J. Matrix Anal. Appl. 38(4), 1227–1248 (2017)
Benner, P., Bujanović, Z., Kürschner, P., Saak, J.: A numerical comparison of solvers for large-scale, continuous-time algebraic Riccati equations and LQR problems. SIAM J. Sci. Comput. 42(2), A957–A996 (2020)
Benner, P., Bujanović, Z., Kürschner, P., Saak, J.: RADI: a low-rank ADI-type algorithm for large scale algebraic Riccati equations. Numer. Math. 138(2), 301–330 (2018)
Benner, P., Heinkenschloss, M., Saak, J., Weichelt, H.K.: An inexact low-rank Newton-ADI method for large-scale algebraic Riccati equations. Appl. Numer. Math. 108, 125–142 (2016)
Benner, P., Kürschner, P.: Computing real low-rank solutions of Sylvester equations by the factored ADI method. Comput. Math. Appl. 67(9), 1656–1672 (2014)
Benner, P., Kürschner, P., Saak, J.: An improved numerical method for balanced truncation for symmetric second order systems. Math. Comput. Model. Dyn. Sys. 19(6), 593–615 (2013)
Benner, P., Kürschner, P., Saak, J.: Efficient Handling of complex shift parameters in the low-rank Cholesky factor ADI method. Numer. Algorithms 62(2), 225–251 (2013)
Benner, P., Kürschner, P., Saak, J.: Self-generating and efficient shift parameters in ADI methods for large Lyapunov and Sylvester equations. Electron. Trans. Numer. Anal. 43, 142–162 (2014)
Benner, P., Li, R.-C., Truhar, N.: On the ADI method for Sylvester equations. J. Comput. Appl. Math. 233(4), 1035–1045 (2009)
Benner, P., Saak, J.: Numerical solution of large and sparse continuous time algebraic matrix Riccati and Lyapunov equations: a state of the art survey. GAMM Mitteilungen 36(1), 32–52 (2013)
Bergamaschi, L., Martínez, Á.: Two-stage spectral preconditioners for iterative eigensolvers. Numer. Linear Algebra Appl. 24(3), e2084 (2017)
Berljafa, M., Güttel, S.: Generalized rational Krylov decompositions with an application to rational approximation. SIAM J. Matrix Anal. Appl. 36(2), 894–916 (2015)
Bouras, A., Frayssé, V.: Inexact matrix–vector products in Krylov methods for solving linear systems: a relaxation strategy. SIAM J. Matrix Anal. Appl. 26(3), 660–678 (2005)
Canuto, C., Simoncini, V., Verani, M.: On the decay of the inverse of matrices that are sum of Kronecker products. Linear Algebra Appl. 452, 21–39 (2014)
Crouzeix, M., Palencia, C.: The numerical range is a \((1+\sqrt{2})\)-spectral set. SIAM J. Matrix Anal. Appl. 38(2), 649–655 (2017)
Druskin, V., Knizhnerman, L.A., Simoncini, V.: Analysis of the rational Krylov subspace and ADI methods for solving the Lyapunov equation. SIAM J. Numer. Anal. 49, 1875–1898 (2011)
Druskin, V., Simoncini, V.: Adaptive rational Krylov subspaces for large-scale dynamical systems. Syst. Control Lett. 60(8), 546–560 (2011)
Freitag, M.A., Spence, A.: Convergence theory for inexact inverse iteration applied to the generalised nonsymmetric eigenproblem. Electron. Trans. Numer. Anal. 28, 40–64 (2007/08)
Freitag, M.A., Spence, A.: Shift-invert Arnoldi’s method with preconditioned iterative solves. SIAM J. Matrix Anal. Appl. 31(3), 942–969 (2009)
Gaaf, S.W., Simoncini, V.: Approximating the leading singular triplets of a large matrix function. Appl. Numer. Math. 113, 26–43 (2017)
Grasedyck, L.: Existence of a low rank or \({\cal{H}}\)-matrix approximant to the solution of a Sylvester equation. Numer. Linear Algebra Appl. 11(4), 371–389 (2004)
Güttel, S.: Rational Krylov methods for operator functions. Ph.D. thesis, Technische Universität Bergakademie Freiberg, Germany. http://eprints.ma.man.ac.uk/2586/ (2010)
Güttel, S.: Rational Krylov approximation of matrix functions: numerical methods and optimal pole selection. GAMM-Mitteilungen 36(1), 8–31 (2013)
Jaimoukha, I.M., Kasenally, E.M.: Krylov subspace methods for solving large Lyapunov equations. SIAM J. Numer. Anal. 31, 227–251 (1994)
Kressner, D., Massei, S., Robol, L.: Low-rank updates and a divide-and-conquer method for linear matrix equations. SIAM J. Sci. Comput. 41(2), A848–A876 (2019)
Kürschner, P.: Efficient low-rank solution of large-scale matrix equations. Dissertation, Otto-von-Guericke-Universität, Magdeburg, Germany. http://hdl.handle.net/11858/00-001M-0000-0029-CE18-2 (2016)
Lehoucq, R.B., Meerbergen, K.: Using generalized Cayley transformations within an inexact rational Krylov sequence method. SIAM J. Matrix Anal. Appl. 20(1), 131–148 (1998)
Li, J.-R.:Model reduction of large linear systems via low rank system Gramians. Ph.D. thesis, Massachusettes Institute of Technology (2000)
Li, J.-R., White, J.: Low rank solution of Lyapunov equations. SIAM J. Matrix Anal. Appl. 24(1), 260–280 (2002)
Opmeer, M., Reis, T., Wollner, W.: Finite-rank ADI iteration for operator Lyapunov equations. SIAM J. Control Optim. 51(5), 4084–4117 (2013)
Palitta, D., Simoncini, V.: Numerical methods for large-scale Lyapunov equations with symmetric banded data. SIAM J. Sci. Comput. 40(5), A3581–A3608 (2018)
Penzl, T.: A cyclic low rank Smith method for large sparse Lyapunov equations. SIAM J. Sci. Comput. 21(4), 1401–1418 (2000)
Ruhe, Axel: The rational Krylov algorithm for nonsymmetric eigenvalue problems. III: complex shifts for real matrices. BIT 34, 165–176 (1994)
Saad, Y.: Numerical solution of large Lyapunov equation. In: Kaashoek, M.A., van Schuppen, J.H., Ran, A.C.M. (eds.) Signal Processing. Scattering, Operator Theory and Numerical Methods, pp. 503–511. Birkhäuser, Basel (1990)
Saak, J.: Efficient numerical solution of large scale algebraic matrix equations in PDE control and model order reduction. Ph.D. thesis, TU Chemnitz. http://nbn-resolving.de/urn:nbn:de:bsz:ch1-200901642 (2009)
Sabino, J.: Solution of large-scale Lyapunov equations via the block modified Smith method. Ph.D. thesis, Rice University, Houston, TX. http://www.caam.rice.edu/tech_reports/2006/TR06-08.pdf (2007)
Shank, S.D., Simoncini, V., Szyld, D.B.: Efficient low-rank solution of generalized Lyapunov equations. Numer. Math. 134, 327–342 (2015)
Simoncini, V.: Variable accuracy of matrix–vector products in projection methods for eigencomputation. SIAM J. Numer. Anal. 43(3), 1155–1174 (2005)
Simoncini, V.: A new iterative method for solving large-scale Lyapunov matrix equations. SIAM J. Sci. Comput. 29(3), 1268–1288 (2007)
Simoncini, V.: Analysis of the rational Krylov subspace projection method for large-scale algebraic Riccati equations. SIAM J. Matrix Anal. Appl. 37(4), 1655–1674 (2016)
Simoncini, V.: Computational methods for linear matrix equations. SIAM Rev. 58(3), 377–441 (2016)
Simoncini, V., Eldén, L.: Inexact Rayleigh quotient-type methods for eigenvalue computations. BIT Numer. Math. 42(1), 159–182 (2002)
Simoncini, V., Szyld, D.B.: Theory of inexact Krylov subspace methods and applications to scientific computing. SIAM J. Sci. Comput. 25(2), 454–477 (2003)
Simoncini, V., Szyld, D.B., Monsalve, M.: On two numerical methods for the solution of large-scale algebraic Riccati equations. IMA J. Numer. Anal. 34(3), 904–920 (2014)
Soodhalter, K.: A block MINRES algorithm based on the banded Lanczos method. Numer. Algorithms 69, 473–494 (2015)
Sun, K.: Model order reduction and domain decomposition for large-scale dynamical systems. Ph.D. thesis, Rice University, Houston. http://search.proquest.com/docview/304507831 (2008)
Truhar, N., Veselić, K.: Bounds on the trace of a solution to the Lyapunov equation with a general stable matrix. Syst. Control Lett. 56(7–8), 493–503 (2007)
Truhar, N., Veselić, K.: An efficient method for estimating the optimal dampers’ viscosity for linear vibrating systems using Lyapunov equation. SIAM J. Matrix Anal. Appl. 31(1), 18–39 (2009)
van den Eshof, J., Sleijpen, G.: Inexact Krylov subspace methods for linear systems. SIAM J. Matrix Anal. Appl. 26(1), 125–153 (2004)
Wachspress, E.L.: The ADI Model Problem. Springer, New York (2013)
Wolf, T., Panzer, H.K.-F.: The ADI iteration for Lyapunov equations implicitly performs \({\cal{H}}_2\) pseudo-optimal model order reduction. Int. J. Control 89(3), 481–493 (2016)
Acknowledgements
The authors are grateful to Cost Action EU-MORNET (TD1307) and the Department of Mathematical Sciences at Bath, that provided funding for research visits of PK to the University of Bath, where substantial parts of this work have been conducted. This work was primarily generated while PK was affiliated with the Max Planck Institute for Dynamics of Complex Technical Systems in Magdeburg, Germany and MAF was affiliated with the University of Bath, United Kingdom. Furthermore, the authors thank Kirk Soodhalter (Trinity College Dublin) for insightful discussion regarding block Krylov subspace methods.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Daniel Kressner.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A: Proof of Lemma 2.2
Proof
From \(\omega \underline{H}_j=0\) we have for \(k\le j\)
which, for \(k=j\), immediately leads to \(f_j^{(j)}=-\omega _{1:j}/(h_{j+1,j}\omega _{j+1})\), the first equality in (2.21a). Similarly, it is easy to show that for \(k<j\), a left null space vector \(\hat{\omega }\in \mathbb {C}^{1\times k+1}\) of \(\underline{H}_{k}\) is given by the first \(k+1\) entries of the null vector \(\omega \) of \(\underline{H}_j\). Hence, \(f^{(k)}_k=-\omega _{1:k}/(h_{k+1,k}\omega _{k+1})\) holds for all \(k\le j\).
For computing \(f_j^{(k)}= e_k^*H_j^{-1}\), \(k<j\), we use the following partition of \(H_j\) and consider the splitting \(f^{(k)}_j = [u,y]\), \(u\in \mathbb {C}^{1\times k}\), \(y\in \mathbb {C}^{1\times k-j}\):
This structure enforces conditions on [u, y], which we now explore. First, u has to be a multiple of \(\omega _{1:k}\). Here we exploited that due to the Hessenberg structure, \(\omega _{1:k}\underline{H}_{k-1}=0\). In particular,
such that \(u_kh_{k,k-1}e_{k-1}^*H_{k-1}^{-1}=-u_{1:k-1}\). Since \(f^{(k-1)}_{k-1}=e_{k-1}^*H_{k-1}^{-1}\) we can infer \(u_{1:k-1}=-u_kh_{k,k-1}f^{(k-1)}_{k-1}\) and, consequently, (2.21c) for \(k>1\). Similarly, \([\omega _{1:k},y]\) has to satisfy
leading to
where \(e_{j-k}\) is a canonical vector of length \(j-k\). Hence,
leading to (2.21b). Finally, the normalization constant \(\phi _{j}^{(k)}\) is obtained by the requirement \([u,y]H_{1:j,k}=1\). \(\square \)
Appendix B: Setup of example msd
The example msd represents a variant of one realization of the series of examples in [52]. Set \(K:={\text {diag}}\!\left( I_3\otimes K_1,k_0+k_1+k_2+k_3\right) +k_+e_{3n}^T-e_{3n}k_+\in {\mathbb {R}}^{n_2\times n_2},~K_1:=\mathrm {tridiag}(-1,2,-1)\in {\mathbb {R}}^{n_1\times n_1}\) with \(k_0=0.1\), \(k_1=1\), \(k_2=2\), \(k_3=4\), \(n_2=3n_1+1\), and \(k_+:=[(k_1,k_2,k_3)\otimes e_{n_1}^T,0]^T\). Further, \(M_1:={\text {diag}}\!\left( m_1I_{n_1},m_2I_{n_1},m_3I_{n_1}m_0\right) \in {\mathbb {R}}^{n_2\times n_2}\) with \(m_0=1000\), \(m_1=10\), \(m_2=k_2\), \(m_3=k_3\) and \(D:=\alpha M+\beta (K+K(M^{-1}K)+K(M^{-1}K)^2+K(M^{-1}K)^3)+\nu [e_1,e_{n_1},e_{2n_1+1}][e_1,e_{n_1},e_{2n_1+1}]^T\), \(\alpha =0.8\), \(\beta =0.1\), \(\nu =16\). In [52] a different matrix D was used involving a term \(M^{\frac{1}{2}}\sqrt{M^{-\frac{1}{2}}KM^{-\frac{1}{2}}}M^{\frac{1}{2}}\) which was infeasible to set up in a large scale setting (the middle matrix square is a dense matrix). The version of D used here was similarly used in [10]. Construct the \(n\times n\), \(n:=2n_2\) block matrices \(\hat{A}:=\left[ \begin{array}{ll} 0&{}\quad I_{n_2}\\ -K&{}\quad -D\end{array}\right] \), \(\hat{M}:={\text {diag}}\!\left( I_{n_2},M_1\right) \) representing a linearization of the quadratic matrix pencil \(\lambda ^2M_1+\lambda D+K\). The right hand side factor is set up as \(\hat{B}=\left[ \begin{matrix} I_{2m}&{}0_{2m,n_2-m}&{}\left[ \begin{matrix}0_{m,n_2-m}&{}I_m\\ I_m&{}0_{m,n_2-m}\end{matrix}\right] \end{matrix}\right] ^T\in {\mathbb {R}}^{n\times 2m}\). In Sect. 4 we use \(n_1=4000\), \(m=2\) leading to \(n=24{,}002\), \(r=4\). Finally, \(A:=P^T\hat{A}P\), \(M:=P^T\hat{M}P\), \(B:=P^T\hat{B}\) as in [29, Section 5.6], where P is a perfect shuffle permutation: This leads to banded matrices A, M, except for some sole off-band entries from the low-rank update of D, resulting in noticable computational savings when applying sparse direct solvers or computing sparse (incomplete) factorizations. An alternative to exploit the structure in \(\hat{A},\hat{M}\) within RKSM, LR-ADI is described in, e.g., [10, 30].
Rights and permissions
About this article
Cite this article
Kürschner, P., Freitag, M.A. Inexact methods for the low rank solution to large scale Lyapunov equations. Bit Numer Math 60, 1221–1259 (2020). https://doi.org/10.1007/s10543-020-00813-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10543-020-00813-4