Skip to main content
SpringerLink
Log in

Efficient implementations of the modified Gram–Schmidt orthogonalization with a non-standard inner product

  • Special Feature: Original Paper
  • International Workshop on Eigenvalue Problems: Algorithms; Software and Applications, in Petascale Computing (EPASA2018)
  • Published:
Japan Journal of Industrial and Applied Mathematics Aims and scope Submit manuscript

Abstract

The modified Gram–Schmidt (MGS) orthogonalization is one of the most well-used algorithms for computing the thin QR factorization. MGS can be straightforwardly extended to a non-standard inner product with respect to a symmetric positive definite matrix A. For the thin QR factorization of an \(m \times n\) matrix with the non-standard inner product, a naive implementation of MGS requires 2n matrix-vector multiplications (MV) with respect to A. In this paper, we propose n-MV implementations: a high accuracy (HA) type and a high performance type, of MGS. We also provide error bounds of the HA-type implementation. Numerical experiments and analysis indicate that the proposed implementations have competitive advantages over the naive implementation in terms of both computational cost and accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. The definition of \([\varDelta E^{(2)}]_{ij}\) given in [11] is actually the definition of \([\varDelta E^{(2)}]_{ji}\). We corrected this in Eq. (27).

References

  1. Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)

    Book  MATH  Google Scholar 

  2. Dubrulle, A.A.: Retooling the method of block conjugate gradients. ETNA 12, 216–233 (2001)

    MathSciNet  MATH  Google Scholar 

  3. ELSES matrix library. http://www.elses.jp/matrix/. Accessed 15 Apr 2018

  4. Essai, A.: Weighted FOM and GMRES for solving nonsymmetric linear systems. Numer. Algorithms 18, 277–292 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  5. Gulliksson, M.: On the modified Gram–Schmidt algorithm for weighted and constrained linear least squares problems. BIT Numer. Math. 35, 453–468 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  6. Higham, N.J.: Accuracy and Stability of Numerical Algorithms, 2nd edn. SIAM, Philadelphia (2002)

    Book  MATH  Google Scholar 

  7. Imakura, A., Du, L., Tadano, H.: A weighted block GMRES method for solving linear systems with multiple right-hand sides. JSIAM Lett. 5, 65–68 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  8. Imakura, A., Sakurai, T.: Block Krylov-type complex moment-based eigensolvers for solving generalized eigenvalue problems. Numer. Algorithm 75, 413–433 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  9. Knyazev, A.V.: Toward the optimal preconditioned eigensolver: locally optimal block preconditioned conjugate gradient method. SIAM J. Sci. Comput. 23, 517–541 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  10. Lowery, B.R., Langou, J.: Stability analysis of QR factorization in an oblique inner product. arXiv:1401.5171 [math.NA] (2014)

  11. Rozložník, M., Tůma, M., Smoktunowicz, A., Kopal, J.: Numerical stability of orthogonalization methods with a non-standard inner product. BIT 52, 1035–1058 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  12. Smoktunowicz, A., Barlow, J.L., Langou, J.: A note on the error analysis of classical Gram–Schmidt. Numer. Math. 105, 299–313 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  13. Stewart, G.W.: Matrix Algorithms Volume II: Eigensysmtems. SIAM, Philadelphia (2001)

    Book  Google Scholar 

  14. Trefethen, L.N.: Householder triangularization of a quasimatrix. IMA J. Numer. Anal. 30, 887–897 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  15. Trefethen, L.N., Bau III, D.: Numerical Linear Algebra. SIAM, Philadelphia (1997)

    Book  MATH  Google Scholar 

  16. Yamamoto, Y., Nakatsukasa, Y., Yanagisawa, Y., Fukaya, T.: Roundoff error analysis of the CholeskyQR2 algorithm in an oblique inner product. JSIAM Lett. 8, 5–8 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  17. Yanagisawa, Y., Nakatsukasa, Y., Fukaya, T.: Cholesky-QR and Householder-QR factorizations in nonstandard inner product spaces. International Workshop on Eigenvalue Problems: Algorithms; Software and Applications, in Petascale Computing (EPASA2014) (2014)

  18. Zhao, J.Q.: S-Orthogonal QR decomposition algorithms on multicore systems. ProQuest Dissertations Publishing, University of California, Davis (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Engineering, Information and Systems, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8573, Japan

    Akira Imakura

  2. Department of Communication Engineering and Informatics, The University of Electro-Communications, 1-5-1 Chofugaoka, Chofu, Tokyo, 182-8585, Japan

    Yusaku Yamamoto

Authors
  1. Akira Imakura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yusaku Yamamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Akira Imakura.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The present study is supported in part by Japan Science and Technology Agency, ACT-I (no. JPMJPR16U6) and the Japanese Ministry of Education, Culture, Sports, Science and Technology, Grant-in-Aid for Scientific Research (nos. 16KT0016, 17H02828, 17K12690, 17K1996)

Appendix. Row-oriented versions

Appendix. Row-oriented versions

figure e
figure f

In Appendix, we focus on the row-oriented version of MGS (Algorithm 5) and introduce n-MV implementations: MGS-HA(row) and MGS-HP(row).

A naive implementation with 2n MV of the row-oriented MGS is shown in Algorithm 6. As is the case with the column-oriented versions (Algorithms 3, 4), we can derive n-MV implementations of the row-oriented MGS. The vector \({\varvec{p}}_i = A{\varvec{q}}_i\) is computed without MV by

$$\begin{aligned} {\varvec{p}}_i \left( = A {\varvec{q}}_i \right) = \frac{ {\varvec{x}}_i^{(i-1)} }{ r_{ii} }, \end{aligned}$$

as well as (4) and the vector \({\varvec{x}}_i^{(i-1)} = A{\varvec{z}}_i^{(i-1)}\) is computed without a sequential MV by

$$\begin{aligned} {\varvec{x}}_i^{(i-1)} \left( = A{\varvec{z}}_i^{(i-1)}\right) = {\varvec{x}}_i^{(0)} - \sum _{j=i+1}^{n} r_{ij} {\varvec{p}}_i \end{aligned}$$

as well as (5) for MGS-HP(row). The algorithms of MGS-HA(row) and MGS-HP(row) are shown in Algorithms 7 and 8, respectively.

figure g
figure h

The proposed concept can also be applied to the row-oriented version of CGS for its n-MV implementations, CGS-HA(row) and CGS-HP(row). It is also noted that the HP-type of row-oriented versions, MGS-HP(row) and CGS-HP(row), are equivalent to the algorithms introduced in [2] to use in the block conjugate gradient method for solving linear systems with multiple right-hand sides. However, the performance of these algorithms are not analyzed and evaluated in [2], because the main objective of [2] is to propose the block conjugate gradient method.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Imakura, A., Yamamoto, Y. Efficient implementations of the modified Gram–Schmidt orthogonalization with a non-standard inner product. Japan J. Indust. Appl. Math. 36, 619–641 (2019). https://doi.org/10.1007/s13160-019-00356-4

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13160-019-00356-4

Keywords

Search

Navigation