Skip to main content
SpringerLink
Log in

Numerical stability of orthogonalization methods with a non-standard inner product

  • Published:
BIT Numerical Mathematics Aims and scope Submit manuscript

Abstract

In this paper we study the numerical properties of several orthogonalization schemes where the inner product is induced by a nontrivial symmetric and positive definite matrix. We analyze the effect of its conditioning on the factorization and the loss of orthogonality between vectors computed in finite precision arithmetic. We consider the implementation based on the backward stable eigendecomposition, modified and classical Gram–Schmidt algorithms, Gram–Schmidt process with reorthogonalization as well as the implementation motivated by the AINV approximate inverse preconditioner.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Abdelmalek, N.I.: Roundoff error analysis for Gram–Schmidt method and solution of linear least squares problems. BIT Numer. Math. 11(4), 354–367 (1971)

    Article  MathSciNet  Google Scholar 

  2. Barlow, J.L., Smoktunowicz, A.: Reorthogonalized Block Classical Gram–Schmidt. Available electronically at http://arxiv.org/pdf/1108.4209.pdf

  3. Barrlund, A.: Perturbation bounds for the LDL T and LU decompositions. BIT Numer. Math. 31(2), 358–363 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  4. Benzi, M.: Preconditioning techniques for large linear systems: a survey. J. Comput. Phys. 182(2), 418–477 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  5. Benzi, M., Cullum, J.K., Tůma, M.: Robust approximate inverse preconditioning for the conjugate gradient method. SIAM J. Sci. Comput. 22(4), 1318–1332 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  6. Benzi, M., Meyer, C.D., Tůma, M.: A sparse approximate inverse preconditioner for the conjugate gradient method. SIAM J. Sci. Comput. 17(5), 1135–1149 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  7. Benzi, M., Tůma, M.: A robust incomplete factorization preconditioner for positive definite matrices. Numer. Linear Algebra Appl. 10(5–6), 385–400 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  8. Björck, Å.: Solving linear least squares problems by Gram–Schmidt orthogonalization. BIT Numer. Math. 7(1), 1–21 (1967)

    Article  MATH  Google Scholar 

  9. Björck, Å.: Numerics of Gram–Schmidt orthogonalization. Linear Algebra Appl. 197–198, 297–316 (1994)

    Article  Google Scholar 

  10. Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)

    Book  MATH  Google Scholar 

  11. Challacombe, M.: A simplified density matrix minimization for linear scaling self-consistent field theory. J. Chem. Phys. 110(5), 2332–2342 (1999)

    Article  Google Scholar 

  12. Fox, L., Huskey, H.D., Wilkinson, J.H.: Notes on the solution of algebraic linear simultaneous equations. Q. J. Mech. Appl. Math. 1(1), 149–173 (1948)

    Article  MathSciNet  MATH  Google Scholar 

  13. Giraud, L., Langou, J., Rozložník, M.: The loss of orthogonality in the Gram–Schmidt orthogonalization process. Comput. Math. Appl. 50(7), 1069–1075 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  14. Giraud, L., Langou, J., Rozložník, M., van den Eshof, J.: Rounding error analysis of the classical Gram–Schmidt orthogonalization process. Numer. Math. 101(1), 97–100 (2005)

    Article  Google Scholar 

  15. Golub, G.H., Van Loan, C.F.: Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences, 3rd edn. Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  16. Gulliksson, M.: Backward error analysis for the constrained and weighted linear least squares problem when using the weighted QR factorization. SIAM J. Matrix Anal. Appl. 16(2), 675–687 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  17. Gulliksson, M.: On the modified Gram–Schmidt algorithm for weighted and constrained linear least squares problems. BIT Numer. Math. 35(4), 453–468 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  18. Gulliksson, M., Wedin, P.-Å.: Modifying the QR-decomposition to constrained and weighted linear least squares. SIAM J. Matrix Anal. Appl. 13(4), 1298–1313 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  19. Hestenes, M.R.: Inversion of matrices by biorthogonalization and related results. J. SIAM 6(1), 51–90 (1958)

    MathSciNet  MATH  Google Scholar 

  20. Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–435 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  21. Higham, N.J.: Accuracy and Stability of Numerical Algorithms, 2nd edn. SIAM, Philadelphia (2002)

    Book  MATH  Google Scholar 

  22. Householder, A.S.: Terminating and nonterminating iterations for solving linear systems. J. SIAM 3(2), 67–72 (1955)

    MathSciNet  MATH  Google Scholar 

  23. Kharchenko, S.A., Kolotilina, L.Y., Nikishin, A.A., Yeremin, A.Y.: A robust AINV-type method for constructing sparse approximate inverse preconditioners in factored form. Numer. Linear Algebra Appl. 8(3), 165–179 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  24. Lawson, C.L., Hanson, R.J.: Solving Least Squares Problems. Prentice-Hall Series in Automatic Computation. Prentice-Hall, Englewood Cliffs (1974)

    MATH  Google Scholar 

  25. Martin, R.S., Wilkinson, J.H.: Reduction of the symmetric eigenproblem Ax=λBx and related problems to standard form. In: Handbook Series Linear Algebra. Numer. Math., vol. 11(2), pp. 99–110 (1968)

    Google Scholar 

  26. Mazzia, A., Pini, G.: Numerical performance of preconditioning techniques for the solution of complex sparse linear systems. Commun. Numer. Methods Eng. 19(1), 37–48 (2003)

    Article  MATH  Google Scholar 

  27. Morris, J.: An escalator process for the solution of linear simultaneous equations. Philos. Mag. 37(7), 106–120 (1946)

    MathSciNet  MATH  Google Scholar 

  28. Saberi Najafi, H., Ghazvini, H.: Weighted restarting method in the weighted Arnoldi algorithm for computing the eigenvalues of a nonsymmetric matrix. Appl. Math. Comput. 175(2), 1276–1287 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  29. Sun, J.-G.: Perturbation bounds for the Cholesky and QR factorizations. BIT Numer. Math. 31, 341–352 (1991)

    Article  MATH  Google Scholar 

  30. Parlett, B.N.: The Symmetric Eigenvalue Problem. Prentice-Hall Series in Computational Mathematics. Prentice-Hall, Englewood Cliffs (1980)

    MATH  Google Scholar 

  31. Pietrzykowski, T.: Projection method. Prace ZAM Ser. A 8, 9 (1960)

    MathSciNet  Google Scholar 

  32. Purcell, E.W.: The vector method of solving simultaneous linear equations. J. Math. Phys. 32, 150–153 (1953)

    MathSciNet  Google Scholar 

  33. Smoktunowicz, A., Barlow, J.L., Langou, J.: A note on the error analysis of classical Gram–Schmidt. Numer. Math. 105(2), 299–313 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  34. Thomas, S.J.: A block algorithm for orthogonalization in elliptic norms. Lect. Notes Comput. Sci. 634, 379–385 (1992)

    Article  Google Scholar 

  35. Thomas, S.J., Zahar, R.V.M.: Efficient orthogonalization in the M-norm. Congr. Numer. 80, 23–32 (1991)

    MathSciNet  Google Scholar 

  36. Thomas, S.J., Zahar, R.V.M.: An analysis of orthogonalization in elliptic norms. Congr. Numer. 86, 193–222 (1992)

    MathSciNet  Google Scholar 

  37. Wilkinson, J.H.: The Algebraic Eigenvalue Problem. Clarendon Press, Oxford (1965)

    MATH  Google Scholar 

  38. Yin, J.-F., Yin, G.-J., Ng, M.: On adaptively accelerated Arnoldi method for computing PageRank. Numer. Linear Algebra Appl. 19(1), 73–85 (2012)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank for the fruitful discussion and useful comments to G. Meurant and S. Gratton as well as to M. Hochstenbach and anonymous referee.

Author information

Authors and Affiliations

  1. Institute of Computer Science, Academy of Sciences of the Czech Republic, Pod vodárenskou věží 2, 182 07, Prague 8, Czech Republic

    Miroslav Rozložník & Miroslav Tůma

  2. Faculty of Mathematics and Information Science, Warsaw University of Technology, Pl. Politechniki 1, 00-661, Warsaw, Poland

    Alicja Smoktunowicz

  3. Institute of Novel Technologies and Applied Informatics, Technical University of Liberec, Hálkova 6, 461 17, Liberec, Czech Republic

    Jiří Kopal

Authors
  1. Miroslav Rozložník
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miroslav Tůma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alicja Smoktunowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiří Kopal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miroslav Rozložník.

Additional information

Communicated by Michiel Hochstenbach.

The work of M. Rozložník and M. Tůma was supported by Grant Agency of the Czech Republic under the project 108/11/0853 and by the Grant Agency of the Academy of Sciences of the Czech Republic under the project IAA100300802. The work of J. Kopal was supported by the Ministry of Education of the Czech Republic under the project no. 7822/115.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rozložník, M., Tůma, M., Smoktunowicz, A. et al. Numerical stability of orthogonalization methods with a non-standard inner product. Bit Numer Math 52, 1035–1058 (2012). https://doi.org/10.1007/s10543-012-0398-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10543-012-0398-9

Keywords

Mathematics Subject Classification (2010)

Search

Navigation