Advertisement

On Convergence Speed of Parallel Variants of BiCGSTAB for Solving Linear Equations

  • Kuniyoshi AbeEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 946)

Abstract

A number of hybrid Bi-Conjugate Gradient (Bi-CG) methods such as the Bi-CG STABilized (BiCGSTAB) method have been developed for solving linear equations. BiCGSTAB has been most often used for efficiently solving the linear equations, but we have sometimes seen the convergence behavior with a long stagnation phase. In such cases, it is important to have Bi-CG coefficients that are as accurate as possible, and the stabilization strategy for improving the accuracy of the Bi-CG coefficients has been proposed. In present petascale high-performance computing hardware, the main bottleneck of Krylov subspace methods for efficient parallelization is the inner products which require a global reduction. The parallel variants of BiCGSTAB such as communication avoiding and pipelined BiCGSTAB reducing the number of global communication phases and hiding the communication latency have been proposed. However, the numerical stability, specifically, the convergence speed of the parallel variants of BiCGSTAB has not previously been clarified on problems with situations where the convergence is slow (strongly affected by rounding errors). In this paper, therefore, we examine the convergence speed between the standard BiCGSTAB and the parallel variants, and the effectiveness of the stabilization strategy by numerical experiments on the problems where the convergence has a long stagnation phase.

Keywords

Linear equations Krylov subspace methods Bi-conjugate gradient stabilized method Convergence speed Numerical stability Parallel variants 

References

  1. 1.
    Abe, K., Sleijpen, G.L.G.: Solving linear equations with a stabilized GPBiCG method. Appl. Numer. Math. 67, 4–16 (2013)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Carson, E.: Communication-avoiding Krylov subspace methods in theory and practice. Ph.D. thesis, U.C. Berkeley, EECS (2015)Google Scholar
  3. 3.
    Carson, E., Demmel, J.: A residual replacement strategy for improving the maximum attainable accuracy of s-step Krylov subspace methods. SIAM J. Matrix Anal. Appl. 35, 22–43 (2014)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Carson, E., Knight, N., Demmel, J.: Avoiding communication in nonsymmetric Lanczos-based Krylov subspace methods. SIAM J. Sci. Comput. 35, S42–S61 (2013)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Chronopoulos, A.T.: \(s\)-step iterative methods for (non)symmetric (in)definite linear systems. SIAM J. Numer. Anal. 28, 1776–1789 (1991)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chronopoulos, A.T., Gear, C.W.: \(s\)-step iterative methods for symmetric linear systems. J. Comput. Math. 25, 153–168 (1989)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Chronopoulos, A.T., Swanson, C.D.: Parallel iterative \(s\)-step methods for unsymmetric linear systems. Parallel Comput. 22, 623–641 (1996)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Cools, S.: Numerical stability analysis of the class of communication hiding pipelined conjugate gradient methods. arXiv:1804.02962v1 [cs.NA] (2018)
  9. 9.
    Cools, S., Vanroose, W.: The communication-hiding pipelined BiCGStab method for the parallel solution of large unsymmetric linear systems. Parallel Comput. 65, 1–20 (2017)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Cools, S., Vanroose, W.: Numerically stable variants of the communication-hiding pipelined conjugate gradients algorithm for parallel solution of large scale symmetric linear systems. arXiv:1706.05988v1 [cs.NA] (2017)
  11. 11.
    Cools, S., Yetkin, E.F., Agullo, E., Giraud, L., Vanroose, W.: Analyzing the effect of local rounding error propagation on the maximal attainable accuracy of the pipelined conjugate gradient method. SIAM J. Matrix Anal. Appl. 39, 426–450 (2018)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Demmel, J.W., Heath, M.T., van der Vorst, H.A.: Parallel numerical linear algebra. Acta Numerica 2, 111–197 (1993)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Erhel, J.: A parallel GMRES version for general sparse matrices. ETNA 3, 160–176 (1995)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Fletcher, R.: Conjugate gradient methods for indefinite systems. In: Watson, G.A. (ed.) Numerical Analysis. LNM, vol. 506, pp. 73–89. Springer, Heidelberg (1976).  https://doi.org/10.1007/BFb0080116CrossRefGoogle Scholar
  15. 15.
    Ghysels, P., Ashby, T.J., Meerbergen, K., Vanroose, W.: Hiding global communication latency in the GMRES algorithm on massively parallel machines. SIAM J. Sci. Comput. 35, C48–C71 (2013)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Ghysels, P., Vanroose, W.: Hiding global synchronization latency in the preconditioned conjugate gradient algorithm. Parallel Comput. 40, 224–238 (2014)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Gutknecht, M.H.: Variants of BiCGStab for matrices with complex spectrum. SIAM J. Sci. Comput. 14, 1020–1033 (1993)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Saad, Y.: A flexible inner-outer preconditioned GMRES algorithm. SIAM J. Sci. Stat. Comput. 14, 461–469 (1993)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Sleijpen, G.L.G., Fokkema, D.R.: BiCGstab(\(l\)) for solving linear equations involving unsymmetric matrices with complex spectrum. ETNA 1, 11–31 (1993)zbMATHGoogle Scholar
  20. 20.
    Sleijpen, G.L.G., Sonneveld, P., van Gijzen, M.B.: Bi-CGSTAB as induced dimension reduction method. Appl. Numer. Math. 60, 1100–1114 (2010)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Sleijpen, G.L.G., van der Vorst, H.A.: Maintaining convergence properties of BiCGstab methods in finite precision arithmetic. Numer. Algorithms 10, 203–223 (1995)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Sleijpen, G.L.G., van der Vorst, H.A.: An overview of approaches for the stable computation of hybrid BiCG methods. Appl. Numer. Math. 19, 235–254 (1995)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Sonneveld, P.: CGS, a fast Lanczos-type solver for nonsymmetric linear systems. SIAM J. Sci. Comput. 10, 36–52 (1989)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Sturler, E.D.: A parallel variant of GMRES(m). In: Proceedings of the 13th IMACS World Congress on Computational and Applied Mathematics, IMACS, vol. 9. Criterion Press (1991)Google Scholar
  25. 25.
    Sturler, E.D., van der Vorst, H.A.: Reducing the effect of global communication in GMRES(m) and CG on parallel distributed memory computers. Appl. Numer. Math. 18, 441–459 (1995)CrossRefGoogle Scholar
  26. 26.
    van der Vorst, H.A.: Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 13, 631–644 (1992)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Yang, L.T.: The improved CGS method for large and sparse linear systems on bulk synchronous parallel architectures. In: Proceedings of the Fifth International Conference on Algorithm and Architectures for Parallel Processing, pp. 232–237. IEEE (2002)Google Scholar
  28. 28.
    Yang, L.T., Brent, R.P.: The improved BiCGStab method for large and sparse unsymmetric linear systems on parallel distributed memory architectures. In: Proceedings of the Fifth International Conference on Algorithm and Architectures for Parallel Processing, pp. 324–328. IEEE (2002)Google Scholar
  29. 29.
    Yang, L.T., Brent, R.P.: The improved parallel BiCG method for large and sparse unsymmetric linear systems on distributed memory architectures. In: Proceedings of the 16th International Parallel and Distributed Processing Symposium, IPDPD, pp. 349–360 (2003)Google Scholar
  30. 30.
    Zhang, S.L.: GPBi-CG: generalized product-type methods based on Bi-CG for solving nonsymmetric linear systems. SIAM J. Sci. Comput. 18, 537–551 (1997)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Gifu Shotoku UniversityGifuJapan

Personalised recommendations