Scaling Hypre’s Multigrid Solvers to 100,000 Cores

  • Allison H. Baker
  • Robert D. Falgout
  • Tzanio V. Kolev
  • Ulrike Meier Yang


The hypre software library ( is a collection of high performance preconditioners and solvers for large sparse linear systems of equations on massively parallel machines. This paper investigates the scaling properties of several of the popular multigrid solvers and system building interfaces in hypre on two modern parallel platforms. We present scaling results on over 100,000 cores and even solve a problem with over a trillion unknowns.


Multigrid Method Linear Solver Lawrence Livermore National Laboratory Grid Level Setup Phase 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344 (LLNL-JRNL-479591).


  1. 1.
    Ashby, S.F., Falgout, R.D.: A parallel multigrid preconditioned conjugate gradient algorithm for groundwater flow simulations. Nucl. Sci. Eng. 124(1), 145–159 (1996) UCRL-JC-122359 Google Scholar
  2. 2.
    Baker, A., Falgout, R., Gamblin, T., Kolev, T., Schulz, M., Yang, U.M.: Scaling algebraic multigrid solvers: On the road to exascale. In: Wittum, G. (ed.) Competence in High Performance Computing 2010. Springer, Berlin (2012). LLNL-PROC-463941 Google Scholar
  3. 3.
    Baker, A.H., Falgout, R.D., Kolev, T.V., Yang, U.M.: Multigrid smoothers for ultra-parallel computing. SIAM J. Sci. Comput. 33, 2864–2887 (2011). LLNL-JRNL-473191 CrossRefGoogle Scholar
  4. 4.
    Baker, A.H., Falgout, R.D., Yang, U.M.: An assumed partition algorithm for determining processor inter-communication. Parallel Comput. 32(5–6), 394–414 (2006). UCRL-JRNL-215757 MathSciNetCrossRefGoogle Scholar
  5. 5.
    Baker, A.H., Gamblin, T., Schulz, M., Yang, U.M.: Challenges of scaling algebraic multigrid across modern multicore architectures. In: 25th IEEE International Symposium on Parallel and Distributed Processing, IPDPS 2011, Anchorge, AK, USA, 16–20 May, 2011 – Conference Proceedings, pp. 275–286, IEEE, 2011. LLNL-CONF-458074 Google Scholar
  6. 6.
    Baker, A.H., Schulz, M., Yang, U.M.: On the performance of an algebraic multigrid solver on multicore clusters. In: Palma, J.M.L.M., et al. (ed.) VECPAR 2010. Lecture Notes in Computer Science, vol. 6449, pp. 102–115. Springer, Berlin (2011). Google Scholar
  7. 7.
    Baldwin, C., Brown, P.N., Falgout, R.D., Jones, J., Graziani, F.: Iterative linear solvers in a 2D radiation-hydrodynamics code: methods and performance. J. Comput. Phys. 154, 1–40 (1999) UCRL-JC-130933 MATHCrossRefGoogle Scholar
  8. 8.
    Brandt, A., McCormick, S.F., Ruge, J.W.: Algebraic multigrid (AMG) for sparse matrix equations. In: Evans, D.J. (ed.) Sparsity and Its Applications. Cambridge University Press, Cambridge (1984) Google Scholar
  9. 9.
    Brown, P.N., Falgout, R.D., Jones, J.E.: Semicoarsening multigrid on distributed memory machines. SIAM J. Sci. Comput. 21(5), 1823–1834 (2000). Special issue on the Fifth Copper Mountain Conference on Iterative Methods. UCRL-JC-130720 MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Falgout, R., Jones, J., Yang, U.M.: Pursuing scalability for hypre’s conceptual interfaces. ACM Trans. Math. Softw. 31, 326–350 (2005) MATHCrossRefGoogle Scholar
  11. 11.
    Falgout, R.D.: An introduction to algebraic multigrid. Comput. Sci. Eng. 8(6), 24–33 (2006). Special issue on Multigrid Computing. UCRL-JRNL-220851 CrossRefGoogle Scholar
  12. 12.
    Falgout, R.D., Jones, J.E.: Multigrid on massively parallel architectures. In: Dick, E., Riemslagh, K., Vierendeels, J. (eds.) Multigrid Methods VI. Lecture Notes in Computational Science and Engineering, vol. 14, pp. 101–107. Springer, Berlin (2000). Proc. of the Sixth European Multigrid Conference held in Gent, Belgium, September 27-30, 1999. UCRL-JC-133948 CrossRefGoogle Scholar
  13. 13.
    Gahvari, H., Baker, A.H., Schulz, M., Yang, U.M., Jordan, K.E., Gropp, W.: Modeling the performance of an algebraic multigrid cycle on HPC platforms. In: Proceedings of the International Conference on Supercomputing (ICS 2011), pp. 172–181. ACM, New York (2011) Google Scholar
  14. 14.
    Henson, V., Yang, U.: BoomerAMG: A parallel algebraic multigrid solver and preconditioner. Appl. Numer. Math. 41, 155–177 (2002) MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    hypre: High performance preconditioners.
  16. 16.
    Kolev, T., Vassilevski, P.: Parallel auxiliary space AMG for H(curl) problems. J. Comput. Math. 27, 604–623 (2009). Special issue on Adaptive and Multilevel Methods in Electromagnetics. Also available as a Lawrence Livermore National Laboratory technical report UCRL-JRNL-237306 MathSciNetMATHCrossRefGoogle Scholar
  17. 17.
    Meier, U., Sameh, A.: The behavior of conjugate gradient algorithms on a multivector processor with a hierarchical memory. J. Comput. Appl. Math. 24, 13–32 (1988) MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    Ruge, J.W., Stüben, K.: Algebraic multigrid (AMG). In: McCormick, S.F. (ed.) Multigrid Methods. Frontiers in Applied Mathematics, vol. 3, pp. 73–130. Philadelphia, SIAM (1987) CrossRefGoogle Scholar
  19. 19.
    Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia (2003) MATHCrossRefGoogle Scholar
  20. 20.
    Schaffer, S.: A semi-coarsening multigrid method for elliptic partial differential equations with highly discontinuous and anisotropic coefficients. SIAM J. Sci. Comput. 20(1), 228–242 (1998) MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Sequoia: ASC sequoia benchmark codes.
  22. 22.
    Sterck, H.D., Falgout, R.D., Nolting, J.W., Yang, U.M.: Distance-two interpolation for parallel algebraic multigrid. Numer. Linear Algebra Appl. 15(2–3), 115–139 (2008). Special issue on Multigrid Methods. UCRL-JRNL-230844 MathSciNetMATHCrossRefGoogle Scholar
  23. 23.
    Sterck, H.D., Yang, U.M., Heys, J.J.: Reducing complexity in parallel algebraic multigrid preconditioners. SIAM J. Matrix Anal. Appl. 27(4), 1019–1039 (2006) MathSciNetMATHCrossRefGoogle Scholar
  24. 24.
    Stüben, K.: Algebraic multigrid (AMG): an introduction with applications. In: Hackbusch, U., Oosterlee, C., Schueller, A. (eds.) Multigrid. Academic Press, San Diego (2000) Google Scholar
  25. 25.
    Yang, U.: Parallel algebraic multigrid methods – high performance preconditioners. In: Bruaset, A., Tveito, A. (eds.) Numerical Solution of Partial Differential Equations on Parallel Computers, vol. 51, pp. 209–236. Springer, Berlin (2006) CrossRefGoogle Scholar
  26. 26.
    Yang, U.M.: On long range interpolation operators for aggressive coarsening. Numer. Linear Algebra Appl. 17, 453–472 (2010). LLNL-JRNL-417371 MathSciNetMATHGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2012

Authors and Affiliations

  • Allison H. Baker
    • 1
  • Robert D. Falgout
    • 1
  • Tzanio V. Kolev
    • 1
  • Ulrike Meier Yang
    • 1
  1. 1.Lawrence Livermore National LaboratoryCenter for Applied Scientific ComputingLivermoreUSA

Personalised recommendations