Skip to main content
SpringerLink
Log in

Physics-Based Balancing Domain Decomposition by Constraints for Multi-Material Problems

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

In this work, we present a new variant of the balancing domain decomposition by constraints preconditioner that is robust for multi-material problems. We start with a well-balanced subdomain partition, and based on an aggregation of elements according to their physical coefficients, we end up with a finer physics-based (PB) subdomain partition. Next, we define corners, edges, and faces for this PB partition, and select some of them to enforce subdomain continuity (primal faces/edges/corners). When the physical coefficient in each PB subdomain is constant and the set of selected primal faces/edges/corners satisfy a mild condition on the existence of acceptable paths, we can show both theoretically and numerically that the condition number does not depend on the contrast of the coefficient across subdomains. An extensive set of numerical experiments for 2D and 3D for the Poisson and linear elasticity problems is provided to support our findings. In particular, we show robustness and weak scalability of the new preconditioner variant up to 8232 cores when applied to 3D multi-material problems with the contrast of the physical coefficient up to \(10^8\) and more than half a billion degrees of freedom. For the scalability analysis, we have exploited a highly scalable advanced inter-level overlapped implementation of the preconditioner that deals very efficiently with the coarse problem computation. The proposed preconditioner is compared against a state-of-the-art implementation of an adaptive BDDC method in PETSc for thermal and mechanical multi-material problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. This geometrical definition is not common in the literature, but it has been proved to be the best choice in our practical experience, and what is implemented in FEMPAR. It allows us to define the set of globs at the geometrical step, independently of the FE space to be used, which is particularly useful for multi-physics problems.

  2. We note that the definition of constraint sets in 3D linear elasticity for FETI-DP methods in the seminal work [36] rely on edge constraints.

  3. This choice is motivated by (5) in the definition of the acceptable path.

  4. Clearly, by restricting the connections to PB subdomains in \(\Theta _\mathrm{pb}(\mathcal {D}_1) \cup \Theta _\mathrm{pb}(\mathcal {D}_2)\), we restrict the length of the acceptable paths, not attaining the minimum coarse space. However, it is essential to expose concurrency in the implementation of the algorithm by controlling the length of the acceptable paths.

  5. Indeed, since 2014, the multilevel DD solvers within FEMPAR have been in the High-Q club of the most scalable European codes, maintained by the Jülich supercomputing center [14].

  6. For reproducibility purposes, the exact command-line argument values that we used to invoke the PETSc PCBDDC driver program were [59]: -pc_bddc_use_vertices true -pc_bddc_use_edges false -pc_bddc_use_faces false -pc_bddc_adaptive_threshold {2,5,10} -pc_bddc_use_deluxe_scaling true -pc_bddc_coarse_redistribute {2,4,8,12,24} -pc_bddc_coarse_pc_type lu -pc_bddc_coarse_pc_factor_mat_solver_package mumps.

  7. A memory profile of PCBDDC(c) revealed that there are two main sources of extra memory consumption over PB-BDDC. First, the PETSc implementation of PCBDDC is such that it does not exploit the symmetry of the problem at hand, i.e., both the lower and upper triangles of matrices and their respective LU factors are stored. Second, adaptive selection of constraints and deluxe scaling need to explicitly compute the Schur complement matrix related to subdomain interface DOFs, and to set up auxiliary eigenvalue problems. The second source was confirmed as PCBDDC(c) for \(H/h=40\) without deluxe scaling and adaptive selection of constraints did fit into memory.

References

  1. Badia, S., Martín, A.F., Principe, J.: Enhanced balancing Neumann–Neumann preconditioning in computational fluid and solid mechanics. Int. J. Numer. Methods Eng. 96, 203–230 (2013)

    MathSciNet  MATH  Google Scholar 

  2. Badia, S., Martín, A.F., Principe, J.: Implementation and scalability analysis of balancing domain decomposition methods. Arch. Comput. Methods Eng. 20, 239–262 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  3. Badia, S., Martín, A.F., Principe, J.: A highly scalable parallel implementation of balancing domain decomposition by constraints. SIAM J. Sci. Comput. 36, C190–C218 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  4. Badia, S., Martín, A.F., Principe, J.: On the scalability of inexact balancing domain decomposition by constraints with overlapped coarse/fine corrections. Parallel Comput. 50, 1–24 (2015)

    Article  MathSciNet  Google Scholar 

  5. Badia, S., Martín, A.F., Principe, J.: Multilevel balancing domain decomposition at extreme scales. SIAM J. Sci. Comput. 38, C22–C52 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  6. Badia, S., Martín, A. F., Principe, J.: FEMPAR Web page. http://www.fempar.org, 2017

  7. Badia, S., Martín, A.F., Nguyen, H.: Balancing domain decomposition by constraints associated with subobjects. Appl. Math. Lett. 87, 93–100 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  8. Badia, S., Martín, A.F., Principe, J.: FEMPAR: an object-oriented parallel finite element framework. Arch. Comput. Methods Eng. 25, 195–271 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  9. Badia, S., Nguyen, H.: Balancing domain decomposition by constraints and perturbation. SIAM J. Numer. Anal. 54, 3436–3464 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  10. Badia, S., Nguyen, H.: Relaxing the roles of corners in BDDC by perturbed formulation. In: Domain Decomposition Methods in Science and Engineering XXIII. Springer, pp. 397–405 (2017)

  11. Balay, S., Abhyankar, S., Adams, M. F., Brown, J., Brune, P., Buschelman, K., Dalcin, L., Eijkhout, V., Gropp, W.D., Kaushik, D., Knepley, M.G., May, D.A., McInnes, L.C., Rupp, K., Sanan, P., Smith, B.F., Zampini, S., Zhang, H., Zhang, H.: PETSc users manual, Tech. Report ANL-95/11 - Revision 3.8, Argonne National Laboratory (2017)

  12. Brenner, S.C., Scott, L.R.: The Mathematical Theory of Finite Element Methods. Texts in Applied Mathematics, vol. 15, 3rd edn. Springer, New York (2008)

    Book  Google Scholar 

  13. Brenner, S.C., Sung, L.-Y.: BDDC and FETI-DP without matrices or vectors. Comput. Methods Appl. Mech. Eng. 196, 1429–1435 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  14. Brömmel, D., Wylie, B.J.N., Frings, W.: JUQUEEN Extreme Scaling Workshop 2015, Tech. Report FZJ-2015-01645, Jülich Supercomputing Center (2015)

  15. Chan, T.F., Mathew, T.P.: Domain decomposition algorithms. Acta Numer. 3, 61–143 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  16. Dohrmann, C.R.: A preconditioner for substructuring based on constrained energy minimization. SIAM J. Sci. Comput. 25, 246–258 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  17. Dohrmann, C.R.: An approximate BDDC preconditioner. Numer. Linear Algebra Appl. 14, 149–168 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  18. Dolean, V., Nataf, F., Scheichl, R., Spillane, N.: Analysis of a two-level Schwarz method with coarse spaces based on local Dirichlet-to-Neumann maps. Comput. Methods Appl. Math. 12, 391–414 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  19. Dryja, M., Galvis, J., Sarkis, M.: BDDC methods for discontinuous Galerkin discretization of elliptic problems. J. Complex. 23, 715–739 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  20. Dryja, M., Sarkis, M.: Technical tools for boundary layers and applications to heterogeneous coefficients. In: Huang, Y., Kornhuber, R., Widlund, O., Xu, J. (eds.) Domain Decomposition Methods in Science and Engineering XIX, pp. 205–212. Springer Berlin Heidelberg, Berlin (2011)

    Chapter  Google Scholar 

  21. Dryja, M., Sarkis, M.V., Widlund, O.B.: Multilevel Schwarz methods for elliptic problems with discontinuous coefficients in three dimensions. Numer. Math. 72, 313–348 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  22. Dryja, M., Smith, B.F., Widlund, O.B.: Schwarz analysis of iterative substructuring algorithms for elliptic problems in three dimensions. SIAM J. Numer. Anal. 31, 1662–1694 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  23. Farhat, C., Lesoinne, M., Le Tallec, P., Pierson, K., Rixen, D.: FETI-DP: a dual–primal unified FETI method-part I: a faster alternative to the two-level FETI method. Int. J. Numer. Methods Eng. 50, 1523–1544 (2001)

    Article  MATH  Google Scholar 

  24. Farhat, C., Lesoinne, M., Pierson, K.: A scalable dual–primal domain decomposition method. Numer. Linear Algebra Appl. 7, 687–714 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  25. Galvis, J., Efendiev, Y.: Domain decomposition preconditioners for multiscale flows in high-contrast media. Multiscale Model. Simul. 8, 1461–1483 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  26. Galvis, J., Efendiev, Y.: Domain decomposition preconditioners for multiscale flows in high contrast media: reduced dimension coarse spaces. Multiscale Model. Simul. 8, 1621–1644 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  27. Gander, M.J., Loneland, A., Rahman, T.: Analysis of a new harmonically enriched multiscale coarse space for domain decomposition methods (2015). arXiv:1512.05285

  28. Gippert, S., Klawonn, A., Rheinbach, O.: Analysis of FETI-DP and BDDC for linear elasticity in 3D with almost incompressible components and varying coefficients inside subdomains. SIAM J. Numer. Anal. 50, 2208–2236 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  29. Graham, I.G., Lechner, P.O., Scheichl, R.: Domain decomposition for multiscale PDEs. Numer. Math. 106, 589–626 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  30. Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput 20, 359–392 (1998). (electronic)

    Article  MathSciNet  MATH  Google Scholar 

  31. Kim, H.H., Chung, E.T.: A BDDC algorithm with enriched coarse spaces for two-dimensional elliptic problems with oscillatory and high contrast coefficients. Multiscale Model. Simul. 13, 571–593 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  32. Klawonn, A., Radtke, P., Rheinbach, O.: FETI-DP methods with an adaptive coarse space. SIAM J. Numer. Anal. 53, 297–320 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  33. Klawonn, A., Radtke, P., Rheinbach, O.: Adaptive coarse spaces for BDDC with a transformation of basis. In: Dickopf, T., Gander, J.M., Halpern, L., Krause, R., Pavarino, F.L. (eds.) Domain Decomposition Methods in Science and Engineering XXII, pp. 301–309. Springer, Berlin (2016)

    Chapter  Google Scholar 

  34. Klawonn, A., Radtke, P., Rheinbach, O.: A comparison of adaptive coarse spaces for iterative substructuring in two dimensions. Electron. Trans. Numer. Anal. 45, 75–106 (2016)

    MathSciNet  MATH  Google Scholar 

  35. Klawonn, A., Rheinbach, O.: Robust FETI-DP methods for heterogeneous three dimensional elasticity problems. Comput. Methods Appl. Mech. Eng. 196, 1400–1414 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  36. Klawonn, A., Widlund, O.B.: Dual–primal FETI methods for linear elasticity. Commun. Pure Appl. Math. 59, 1523–1572 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  37. Klawonn, A., Widlund, O.B., Dryja, M.: Dual–primal FETI methods for three-dimensional elliptic problems with heterogeneous coefficients. SIAM J. Numer. Anal. 40, 159–179 (2002). (electronic)

    Article  MathSciNet  MATH  Google Scholar 

  38. Li, J., Widlund, O.B.: FETI-DP, BDDC, and block Cholesky methods. Int. J. Numer. Methods Eng. 66, 250–271 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  39. Li, J., Widlund, O.B.: On the use of inexact subdomain solvers for BDDC algorithms. Comput. Methods Appl. Mech. Eng. 196, 1415–1428 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  40. Loisel, S., Nguyen, H., Scheichl, R.: Optimized Schwarz and 2-Lagrange multiplier methods for multiscale elliptic PDEs. SIAM J. Sci. Comput. 37, A2896–A2923 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  41. Mandel, J.: Balancing domain decomposition. Commun. Numer. Methods Eng. 9, 233–241 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  42. Mandel, J., Dohrmann, C.R.: Convergence of a balancing domain decomposition by constraints and energy minimization. Numer. Linear Algebra Appl. 10, 639–659 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  43. Mandel, J., Dohrmann, C.R., Tezaur, R.: An algebraic theory for primal and dual substructuring methods by constraints. Appl. Numer. Math. 54, 167–193 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  44. Mandel, J., Sousedík, B., Dohrmann, C.: Multispace and multilevel BDDC. Computing 83, 55–85 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  45. Mathew, T.P.A.: Domain Decomposition Methods for the Numerical Solution of Partial Differential Equations, vol. 61. Springer, Berlin (2008)

    MATH  Google Scholar 

  46. Nataf, F., Xiang, H., Dolean, V., Spillane, N.: A coarse space construction based on local Dirichlet-to-Neumann maps. SIAM J. Sci. Comput. 33, 1623–1642 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  47. Pechstein, C.: On iterative substructuring methods for multiscale problems. In: Erhel, J., Gander, M.J., Halpern, L., Pichot, G., Sassi, T., Widlund, O. (eds.) Domain Decomposition Methods in Science and Engineering XXI, pp. 85–98. Springer, Berlin (2014)

    Google Scholar 

  48. Pechstein, C., Dohrmann, C.R.: A unified framework for adaptive BDDC. Electron. Trans. Numer. Anal. 46, 273–336 (2017)

    MathSciNet  MATH  Google Scholar 

  49. Pechstein, C., Scheichl, R.: Analysis of FETI methods for multiscale PDEs. Numer. Math. 111, 293–333 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  50. Pechstein, C., Scheichl, R.: Analysis of FETI methods for multiscale PDEs. Part II: Interface variation. Numer. Math. 118, 485–529 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  51. Scheichl, R., Vainikko, E.: Additive Schwarz with aggregation-based coarsening for elliptic problems with highly variable coefficients. Computing 80, 319–343 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  52. Šístek, J., Čertíková, M., Burda, P., Novotný, J.: Face-based selection of corners in 3D substructuring. Math. Comput. Simul. 82, 1799–1811 (2012)

    Article  MathSciNet  Google Scholar 

  53. Spillane, N., Dolean, V., Hauret, P., Nataf, F., Pechstein, C., Scheichl, R.: A robust two-level domain decomposition preconditioner for systems of PDEs. C. R. Math. Acad. Sci. Paris 349, 1255–1259 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  54. Spillane, N., Dolean, V., Hauret, P., Nataf, F., Pechstein, C., Scheichl, R.: Abstract robust coarse spaces for systems of PDEs via generalized eigenproblems in the overlaps. Numer. Math. 126, 741–770 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  55. Spillane, N., Dolean, V., Hauret, P., Nataf, F., Rixen, D.J.: Solving generalized eigenvalue problems on the interfaces to build a robust two-level FETI method. C. R. Math. Acad. Sci. Paris 351, 197–201 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  56. Toselli, A., Widlund, O.: Domain Decomposition Methods—Algorithms and Theory. Springer Series in Computational Mathematics, vol. 34. Springer, Berlin (2005)

    Book  MATH  Google Scholar 

  57. Tu, X.: Three-level BDDC in three dimensions. SIAM J. Sci. Comput. 29, 1759–1780 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  58. Widlund, O.B., Dohrmann, C.R.: BDDC deluxe domain decomposition. In: Dickopf, T., Gander, J.M., Halpern, L., Krause, R., Pavarino, F.L. (eds.) Domain Decomposition Methods in Science and Engineering XXII, pp. 93–103. Springer, Cham (2016)

    Chapter  Google Scholar 

  59. Zampini, S.: PCBDDC: a class of robust dual–primal methods in PETSc. SIAM J. Sci. Comput. 38, S282–S306 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  60. Zhu, Y.: Domain decomposition preconditioners for elliptic equations with jump coefficients. Numer. Linear Algebra Appl. 15, 271–289 (2008)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universitat Politècnica de Catalunya, Jordi Girona 1-3, Edifici C1, 08034, Barcelona, Spain

    Santiago Badia & Alberto F. Martín

  2. CIMNE - Centre Internacional de Mètodes Numèrics en Enginyeria, UPC, Esteve Terradas 5, 08860, Castelldefels, Spain

    Santiago Badia & Alberto F. Martín

  3. Institute of Research and Development, Duy Tan University, 3 Quang Trung, Da Nang, Vietnam

    Hieu Nguyen

Authors
  1. Santiago Badia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alberto F. Martín
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hieu Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Santiago Badia.

Additional information

This work has partially been funded by the European Research Council under the FP7 Program Ideas through the Starting Grant No. 258443—COMFUS: Computational Methods for Fusion Technology and the FP7 NUMEXAS project under Grant Agreement 611636. Financial support from the EC—International Cooperation in Aeronautics with China (Horizon 2020) under the project: Efficient Manufacturing for Aerospace Components USing Additive Manufacturing, Net Shape HIP and Investment Casting (EMUSIC) and the H2020-FoF-2015 under the project: Computer Aided Technologies for Additive Manufacturing (CAxMan) are also acknowledged. S. Badia gratefully acknowledges the support received from the Catalan Government through the ICREA Acadèmia Research Program. H. Nguyen thanks Vietnam National Foundation for Science and Technology Development (NAFOSTED) for the financial support through Grant No. 101.99-2017.13. Finally, the authors thankfully acknowledge the computer resources at Marenostrum III and IV and the technical support provided by BSC under the RES (Spanish Supercomputing Network), and the North-German Supercomputing Alliance (HLRN) for providing HPC resources that have contributed to the research results reported in this paper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Badia, S., Martín, A.F. & Nguyen, H. Physics-Based Balancing Domain Decomposition by Constraints for Multi-Material Problems. J Sci Comput 79, 718–747 (2019). https://doi.org/10.1007/s10915-018-0870-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-018-0870-z

Keywords

Mathematics Subject Classification

Search

Navigation