Parallel Numerical Methods for Differential Equations

  • Carlos A. de Moura
Part of the Applied Optimization book series (APOP, volume 67)


Some of the methods designed for the numerical solution of differential equations and which present an efficient implementation within a parallel environment are briefly surveyed. Included among these are: the Domain Decomposition and Multigrid hyper-algorithms, the Piecewise Parabolic method, the spectral (frequency) approach, some strategies for finite-element, and finite-difference higher accuracy techniques.


Parallel algorithms numerical methods differential equations 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    J. M. Ortega and R. G. Voigt. Solution of PDE on vector and parallel computers. SIAM Rev., 27(2):149–240, June 1985.MathSciNetzbMATHCrossRefGoogle Scholar
  2. [2]
    J. L. Gustavson et al. Development of parallel methods for a 1024- processor hypercube. SIAMJour. Stat. Scient. Comp., 9(4):594–638, 1988.Google Scholar
  3. [3]
    G. R. Joubert. Foreword. Parallel Computing, 23(1–2):vii, 1997.Google Scholar
  4. [4]
    J. R. Rice. Parallel methods for PDE’s. in The Characteristics of Paral-lel Algorithms, pages 209–231, 1987. Jamiesson, Gannon and Douglass (eds.).Google Scholar
  5. [5]
    D. E. Keyes. DD: a bridge between nature and parallel computers. ICASE Report, 27(92/44):149–240, Sep.’92.Google Scholar
  6. [6]
    C. A. de Moura. Parallel Algorithms for Differential Equations. LMGC, Univ. de Montpellier II, 1994. Rev. Edition: Parallel Numerical Methods for Differential Equations — a Survey. CIMPA Int. School on Adv. Algor. Tech. for Par. Comp. with Applic., Natal, Sep-Oct.1999, 38+ii pages. Rel. 08/99, IC-UFF, 11 Nov. 99, Scholar
  7. [7]
    C. A. de Moura. Massive parallelism as the new track for scientific com- puting. Proceedings Symposia Gaussiana, Conf. A, München, August 1993, pages 591–598, 1995. Behara, M. et al. (eds.).Google Scholar
  8. [8]
    H. A. Schwarz. Gesammelte Mathematische Abhandlungen, 2:133— 143, 1890. prey. published in Vierteljahrsschrift der Naturforschenden Gesellschaft, Zürich, v. 15, 272–286, 1870.Google Scholar
  9. [9]
    M. Konchady et al. Implementation and performance evaluation of a parallel ocean model. Parallel Computing, 24(2):181–203, 1998.zbMATHCrossRefGoogle Scholar
  10. [10]
    M. Marrocu et al. Parallelization and performance of a meteorological limited area model. Parallel Computing, 24(5–6):911–922, 1998.zbMATHCrossRefGoogle Scholar
  11. [11]
    D. Dabdub et al. Parallel computation in atmospheric chemical modelling. Parallel Computing, 22(1):111–130, 1996.zbMATHCrossRefGoogle Scholar
  12. [12]
    S. Lanteri et al. Parallel solution of compressible flows using overlapping and non-overlapping mesh partitioning strategies. Parallel Computing, 22(7):943–968, 1996.MathSciNetzbMATHCrossRefGoogle Scholar
  13. [13]
    P. LeTallec. DD in computational mechanics. Comp.Mech.Adv., 1:121–220, 1994.MathSciNetGoogle Scholar
  14. [14]
    P. LeTallec et al. Balancing DD for plates in DD methods in Sc. and Eng. Computing. Contemporary Math., 180:515–524, 1994.MathSciNetCrossRefGoogle Scholar
  15. [15]
    P. LeTallec et al. A Neumann-Neumann DD algorithm for solving plate and shell problems. SIAM J. Numerical Analysis, 35:836–867, 1998.MathSciNetCrossRefGoogle Scholar
  16. [16]
    S. C. Brenner et al. Balancing DD for non-conforming plate elements. Numerische Mathematik, 82(1):25–54, 1999.CrossRefGoogle Scholar
  17. [17]
    D. C. Hodgson et al. A DD preconditioner for a parallel finite element solver on distributed unstructured grids. Parallel Computing, 23(8):1157–1181, 1997.MathSciNetzbMATHCrossRefGoogle Scholar
  18. [18]
    T. F. Chan and D. C. Resasco. Analysis of DD preconditioners on irregular regions. Advances in Computer Methods for PDE, 1987. R. Vichnevetsky and R. Stepleman (eds.).Google Scholar
  19. [19]
    P. Fisette et al. Contribution to parallel and vector computing in multibody dynamics. Parallel Computing, 24(5–6):717–728, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  20. [20]
    P. E. Bjørstad and O. Widlund. To overlap or not to overlap: a note on a DD method for elliptic problems. SIAMJ. Sci. Stat. Comput.,10(5):1053–1051, Sept. 1989.CrossRefGoogle Scholar
  21. [21]
    M. Sultan, M. Hassan, J. Calvet, and A. Bilal. A first-order parallel decomposition method for parameter estimation in large scale systems. Large Sc. Syst., 9:51–62, 1985.MathSciNetzbMATHGoogle Scholar
  22. [22]
    H. J. Kushner. DD method for large Markov chain control problems and nonlinear elliptic like equation. SIAM Jour. Scient. Comp., 18(5):1494–516, 1997.MathSciNetzbMATHCrossRefGoogle Scholar
  23. [23]
    A. Baronio et al. A DD technique for spline image restoration on distributed memory systems. Parallel Computing, 22(1):1–18, 1996.MathSciNetCrossRefGoogle Scholar
  24. [24]
    E. P. Stephan et al. DD for indefinite hypersingular integral equation: The h and p version. SIAM Jour. Scient. Comp., 19(4):1139–1153, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  25. [25]
    C. A. de Moura, and M. Lentini. A parallel multi-shooting algorithm for TPBVP problems. H Simp. Bras. Arq. Computadores: Proc. Paralelo, 1988. also: 34°. S. Bras. Análise, IME-USP.Google Scholar
  26. [26]
    P. S. Rao et al. Data communication in parallel block pre-conditioned methods for solving ODE’s. Par. Comp., 23(13):1889–1907, 1997.CrossRefGoogle Scholar
  27. [27]
    A. Quarteroni. DD and Parallel processing for the numerical solution of PDE. Surv. Math. Ind., 1:75–118, 1991.MathSciNetzbMATHGoogle Scholar
  28. [28]
    T. P. Mathew et al. DD operator splitting for the solution of parabolic equations. SIAM Jour. Scient. Comp., 19(3):912–932, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  29. [29]
    F.-X. Roux. DD methods for static problems. Rech. Aérosp., 1:38–48, 1990.MathSciNetGoogle Scholar
  30. [30]
    P.-L. Lions. On the Schwarz alternating method. Proc. I Int. Symp. D. D. Meth. for PDE, pages 1–42, 1988. R. Glowinsky, G.H. Golub, G.A. Meurant and J. Périaux (eds.).Google Scholar
  31. [31]
    P.-L. Lions. On the Schwarz alternating method. DD Methods, pages 47–70, 1989. T. Chan, R. Glowinsky, J. Périaux and O. Widlund (eds.).Google Scholar
  32. [32]
    M. Djryja and O. Widlund. Towards a unified theory of DD algorithms for elliptic problems. Ultracomputer, Note #167, Dec.1989.Google Scholar
  33. [33]
    O. Widlund. Some Schwarz methods for symmetric and nonsymmetric elliptic problems. Tech. Report, 581, Sep. 1991.Google Scholar
  34. [34]
    M. Djryja and O. Widlund. DD algorithms with small overlap. Tech. Report, 606, May 1992.Google Scholar
  35. [35]
    C.-C. J. Kuo and T. F. Chan. Two-color Fourier analysis of iterative algorithms for elliptic problems with red/black ordering. SIAM J. Sc. Stat. Comput, 11(4):767–793, 1990.MathSciNetzbMATHCrossRefGoogle Scholar
  36. [36]
    T. F. Chan & D. Goovaerts. A note on the efficiency of incomplete factorizations. SIAM J. Sci. Stat. Comput., 11(4):794–803, Jul’90.MathSciNetzbMATHCrossRefGoogle Scholar
  37. [37]
    D. Amitai et al. Implicit-explicit parallel asynchronous solvers of parallel PDE’s. SIAM Jour. Scient. Comp., 19(4):1366–1404, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  38. [38]
    W. Hackbush. Multigrid Methods with Applications. Springer-Verlag, N.Y., 1985.CrossRefGoogle Scholar
  39. [39]
    W. Hackbush and U. Trottenberg (eds.). Multigrid Methods, Proc. Conf. at Köln-Porz, Nov. ’81. Springer-Verlag, Berlin, 1982. Lect. Notes Math. # 960.Google Scholar
  40. [40]
    C.-A. Thole and U. Trottenberg. A short note on standard Multigrid algorithms for 3-d problems. Appl. Math. Comp., 27:101–115, 1988.MathSciNetzbMATHCrossRefGoogle Scholar
  41. [41]
    K. Solchenback, C.-A. Thole, and U. Trottenberg. Parallel Multigrid methods: Implementation on Suprenum like architetures and applications. Rapport de Rech. 746, INRIA (Sophia-Antipolis), Nov. 1987.Google Scholar
  42. [42]
    M. Alef. Concepts for efficient Multigrid implementation on Suprenum like architectures. Paral. Comp., 17:1–16, 1991.zbMATHCrossRefGoogle Scholar
  43. [43]
    T. F. Chan and Y. Saad. Multigrid algorithms on the hypercube multiprocessor. IEEE Trans. on Comp, C-35(11):969–977, 1986.CrossRefGoogle Scholar
  44. [44]
    D. Gannon and J. van Rosendale. On the structure of parallelism in a highly concurrent PDE solver. J. Paral. Distr. Comp, 3:106–135, 1986.CrossRefGoogle Scholar
  45. [45]
    P. O. Fredericksen and O. A. McBryan. Parallel superconvergent multigrids. Lect. Notes Math. Appl. Series, # 110, 1988. Multigrid: Theory, Applications and Supercomputing, S. McCormick (ed.).Google Scholar
  46. [46]
    O. A. McBryan and et al.. Multigrid methods on parallel computers — a survey. Impact of Comp. in Sc. and Eng., 3:1–75, 1991.MathSciNetzbMATHCrossRefGoogle Scholar
  47. [47]
    O. A. McBryan. New architetures: performance highlights and new algorithms. Par. Comp., 7:477–499, 1988.zbMATHCrossRefGoogle Scholar
  48. [48]
    H. Ritzdorf and K. Stüben. Adaptive multigrids on distributed memory computers. Preprint, GMD, 1993. Postfach 1316, 53731 S. Augustin, Germany.Google Scholar
  49. [49]
    P. Bastian. Load balancing for adaptive multigrid method. SIAM Jour. Scient. Comp., 19(4):1303–1321, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  50. [50]
    D. Braess et al. A cascade multigrid algorithm for the Stokes equation. Numerische Mathematik, 82(2):179–192, 1999.MathSciNetzbMATHCrossRefGoogle Scholar
  51. [51]
    L. Ferm. The number of coarse-grid iterations every cycle for the 2-grid method. SIAM Jour. Scient. Comp., 19(2):493–501, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  52. [52]
    C. C. Douglas. Madpack: a family of abstract multigrid or multilevel solvers. Comp. & Appl. Math., 14(1):3–20, 1995. Sp. Issue on High Performance Scientific Computing, C.A. de Moura (ed.)zbMATHGoogle Scholar
  53. [53]
    K. Brand, M. Lemke, and J. Linden. Multigrid bibliography. Arbeitspapier der GMD, 206, 1986.Google Scholar
  54. [54]
    J. Linden and K. Stüben. Multigrid methods: an overview with emphasis on grid generation processes. Arbeitspapier der GMD, 207, 1986.Google Scholar
  55. [55]
    W. Briggs. A Multigrid tutorial. SIAM, Philadelphia, 1987.zbMATHGoogle Scholar
  56. [56]
    S. Kim. DD iterative procedure for solving scalar waves in the frequency domain. Numerische Mathematik, 79(2):231–260, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  57. [57]
    C. Carsteusen et al. Fast parallel solvers for symetric boundary element DD equation. Numerische Mathematik, 79(3):321–348, 1998.MathSciNetCrossRefGoogle Scholar
  58. [58]
    R. Hess et al. A computation of parallel MG and a FFT algorithm for the Helmholtz equation in numerical weather prediction. Par. Comp., 22(11):1503–1512, 1997.zbMATHCrossRefGoogle Scholar
  59. [59]
    B. Heise et al. Parallel solver for non linear elliptic problems based on DD ideas. Par. Comp., 22(11):1527–1544, 1997.zbMATHCrossRefGoogle Scholar
  60. [60]
    J. Y. Berthon et al. Which approach to parallelize S.C. codes, that is the question. Par. Comp., 23(1–2):165–179, 1997.CrossRefGoogle Scholar
  61. [61]
    H. Allik and S. Moore. Finite element analysis on the BBN Butterfly multiprocessor. Computer and Structures, 27(1):13–21, 1987.zbMATHCrossRefGoogle Scholar
  62. [62]
    C. Farhat and F.-X. Roux. A method of finite element tearing and interconnecting and its parallel solution algorithm. Int. Jour. Num. Meth. Engin., 32:1205–1227, 1991.zbMATHCrossRefGoogle Scholar
  63. [63]
    M. J. Daydé et al. Element by element preconditioners for large partially separable optimization problems. SIAM Jour. Scient. Comp., 18(6):1767–1787, 1997.CrossRefGoogle Scholar
  64. [64]
    M. Amendola. Cavitation in hydrodynamic lubrication: a parallel algorithm for a free-boundary problem (in Portuguese). PhD thesis, IMECCUnicamp, 1996.Google Scholar
  65. [65]
    B. Radi et al. Adaptive parallelization techniques in global weather models. Par. Comp., 24(8):1167–1176, 1998.CrossRefGoogle Scholar
  66. [66]
    I. T. Foster et al. Parallel algorithm for the spectral tranform method. SIAM Jour. Scient. Comp., 18(3):806–837, 1997.MathSciNetzbMATHCrossRefGoogle Scholar
  67. [67]
    A. Averbuch et al. Two dimensional parallel solver for the N-S equation using ADI on cells. Par. Comp., 24(8):673–699, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  68. [68]
    J. Douglas Jr., J. E. Santos, D. Sheen, and L. S. Bennethum. Frequency domain treatment of 1-d scalar waves model for semiconductors. Math. Models and Meth. in Appl. Sciences, 3(2):171–194, 1993.MathSciNetzbMATHCrossRefGoogle Scholar
  69. [69]
    P. Gauzellino and J. E. Santos. Numerical methods for wave propagation in elastic and anelastic media. Comput. Appl. Math, 12(2):95–111, 1993.MathSciNetzbMATHGoogle Scholar
  70. [70]
    C. A. Ravazzoli and J. E. Santos. Consistency analysis for a model for wave propagation in anelastic media. Latin Am. Appl. Research, 25:141–151, 1995. Tech. Report # 173, Purdue Univ., D. Math., 1993.Google Scholar
  71. [71]
    J. Douglas Jr., J. E. Santos, and D. Sheen. Approximation of scalar waves in the space-frequency domain. Math. Models and Meth. in Appl. Sciences, 4:509–531, 1994.MathSciNetzbMATHCrossRefGoogle Scholar
  72. [72]
    J. E. Santos and P. M. Gauzellino. Parallel algorithms for wave propagation in fluid—saturated porous media. Comput. Mech., New Trends and Applications, pages 1–12, 1998. S.R. Idelsohn et al. (eds.).Google Scholar
  73. [73]
    J. E. Santos and P. M. Gauzellino. Nonconforming iterative domain decomposition procedures for the simulation of waves in fluid—saturated porous solids. Mecánica Computacional, 18:657–664, 1997.Google Scholar
  74. [74]
    J. Douglas Jr., F. Pereira, and J. E. Santos. Parallel numerical simulation of waves in dispersive media. 1 st Conf. and Exp. Lat. Am. Geoph. Union, 4 th Int. Congr. Braz. Geoph. Society, 417–419, 1995.Google Scholar
  75. [75]
    J. Douglas Jr., F. Pereira, and J. E. Santos. A parallelizable approach to the simulation of waves in dispersive media. 3r d Int. Conf. Math. & Num. Asp. Wave Propagation, 673–682, 1995. G. Cohen (ed.).Google Scholar
  76. [76]
    J. E. Santos. Space-frequency domain approximation of waves in dispersive media. Meccánica Computacional, 14:106–113, 1994.Google Scholar
  77. [77]
    P. M. Gauzellino, and J. E. Santos. Parallel algorithms for the numerical simulation of waves. Proc. “Italian-Latinamerican Conference on Appl. and Ind. Math. (ITLA’97) ”, 219–222, 1997. Rome, Italy.Google Scholar
  78. [78]
    S. Petrova. Parallel implementation of fast elliptic solver. Par. Comp., 23(8):1113–1128, 1997.MathSciNetzbMATHCrossRefGoogle Scholar
  79. [79]
    D. J. Evans. Parallel algorithm design. Meeting on Parallel Computing, Verona, ’88, Ch. I:1–24, 1989.Google Scholar
  80. [80]
    C. A. de Moura. Parallel numerical schemes for evolutive differential equations. IV ICIAM, July 1999. Book of Abstracts, p.252.Google Scholar
  81. [81]
    C. A. de Moura. Non-sequential numerical algorithms for time-marching models. 50th SBA, USP, S. Paulo, 699–703, Nov. 1999. RT06/99, IC-UFF, 1999. Scholar
  82. [82]
    H. Stone. An efficient parallel algorithm for the solution of tridiagonal linear systems of equations. J. ACM, 20:27–38, 1973.zbMATHCrossRefGoogle Scholar
  83. [83]
    R. Babb II, L. Storc, and P. Elgroth. Parallelization schemes for 2-d hydrodynamic codes using the independent time-step method. Par. Comp., 8:85–89, 1988.zbMATHCrossRefGoogle Scholar
  84. [84]
    J. H. Saltz and V. K. Naik. Towards developing robust algorithms for solving PDE’s on MIMD machines. Par. Comp., 6:19–44, 1988.MathSciNetzbMATHCrossRefGoogle Scholar
  85. [85]
    D. E. Womble. A time-stepping algorithm for Parallel Computing. SIAM J. Sci. Stat. Comput., 11(5):824–837, 1990.MathSciNetzbMATHCrossRefGoogle Scholar
  86. [86]
    G. Cohen and P. Joly. Fourth order schemes for the heterogeneous acoustics equation. Comp. Meth. in Appl. Mech. and Engin., 80:397–407, 1990.MathSciNetzbMATHCrossRefGoogle Scholar
  87. [87]
    J. Tuomela. A note on high order schemes for the 1-d wave equation. BIT, 35:394–405, 1995.MathSciNetzbMATHCrossRefGoogle Scholar
  88. [88]
    J. Tuomela. On the construction of arbitrary order schemes for manydimensional wave equation. BIT, 36:158–165, 1996.MathSciNetzbMATHCrossRefGoogle Scholar
  89. [89]
    J. B. Cole. A nearly exact second-order finite-difference time-domain wave propagation algorithm on a coarse grid. Comp. in Phys., 8(6):730–734, 1994.CrossRefGoogle Scholar
  90. [90]
    J. B. Cole. High-accuracy solution of Maxwell’s equations using nonstandard finite differences. Comp. in Phys., 11(3): 287–292, 1997.CrossRefGoogle Scholar
  91. [91]
    L. Tang et al. Uniformly accurate finite difference schemes for prefinement. SIAM Jour. Scient. Comp., 20(3):1115–1131, 1999.zbMATHCrossRefGoogle Scholar
  92. [92]
    W. F. Spotz et al. Iterative and parallel performance of high-order compact systems. SIAM Jour. Scient. Comp., 19(1):1–14, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  93. [93]
    J. M. Rosinski et al. The accumulation of rounding error and post validation for global atmospheric models. SIAMJour. Scient. Comp., 18(2):552–564, 1997.MathSciNetzbMATHCrossRefGoogle Scholar
  94. [94]
    E. Carbajal Peńa. A high precision algorithm for linear thermoelasticity and its parallel implementation (in Portuguese). PhD thesis, IMIUFRJ, 1996.Google Scholar
  95. [95]
    I. Altas et al. Multigrid solution of automatically generated high-order discretization for the biharmonic equation. SIAM Jour. Scient. Comp., 19(5):1575–1585, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  96. [96]
    S. Vandewalle, R. V. Driesschie, and R. Piessens. The parallel performance of standard parabolic marching schemes. International Journal of Super Computing, 3(1):1–29, 1991.zbMATHGoogle Scholar
  97. [97]
    C. A. de Moura. DES — an explicit really quadratic 2-level scheme for the diffusion equation. J. Comp. Inf., Toronto, 3(1):99–115, 1994. Avail. as Preprint 42/92, LNCC.MathSciNetGoogle Scholar
  98. [98]
    K. Wilson. Grand challenges to computational sciences. Future Generation Computer Systems, 5:171–189, 1989.CrossRefGoogle Scholar
  99. [99]
    P. R. Woodward et al. Parallel computation of turbulent flow. Comp. Appl. Math., 14(1):97–105, 1995.zbMATHGoogle Scholar
  100. [100]
    P. R. Woodward, and W. Dai. A high-order Godunov-type scheme for shock interactions in ideal MHD. SIAM Jour. Scient. Comp., 18(4):957–981, 1997.MathSciNetzbMATHCrossRefGoogle Scholar
  101. [101]
    P. R. Woodward, and W. Dai. An iterative Riemann solver for relativistic hydrodynamics. SIAM Jour. Scient. Comp., 18(4):982–985, 1997.MathSciNetzbMATHCrossRefGoogle Scholar
  102. [102]
    P. R. Woodward, and W. Dai. A high order iterative implicit-explicit hybrid scheme for MIIHDD. SIAM Jour. Scient. Comp., 19(6):1827–1846, 1998.MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2002

Authors and Affiliations

  • Carlos A. de Moura
    • 1
    • 2
  1. 1.Brazil
  2. 2.Instituto de Computação — UFFNiteróiBrazil

Personalised recommendations