Efficient Inter-process Communication in Parallel Implementation of Grid-Characteristic Method

  • Andrey M. IvanovEmail author
  • Nikolay I. Khokhlov
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 133)


We consider application of Message Passing Interface (MPI) to parallelize a numerical code that solves the seismic wave equation with the grid-characteristic method. The problem is time reducing of communication between MPI processes during computation. The solution to this problem is of great interest, when multiple computational grids are involved. The difficulty of problem is increased with independent distribution of nodes across processes. General approach to handle contacts is formulated in terms of interpolation of nodes from some part of one grid to some part of another. In this chapter, we propose an efficient algorithm and give an example of its implementation. The implementation is not restricted to Cartesian grids, so it is possible to make accurate simulations taking into account earth surface topography and complex geometry of contacts between geological layers.


Parallel programming Grid-characteristic method MPI Structured grids 



This work has been carried out using computing resources of the federal collective usage center Complex for Simulation and Data Processing for Mega-science Facilities at NRC “Kurchatov Institute”,

The reported study was funded by RFBR according to the research project No. 18-07-00914 A.


  1. 1.
    Moczo, P., Kristek, J., Galis, M., Pazak, P., Balazovjech, M.: The finite-difference and finite-element modeling of seismic wave propagation and earthquake motion. Acta. Phys. Slovaca. Rev. Tutorials 57(2), 177–406 (2010)Google Scholar
  2. 2.
    Virieux, J.: P-SV wave propagation in heterogeneous media: velocity-stress finite-difference method. Geophysics 51(4), 889–901 (1986)CrossRefGoogle Scholar
  3. 3.
    Komatitsch, D., Vilotte, J.-P., Vai, R., Castillo-Covarrubias, J.M., Sánchez-Sesma, F.J.: The spectral element method for elastic wave equations—application to 2-D and 3-D seismic problems. Int. J. Numer. Methods in Eng. 49(9), 1149–1164 (1999)zbMATHGoogle Scholar
  4. 4.
    Martin, K., Dumbser, M.: An arbitrary high-order Discontinuous Galerkin method for elastic waves on unstructured meshes—I. The two-dimensional isotropic case with external source terms. Geophys. J. Int. 166(2), 855–877Google Scholar
  5. 5.
    Favorskaya, A.V., Petrov, I.B.: Grid-characteristic method. In: Favorskaya A.V., Petrov I.B. (eds.) Innovations in Wave Processes Modelling and Decision Making. SIST, vol. 90, pp. 117–160. Springer, Cham (2018)CrossRefGoogle Scholar
  6. 6.
    Liu, W., Wang, F., Zhou, H.J.: Parallel seismic modeling based on OpenMP+AVX and Optimization Strategy. Earth Sci. 29, 1–6 (2018)CrossRefGoogle Scholar
  7. 7.
    Micikevicius, P.: 3D finite difference computation on GPUs using CUDA. In: Proceedings of 2nd Workshop on General Purpose Processing on Graphics Processing Units. ACM, pp. 79–84 (2009)Google Scholar
  8. 8.
    Martin, R., Komatitsch, D., Blitz, C., Le Goff, N.: Simulation of seismic wave propagation in an asteroid based upon an unstructured MPI spectral-element method: blocking and non-blocking communication strategies. In: International Conference on High Performance Computing for Computational Science (VECPAR 2008), pp. 350–363. Springer, Berlin, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    Voinov, O.Y., Golubev V.I., Petrov I.B.: Elastic imaging using multiprocessor computer systems. In: CEUR Workshop Proceedings, vol. 1787, pp. 491–495 (2016)Google Scholar
  10. 10.
    Mu, D., Chen, P., Wang, L.: Accelerating the discontinuous Galerkin method for seismic wave propagation simulations using multiple GPUs with CUDA and MPI. Earthquake Sci. 26(6), 377–493 (2013)CrossRefGoogle Scholar
  11. 11.
    Komatitsch, D.: Fluid-solid coupling on a cluster of GPU graphics cards for seismic wave propagation. Academie des Sciences. Comptes Rendus. Mecanique 349(2–3), 125–135 (2011)CrossRefGoogle Scholar
  12. 12.
    Khokhlov, N., Ivanov, A., Zhdanov, M., Petrov, I., Ryabinkin, E.: Applying OpenCL Technology for Modelling Seismic Processes Using Grid-characteristic Methods. In: International Conference on Distributed Computer and Communication Networks (DCCN 2016), pp. 577–588. Springer, Cham (2016)Google Scholar
  13. 13.
    Khokhlov, N., Yavich, N., Malovichko, M., Petrov, I.: Solution of large-scale seismic modeling problems. Procedia Comput. Sci. 66, 191–199 (2015)CrossRefGoogle Scholar
  14. 14.
    Gabriel, E., Fagg, G.E., Bosilca, G., Angskun, T., Dongarra, J.J., Squyres, J.M., Sahay, V., Kambadur, P., Barrett, B., Lumsdaine, A., Castain, R.H., Daniel, D.J., Graham, R.L., Woodall, T.S.: Open MPI: goals, concept, and design of a next generation MPI implementation. In: European Parallel Virtual Machine/Message Passing Interface Users Group Meeting (EuroPVM/MPI 2004), pp. 97–104. Springer, Berlin, Heidelberg (2004)CrossRefGoogle Scholar
  15. 15.
    LeVeque, R.J.: Finite Volume Methods for Hyperbolic Problems, vol. 31. Cambridge University PressGoogle Scholar
  16. 16.
    Nakamura, T., Tanaka, R., Yabec, T., Takizawa, K.: Exactly conservative semi-Lagrangian scheme for multi-dimensional hyperbolic equations with directional splitting technique. J. Comput. Phys. 174(1), 171–207 (2001)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Golubev, V.I., Petrov, I.B., Khokhlov, N.I.: Compact grid-characteristic schemes of higher orders of accuracy for a 3D linear transport equation. Math. Models Comput. Simul. 8(5), 577–584 (2016)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Golubev, V.I., Petrov, I.B.: Improving the simulation accuracy of seismic responses from curvilinear geological boundaries. In: Geomodel 2016: 18th Science and Applied Research Conference on Oil and Gas Geological Exploration and Development. (In Russian)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Moscow Institute of Physics and Technology (MIPT)DolgoprudnyRussian Federation

Personalised recommendations