Development of a High Performance Code for Hydrodynamic Calculations Using Graphics Processor Units

  • Andrey V. SentyabovEmail author
  • Andrey A. Gavrilov
  • Maxim A. Krivov
  • Alexander A. Dekterev
  • Mikhail N. Pritula
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 753)


The paper presents the results of the implementation of computational algorithms of hydrodynamics for using graphics processor units. The implementation was carried out on the basis of the in-house CFD code SigmaFlow. Numerical simulations were based on the solution of the Navier-Stokes equations using SIMPLE-like procedure. The discretization of the differential equations was based on the control volume method on unstructured mesh. In the case of multiple CPU/GPU, parallel calculations were performed by means of domain decomposition. In the GPU-version of the code, basic computational functions were implemented as CUDA kernels to perform on GPUs. The code has been verified using several test cases. The computational efficiencies of several GPUs were compared with each other and that of modern CPUs. A modern GPU can increase the calculation performance of CFD problems by more than two times compared to a modern six-core CPU.


GPGPU CUDA MPI CFD Numerical simulation Control volume method SIMPLE Incompressible flow Domain decomposition 


  1. 1.
  2. 2.
    Li, R., Saad, Y.: GPU-accelerated preconditioned iterative linear solvers. J. Supercomputi. 63(2), 443–466 (2013). doi: 10.1007/s11227-012-0825-3 CrossRefGoogle Scholar
  3. 3.
    Rinaldi, P.R., Dari, E.A., Venere, M.J., Clausse, A.A.: Lattice-Boltzmann solver for 3D Fluid simulation on GPU. Simul. Model. Pract. Theor. 25, 163–171 (2012). doi: 10.1016/j.simpat.2012.03.004 CrossRefGoogle Scholar
  4. 4.
    Corrigan, A., Camelli, F.F., Lohner, R., Wallin, J.: Running unstructured grid-based CFD solvers on modern graphics hardware. Int. J. Numer. Methods Fluids 66(2), 221–229 (2011). doi: 10.1002/fld.2254 MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Waltz, J.: Performance of a three-dimensional unstructured mesh compressible flow solver on NVIDIA Fermi-class graphics processing unit hardware. Int. J. Numer. Methods Fluids 72(2), 259–268 (2013). doi: 10.1002/fld.3744 MathSciNetCrossRefGoogle Scholar
  6. 6.
    Gavrilov, A.A., Dekterev, A.A., Sentyabov, A.V.: Modeling of swirling flows with coherent structures using the unsteady reynolds stress transport model. Fluid Dyn. 50(4), 471–482 (2015). doi: 10.1134/S001546281504002X MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Pope, S.B.: Turbulent Flows. Cambridge University Press, New York (2000)CrossRefzbMATHGoogle Scholar
  8. 8.
    Nicoud, F., Ducros, F.: Subgrid-scale stress modelling based on the square of the velocity gradient tensor. Flow Turbul. Combust. 62(3), 183–200 (1999). doi: 10.1023/A: 1009995426001 CrossRefzbMATHGoogle Scholar
  9. 9.
    Mavriplis, J.: Revisiting the least-squares procedure for gradient reconstruction on unstructured meshes. AIAA-Paper 2003–3986, June 2003. doi: 10.2514/6.2003-3986
  10. 10.
    Ferziger, J.H., Peric, M.: Computational Methods for Fluid Dynamics. Springer, New York (2002). doi: 10.1007/978-3-642-56026-2 CrossRefzbMATHGoogle Scholar
  11. 11.
    Moukalled, F., Darwish, M.: A unified formulation of the segregated class of algorithms for fluid flow at all speeds. Numer. Heat Transf. Part B 37(2), 227–246 (2000). doi: 10.1080/104077900275576 CrossRefGoogle Scholar
  12. 12.
    Patankar, S.: Numerical Heat Transfer and Fluid Flow. Hemisphere Publishing Corporation, New York (1980)zbMATHGoogle Scholar
  13. 13.
    Rhie, C.M., Chow, W.L.: A numerical study of the turbulent flow past an isolated airfoil with trailing edge separation. AIAA J. 21, 1525–1532 (1983)CrossRefzbMATHGoogle Scholar
  14. 14.
    Leonard, B.P.: A stable and accurate convective modeling procedure based on quadratic upstream interpolation. Comput. Math. Appl. Mech. Eng. 19, 59–98 (1979). doi: 10.1016/0045-7825(79)90034-3 CrossRefzbMATHGoogle Scholar
  15. 15.
    Leschziner, M.A., Lien, F.S.: Upstream monotonic interpolation for scalar transport with application to complex turbulent flows. Int. J. Numer. Methods Fluids 19(6), 527–548 (1994). doi: 10.1002/fld.1650190606 CrossRefzbMATHGoogle Scholar
  16. 16.
    Barrett, R., Berry, M., Chan, T.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. SIAM, Philadelphia (1994). doi: 10.1137/1.9781611971538 CrossRefGoogle Scholar
  17. 17.
    Karypis, G., Kumar, V.: A fast and highly quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359–392 (1999). doi: 10.1137/S1064827595287997 CrossRefzbMATHGoogle Scholar
  18. 18.
    Michelsen, J.A.: Modeling incompressible rotating fluid flow. Ph.D. thesis, Technical University of Denmark (1986)Google Scholar
  19. 19.
    Escudier, M.P.: Observation of the flow produced in a cylindrical container by a rotating endwall. Exp. Fluids 2(4), 189–196 (1984). doi: 10.1007/BF00571864 CrossRefGoogle Scholar
  20. 20.
    Zdravkovich, M.M.: Flow Around Circular Cylinders: Fundamentals, vol. 1. Oxford University Press, New York (1997). doi: 10.1017/S0022112097227291 zbMATHGoogle Scholar
  21. 21.
    Rajani, B., Kandasamy, A., Majumdar, S.: Numerical simulation of laminar flow past a circular cylinder. Appl. Math. Model. 33, 1228–1247 (2009). doi: 10.1016/j.apm.2008.01.017 MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Shoeybi, M., Svard, M., Ham, F.E., Moin, P.: An adaptive implicit-explicit scheme for the DNS and LES of compressible flows on unstructured grids. J. Comput. Phys. 229, 5944–5965 (2010). doi: 10.1016/ MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Akula, B., Roy, P., Razi, P., Anderson, S., Girimaji, S.: Partially-Averaged Navier-Stokes (PANS) simulations of lid-driven cavity flow - Part 1: comparison with URANS and LES. In: Girimaji, S., Haase, W., Peng, S.H., Schwamborn, D. (eds.) Progress in Hybrid RANS-LES Modelling. Notes on Numerical Fluid Mechanics and Multidisciplinary Design, vol. 130, pp. 421–430. Springer, Cham (2015). doi: 10.1007/978-3-319-15141-0_29 CrossRefGoogle Scholar
  24. 24.
  25. 25.
    Minakov, A.V., Sentyabov, A.V., Platonov, D.V., Dekterev, A.A., Gavrilov, A.A.: Numerical modeling of flow in the Francis-99 turbine with Reynolds stress model and detached eddy simulation method. J. Phys. Conf. Ser. 579(1) (2015). doi: 10.1088/1742-6596/579/1/012004

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Siberian Federal UniversityKrasnoyarskRussia
  2. 2.Institute of Thermophysics SB RASNovosibirskRussia
  3. 3.Lomonosov Moscow State UniversityMoscowRussia
  4. 4.CTP PCP RASMoscowRussia

Personalised recommendations