Concurrent Parallel Processing on Graphics and Multicore Processors with OpenACC and OpenMP

  • Christopher P. Stone
  • Roger L. Davis
  • Daryl Y. Lee
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10732)


Hierarchical parallel computing is rapidly becoming ubiquitous in high performance computing (HPC) systems. Programming models used commonly in turbomachinery and other engineering simulation codes have traditionally relied upon distributed memory parallelism with MPI and have ignored thread and data parallelism. This paper presents methods for programming multi-block codes for concurrent computational on host multicore CPUs and many-core accelerators such as graphics processing units. Portable and standardized methods are language directives that are used to expose data and thread parallelism within the hybrid shared and distributed-memory simulation system. A single-source/multiple-object strategy is used to simplify code management and allow for heterogeneous computing. Automated load balancing is implemented to determine what portions of the domain are computed by the multi-core CPUs and GPUs. Preliminary results indicate that a moderate overall speed-up is possible by taking advantage of all processors and accelerators on a given HPC node.


High-performance computing Heterogeneous Accelerators 



This material is based upon work supported by, or in part by, the Department of Defense High Performance Computing Modernization Program (HPCMP) under User Productivity, Technology Transfer and Training (PETTT) contract number GS04T09DBC0017.

US Department of Defense (DoD) Distribution Statement A: Approved for public release. Distribution is unlimited.


  1. 1.
    Martin, C.: Multicore processors: challenges, opportunities, emerging trends. In: Proceedings of Embedded World Conference 2014, Nuremberg, Germany (2014)Google Scholar
  2. 2.
    OpenACC Specification Page. Accessed 31 July 2017
  3. 3.
    Stone, C., Davis, R.: High-performance 3D multi-disciplinary fluid/thermal prediction using combined multi-core/multi-GPGPU computer systems. In: 22nd AIAA Computational Fluid Dynamics Conference, Dallas, Texas, USA (2015).
  4. 4.
    OpenMP Specification Page. Accessed 31 July 2017
  5. 5.
    Pickering, B.P., Jackson, C.W., Scogland, T.R.W., Feng, W.-C., Roy, C.J.: Directive-based GPU programming for computational fluid dynamics. Comput. Fluids 114, 242–253 (2015)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Kraus, J., Schlottke, M., Adinetz, A., Pleiter, D.: Accelerating a C++ CFD code with OpenACC. In: 1st Workshop on Accelerator Programming Using Directives, pp. 47–54. IEEE (2014).
  7. 7.
    Wilcox, D.C.: Turbulence Modeling for CFD. DCW Industries, La Cannada (1998)Google Scholar
  8. 8.
    Smagorinsky, J.: General circulation experiments with the primitive equations. Mon. Weather Rev. 91, 99–164 (1963)CrossRefGoogle Scholar
  9. 9.
    Strelets, M.: Detached eddy simulation of massively separated flows. In: 39th Aerospace Sciences Meeting and Exhibit, Reno, Nevada (2001).
  10. 10.
    Bush, R.H., Mani, M.: A two-equation large eddy stress model for high sub-grid shear. In: 15th AIAA Computational Fluid Dynamics Conference, Anaheim, CA (2001).
  11. 11.
    Bozinoski, R., Davis, R.L.: General three-dimensional, multi-block, parallel turbulent Navier-Stokes procedure. In: AIAA Aerospace Sciences Meeting. Reno, Nevada (2008).
  12. 12.
    Ni, R.H.: A multiple grid scheme for solving the Euler equations. AIAA J. 20(11), 1565–1571 (1982). CrossRefMATHGoogle Scholar
  13. 13.
    Dannenhoffer, J.F.: Grid Adaptation for Complex Two-Dimensional Transonic Flows. Technical report CFDL-TR-87-10, Institute of Technology, Massachusetts (1987)Google Scholar
  14. 14.
    Davis, R.L., Ni, R.H., Carter, J.E.: Cascade viscous flow analysis using the Navier-Stokes equations. J. Propul. Power 3, 406–414 (1987). CrossRefGoogle Scholar
  15. 15.
    Jameson, A.: Time dependent calculations using multi-grid, with applications to unsteady flows past airfoils and wings. In: 10th AIAA Computational Fluid Dynamics Conference, Honolulu, HI (1991).
  16. 16.
    Davis, R.L., Clark, J.P.: Geometry-grid generation for three-dimensional multidisciplinary simulations in multistage turbomachinery. J. Propul. Power 30, 1502–1509 (2014). CrossRefGoogle Scholar
  17. 17.
    Huismann, I., Stiller, J., Frohlich, J.: Two-level parallelization of a fluid mechanics algorithm exploiting hardware heterogeneity. Comput. Fluids 117, 114–124 (2015). MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Christopher P. Stone
    • 1
  • Roger L. Davis
    • 2
  • Daryl Y. Lee
    • 2
  1. 1.Computational Science and Engineering, LLCAthensUSA
  2. 2.Department of Mechanical and Aerospace EngineeringUniversity of California DavisDavisUSA

Personalised recommendations