Parallel Application Development with the Hybrid MPI+OpenMP Programming Model

Abstract

Many parallel platforms in use today are clusters of SMP systems, connected by Ethernet or one of the high-speed networks available. Some provide global memory addressing. Codes developed to run on these machines are often written using the MPI library for exchanging data. Increasingly, however, application developers have begun exploring the use of OpenMP in conjunction with MPI as a programming model. OpenMP is an industry standard for shared memory parallel program. Unlike MPI, it is based upon a set of directives that are inserted into a Fortran or C/C++ code and translated by a compiler into an explicitly parallel code. When used together with MPI, OpenMP is normally used to exploit the shared memory within each of the SMPs in the target platform.

In this tutorial, we review the OpenMP API for shared memory parallel programming, and then consider how OpenMP directives may be inserted into an existing MPI program. In order to understand how to do this well, we also look at performance issues related to the use of OpenMP. We complete the tutorial by discussing the potential benefits of this model and reporting on experiences already gained in a variety of efforts that have developed codes under this hybrid model.