Evaluating Performance Portability of Accelerator Programming Models using SPEC ACCEL 1.2 Benchmarks
As heterogeneous architectures are becoming mainstream for HPC systems, application programmers are looking for programming model implementations that offer both performance and portability across platforms. Two directive-based programming models for accelerator programming that aim at doing this are OpenMP 4/4.5 and OpenACC. Many users want to know the difference between these two programming models, the state of their implementations, how to use them, and evaluate how suitable they are for their applications.
The Standard Performance Evaluation Corporation (SPEC) ACCEL benchmarks, developed by the SPEC High Performance Group (HPG), recently released SPEC ACCEL 1.2 benchmark suite to help the evaluation of OpenCL, OpenMP 4.5 and OpenACC on different platforms. In this paper we present our preliminary results that evaluates OpenMP 4.5 and OpenACC on a variety of accelerator-based systems: POWER9 with NVIDIA V100 GPUs (Summit), Intel Xeon Phi 7230 (Percival), and AMD Bulldozer Opteron with NVIDIA K20x (Titan). Comparing these benchmarks on different systems gives us insight into the support for OpenMP and OpenACC and their execution times provide insights about their quality of implementations provided by different vendors. We also compare best of OpenMP and OpenACC to see if a particular programming model favors a particular type of benchmark kernel.
This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, under contract number DE-AC05-00OR22725. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.
- 1.EPCC OpenACC Microbenchmarks. https://www.epcc.ed.ac.uk/research/computing/performance-characterisation-and-benchmarking/epcc-openacc-benchmark-suite
- 3.Percival quickstart guide. https://www.olcf.ornl.gov/percival-quickstart-guide/
- 4.SPEC ACCEL: Run and Reporting Rules. https://www.spec.org/accel/docs/runrules.html
- 5.Standard Performance Evaluation Corporation. https://www.spec.org/
- 6.Summit: Scale new heights. discover new solutions. https://www.olcf.ornl.gov/summit/
- 7.Friedline, K., Chandrasekaran, S., Lopez, M.G., Hernandez, O.: OpenACC 2.5 validation testsuite targeting multiple architectures. In: Kunkel, J.M., Yokota, R., Taufer, M., Shalf, J. (eds.) ISC High Performance 2017. LNCS, vol. 10524, pp. 557–575. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67630-2_39CrossRefGoogle Scholar
- 8.Oak Ridge National Lab: Titan supercomputer. https://www.olcf.ornl.gov/titan/
- 9.OpenACC: OpenACC, Directives for Accelerators. http://www.openacc.org/
- 10.OpenMP: Openmp 4.0 specification. http://www.openmp.org/wp-content/uploads/openmp-4.0.pdf
- 11.OpenMP: Openmp 4.5 specification. http://www.openmp.org/wp-content/uploads/openmp-4.5.pdf
- 12.OpenMP Validation and Verification Suite: Openmp 3.1 Specification. https://github.com/sunitachandra/omp-validation
- 13.Pophale, S., Diaz, J.M., Hernandez, O., Bernholdt, D., Chandrasekaran, S.: OpenMP 4.5 Validation and Verification Suite for Device Offload. https://crpl.cis.udel.edu/ompvvsollve/
- 14.Wang, C., Xu, R., Chandrasekaran, S., Chapman, B., Hernandez, O.: A validation testsuite for OpenACC 1.0. In: 2014 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 1407–1416. IEEE (2014)Google Scholar