Application Suitability Assessment for Many-Core Targets

Abstract

Many-core hardware platforms offer a tremendous opportunity for scaling up performance, but not all codes that run on these platforms have been modernized sufficiently to fully utilize the hardware. Assessing whether a code will effectively utilize a given platform can be challenging, particularly for new or potential future platforms where native execution on real hardware is not possible. In this case, one typically relies on architecture simulators and other workload characterization tools, which are often not user-friendly for developers who want to do a quick initial assessment of an application’s suitability for a many-core architecture.

To help address this challenge, we present QMSprof, a tool and a set of analyses for an initial assessment of the suitability of a set of applications for a simulated extremely-parallel many-core target. QMSprof automates the process of running a suite of workload binaries through Intel® Software Development Emulator (SDE) and the Sniper multi-core simulator and extracting high-level summary statistics. The tool generates comparative plots summarizing key metrics across the workload suite, including the mix of vector and nonvector instructions, scalability with increasing thread count, memory bandwidth utilization, and statistics on cache misses and working set size. These summary metrics are designed to aid performance tuners in selecting promising codes for a many-core target and in pinpointing opportunities for additional tuning. To illustrate the utility of our tool, we also describe some sample results from characterizing applications on a hypothetical many-core architecture.

This research was, in part, funded by the U.S. Government and DOE. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government.

A Appendix: Interface for QMSprof

This appendix demonstrates the interface for QMSprof. We first present an example of configuring the collector to run simulations, and then describe how to run the analyzer to extract summary statistics and generate plots.

1.1 A.1 Collector Interface

The collector interface for QMSprof is divided into four major parts, namely configuration for (1) workload binaries and run arguments, (2) Sniper models, (3) environment setup, and (4) experiment script.

For part (1), binaries and run arguments are configured by specifying a Benchmarks dictionary, which maps a key for each benchmark to a per-benchmark dictionary with additional information. When running an experiment, the collector uses information from a per-benchmark dictionary to stage each simulation run, i.e., creating a separate run folder for each simulation run in a staging area, and copying and/or renaming any necessary binaries and input files into that folder. This staging step is needed because workloads are not always built to support concurrent executions from the same run folder.

The per-benchmark dictionary is built to support benchmark variants, i.e., different versions of the same workload, with possibly different binaries or run arguments. This dictionary has several expected fields:

• 1. bindir: This string lists the subdirectory of the root benchmark directory containing the files for this benchmark. The root benchmark directory is a global variable specified separately in the top-level of the configuration file.

• 2. files: This dictionary maps a variant of the benchmark to a list of files needed to run each variant. Each file is itself a pair, with the first value being the name of the file in the source binary directory, and the second value being the name of the target file in the staging area. This pair allows an input file to be renamed in the staging area before it is run.

• 3. runargs: This dictionary maps a variant of a benchmark to the command-line arguments needed to run the variant.

• 4. gen_inputs: This dictionary maps a variant to a list of shell commands to execute in bindir to generate any input files that are needed for a run.

• 5. requires_MPI: This flag is set if this particular binary requires the use of an MPI library to execute. Our current prototype assumes one MPI rank per program, but in principle this assumption could be relaxed.

For the files, runargs, and gen_inputs dictionaries, if no exact match to a particular variant name is found in the dictionary, QMSprof will map to the key that matches the longest prefix.

As an example, Fig. 7 shows part of a configuration file specifying binaries and arguments for two benchmarks: LULESH and SNAP. In this configuration, running the sim_vec variant of the LULESH, uses the source file lulesh2.0_vec from the subdirectory LULESH/binaries. The staging process will copy and rename (or link) to a file named LULESH_sim_vec in a each simulation run directory. This staging allows QMSprof to use a consistent naming convention for its implementation, without requiring users to duplicate or change input file names in the source directory. To run the sim_vec variant of LULESH, QMSprof will use the sim argument of -s 27 -i 6 -p, since sim is the longest matching prefix of sim_vec.

SNAP has a slightly more complicated description, with a script command list in its gen_inputs parameter. This command list indicates that before staging any files in the files list, the collector should run the script genFile from the SNAP/binaries directory to generate extra input files. The strings is a special pattern in arguments and commands that the collector replaces with the thread count for a particular run. Similarly, is a special pattern that gets represents the run directory used to store and run the binary.

Fig. 7.

Example Benchmarks dictionary for configuring workloads in QMSprof.

Configuration of parts (2) and (3) are relatively straightforward, as illustrated in Fig. 8. The SniperConfigs dictionary describes the Sniper models that can be used in an experiment. The key 16C_2wide is a descriptive (usually short) string that the user provides for the Sniper configuration file Manycore_16c_2wide.cfg. Similarly, in the EnvFiles dictionary, DefaultOpenMP is a description of the environment file ICCDefaultOpenMP.sh. Each run script created by QMSprof sources a particular environment file before each run, passing in the thread count of the run as its argument. Thus, the user should use the environment file to setup any necessary runtime libraries or tools (e.g., compiler libraries, Sniper and SDE), and any other environment variables such as OMP_NUM_THREADS. Finally, Fig. 8 also specifies the job manager (e.g., Intel NetBatch) to use to run jobs in the desired compute environment.

Fig. 8.

Example configuration for QMSprof for Sniper configurations and environment setup.

Finally, for part (4), Fig. 9 shows an example experiment script that specifies the runs in the experiment. Each run (e.g., Run0) is specified as a tuple with 5 entries:

• 1. Sniper configuration: The Sniper configuration for the run, as defined by the keys in the input SniperConfigs dictionary. For example, Run0 runs the 16C_2_wide config.

• 2. Thread Set: The thread counts to run. For example, Run0 uses the thread counts of 1, 2, 4, 8, 12, and 16.

• 3. Experiment File: The environment file for the run, as defined by the keys in the input EnvFiles dictionary. For example, Run0 uses the DefaultOpenMP environment file.

• 4. Program list: The workload variants to run. The example in Fig. 9 executes both the sim_vec and sim_novec variants of LULESH and SNAP.

• 5. Experiment Knobs: An object that captures all the other configuration knobs for a particular run. This example uses default values for all the knobs, but additional customization is possible.

Fig. 9.

Example experiment script for QMSprof.

1.2 A.2 Analyzer Interface

The analyzer for QMSprof is a separate script that takes a single input directory as its argument, scans the input directory for SDE and Sniper simulation output, parses the relevant raw statistics output files, and then generates summary plots. Our prototype for QMSprof has the specific plots demonstrated in Sect. 4 hard-coded as output, but in principle one could implement a more complex interface that would allow for some customization in the generated plots. The analyzer generates Gnuplot scripts and data files as output, which can be manually edited (e.g., to change titles, labels, or legends), and rerun manually to recreate plots.

Our prototype analyzer assumes that simulation output for each simulation run is placed in a separate folder, with the thread count appearing in the folder name. The analyzer uses the names of output folders to group different thread counts for a benchmark together in summary plots, and eliminates the common suffix across all runs to shorten legends in generated plots. These assumptions are designed for processing the output from the QMSprof collector, but one can also use the analyzer to generate summary plots from other simulation runs if the output directories follow a compatible naming convention.