Compiler Synthesis of Task Graphs for Parallel Program Performance Prediction

Abstract

Task graphs and their equivalents have proved to be a valuable abstraction for representing the execution of parallel programs in a number of different applications. Perhaps the most widespread use of task graphs has been for performance modeling of parallel programs, including quantitative analytical models [3],[19],[25],[26],[27], theoretical and abstract analytical models [14], and program simulation [5],[13]. A second important use of task graphs is in parallel programming systems. Parallel programming environments such as PYRROS [28], CODE [24], HENCE [24], and Jade [20] have used task graphs at three different levels: as a programming notation for expressing parallelism, as an internal representation in the compiler for computation partitioning and communication generation, and as a runtime representation for scheduling and execution of parallel programs. Although the task graphs used in these systems differ in representation and semantics (e.g., whether task graph edges capture purely precedence constraints or also dataflow requirements), there are close similarities. Perhaps most importantly, they all capture the parallel structure of a program separately from the sequential computations, by breaking down the program into computational “tasks”, precedence relations between tasks, and (in some cases) explicit communication or synchronization operations between tasks.

Acknowledgements

The authors would like to acknowledge the valuable input that several members of the POEMS project have provided to the development of the application representation.We are particularly grateful to the UCLA team and especially Ewa Deelman for her efforts in validating the simulations of the abstractedMPI codes generated by the compiler. This work was carried out while the authors were with the Computer Science Department at Rice University.