An overview of a compiler for scalable parallel machines

Abstract

This paper presents an overview of a parallelizing compiler to automatically generate efficient code for large-scale parallel architectures from sequential input programs. This research focuses on loop-level parallelism in dense matrix computations. We illustrate the basic techniques the compiler uses by describing the entire compilation process for a simple example.

Our compiler is organized into three major phases: analyzing array references, allocating the computation and data to the processors to optimize parallelism and locality, and generating code.

An optimizing compiler for scalable parallel machines requires more sophisticated program analysis than the traditional data dependence analysis. Our compiler uses a precise data-flow analysis technique to identify the producer of the value read by each instance of a read access. In order to allocate the computation and data to the processors, the compiler first transforms the program to expose loop-level parallelism in the computation. It then finds a decomposition of the computation and data such that parallelism is exploited and the communication overhead is minimized. The compiler will trade off extra degrees of parallelism to reduce or eliminate communication. Finally, the compiler generates code to manage the multiple address spaces and to communicate data across processors.

This research was supported in part by DARPA contracts N00039-91-C-0138 and DABT63-91-K-0003, an NSF Young Investigator Award and fellowships from Digital Equipment Corporation's Western Research Laboratory and Intel Corporation.