Enforcing Textual Alignment of Collectives Using Dynamic Checks

Abstract

Many parallel programs are written in a single-program, multiple-data (SPMD) style, in which synchronization is provided using collective operations that all threads execute simultaneously. If these operations are not properly aligned on all threads, deadlock can occur, and many compiler analyses and optimizations that depend on proper alignment fail. In this paper, we discuss the flaws in the Titanium language’s type system for enforcing textual alignment of collectives. We then present a system that uses runtime checks to ensure alignment for two definitions of textual alignment. The system instruments the code to keep track of alignment in each thread and then checks that alignment matches prior to performing a collective operation. We have implemented the system in the Titanium compiler, verifying that it catches alignment errors. We tested its performance on multiple application programs, demonstrating that the checks have no appreciable impact on execution time.

This work was supported in part by the Department of Energy under DE-FC03-01ER25509, FDDE-FC02-07ER25799, and Lawrence Berkeley National Laboratory Contract DE-AC02-05CH11231, by the National Science Foundation under CNS-0325873 and OCI-0749190, by the California State MICRO Program, by Microsoft (Award #024263) and Intel (Award #024894) funding and by matching funding by U.C. Discovery (Award #DIG07-10227), and by gifts from Sun Microsystems. The information presented here does not necessarily reflect the position or the policy of the Government and no official endorsement should be inferred.