Computational Metrics and Fundamental Limits for Parallel Architectures a Survey of Pertinent Research

Abstract

Computational metrics and fundamental limits from several relevant theories are outlined as quantitative measures and target goals for parallel architectures. Theory outlines are given for Lipovsky and Malek’s “physics” of parallel computation, basic graph theory, VLSI complexity theory (abstracted from Ullman’s text, with extensions for interconnection networks by Snir, and for parallel architectures by the author), communication “field theory”, Cohn et al.’s uniform bounds on parallel computation, Cvetanovic’s analysis of application parallelism, and a composite analysis of the “inductiveness” of parallel applications executing on parallel architectures. Terminology is intentionally intermingled to facilitate comparison of concepts. Apparent joint conclusions of several of the theories supports the existence of scalable parallel architectures with O(n) speedup (where n is the number of processors) and O(n ^−1/2) or O(n ^−1/3) efficiency for sufficiently parallel problems and using conventional technology. While better technology may offer constant factor improvements, it cannot do asymptotically better than this. Also it is shown that parallel architectures can saturate, or worse, “thrash” on certain classes of applications, and that careful mappings of applications onto parallel systems is necessary in many cases to take advantage of communications locality. This study addresses only theoretical aspects of parallel architecture, and practical issues of pro-grammability or suitability for particular purposes are not considered.