A Modest Proposal for Communication Costs in Multicomputers

Abstract

Getting rid of the ‘von Neumann’ bottleneck in the shift from sequential to non-sequential computation, a new communication bottleneck arises because of the interplay between locality of computation, communication, and the number of dimensions of physical space. As a consequence, realistic models for non-sequential computation should charge extra for communication, in terms of time and space. We give a proposal for this that is more subtle, but mutatis mutandis similar to the transition from unit cost to logarithmic cost in the sequential Random Access Machine model. The space cost of communication is related with the topology of the communication graph. We use a new lower bound on the average interconnect (edge) length in terms of diameter and symmetry of the topology. This lower bound is technology independent, and shows that many interconnection topologies of today’s multicomputers do not scale well in the physical world with 3 dimensions.

The initial part of this work was performed at the MIT Laboratory for Computer Science, supported in part by the Office of Naval Research under Contract N00014-85-K-0168, by the Office of Army Research under Contract DAAG29-84-K-0058, by the National Science Foundation under Grant DCR-83-02391, and by the Defense Advanced Research Projects Agency (DARPA) under Contract N00014-83-K-0125.