Selected techniques for efficient parallel implementation of dense linear algebra algorithms on the connection machine CM-5/CM-5E

Abstract

Issues raised in the implementation of dense linear algebra algorithms on a distributed memory architecture are discussed along with considerations that arise when the algorithms are implemented as general purpose library routines. We focus on a few methods which have proven to be important in the CMSSL implementation of elimination based algorithms such as LU and QR decomposition and bidiagonalization.

We illustrate how conceptual layout manipulations (without data movements) can be used to transform a multidimensional array — representing multiple problems — into an array with a fixed number of dimensions while preserving the given axes defining the problems.

We discuss the dependence existing between performance and the placement of the temporary arrays used in the implementation of the algorithms under consideration.

We show how, in the context of our analysis, we implement the global BLAS operations as a combination of only structured communication and local BLAS computations.

Finally, timing results show that the performance of the routines scales as we go to larger matrices and larger machines.