The I/O Complexity of Sparse Matrix Dense Matrix Multiplication

Abstract

We consider the multiplication of a sparse N ×N matrix A with a dense N ×N matrix B in the I/O model. We determine the worst-case non-uniform complexity of this task up to a constant factor for all meaningful choices of the parameters N (dimension of the matrices), k (average number of non-zero entries per column or row in A, i.e., there are in total kN non-zero entries), M (main memory size), and B (block size), as long as M ≥ B ² (tall cache assumption).

For large and small k, the structure of the algorithm does not need to depend on the structure of the sparse matrix A, whereas for intermediate densities it is possible and necessary to find submatrices that fit in memory and are slightly denser than on average.

The focus of this work is asymptotic worst-case complexity, i.e., the existence of matrices that require a certain number of I/Os and the existence of algorithms (sometimes depending on the shape of the sparse matrix) that use only a constant factor more I/Os.