Parallel One–Sided Jacobi SVD Algorithm with Variable Blocking Factor

Abstract

Parallel one-sided block-Jacobi algorithm for the matrix singular value decomposition (SVD) requires an efficient computation of symmetric Gram matrices, their eigenvalue decompositions (EVDs) and an update of matrix columns and right singular vectors by matrix multiplication. In our recent parallel implementation with $p$ processors and blocking factor $\ell =2p$, these tasks are computed serially in each processor in a given parallel iteration step because each processor contains exactly two block columns of an input matrix $A$. However, as shown in our previous work, with increasing $p$ (hence, with increasing blocking factor) the number of parallel iteration steps needed for the convergence of the whole algorithm increases linearly but faster than proportionally to $p$, so that it is hard to achieve a good speedup. We propose to break the tight relation $\ell =2p$ and to use a small blocking factor $\ell = p/k$ for some integer $k$ that divides $p$, $\ell$ even. The algorithm then works with pairs of logical block columns that are distributed among processors so that all computations inside a parallel iteration step are themselves parallel. We discuss the optimal data distribution for parallel subproblems in the one-sided block-Jacobi algorithm and analyze its computational and communication complexity. Experimental results with full matrices of order $8192$ show that our new algorithm with a small blocking factor is well scalable and can be $2$–$3$ times faster than the ScaLAPACK procedure PDGESVD.