New Generalized Data Structures for Matrices Lead to a Variety of High Performance Algorithms

  • Fred G. Gustavson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2328)


We describe new data structures for full and packed storage of dense symmetric/triangular arrays that generalize both. Using the new data structures one is led to several new algorithms that save “half” the storage for symmetric matrices and outperform the current blocked based level 3 algorithms in LAPACK. We concentrate on the simplest forms of the new algorithms and show they are a direct generalization of LINPACK. This means that level 3 BLAS’s are not required to obtain level 3 performance. The replacement for Level 3 BLAS are so-called kernel routines, see [1], and on IBM platforms they are producible from simple textbook type codes, by the XLF Fortran compiler. In the sequel I will label these “vanilla” codes. On Power3 with a peak performance of 800 MFlops, the results for Cholesky factorization at order n ≥ 200 is over 720 MFlops and then reaches 735 MFlops at n = 400. Using conventional full format LAPACK DPOTRF with ESSL BLAS’s one first gets to 600 MFlops at n ≥ 600 and only reaches a peak of 620 MFlops. For this result we used simple square blocked full matrix data formats where the blocks themselves are stored in column major ( Fortran ) order or row major ( C ) order. The simple algorithms of LU factorization with partial pivoting for this new data format is a direct generalization of LINPACK algorithm DGEFA. Again, no conventional level 3 BLAS’s are required; the replacements are again so-called kernel routines. Programming for squared blocked full matrix format can be accomplished in standard Fortran through the use of three and four dimensional arrays. Thus, no new compiler support is necessary. Also we mention that other more complicated algorithms are possible; e.g., recursive ones. The recursive algorithms are also easily programmed via the use of tables that address where the blocks are stored in the two dimensional recursive block array. Finally, we describe block hybrid formats. Doing so allows one to use no additional storage over conventional ( full and packed ) matrix storage. This means the new algorithms are completely portable.


Cholesky Factorization Full Format Packed Array Block Column Partial Pivoting 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    R. C. Agarwal, F. G. Gustavson, M. Zubair. Exploiting functional parallelism of POWER2 to design high-performance numerical algorithms. IBM Journal of Research and Development, Vol. 38, No. 5, Sep. 1994, pp. 563,576.CrossRefGoogle Scholar
  2. 2.
    J. J. Dongarra, F. G. Gustavson, A. Karp. Implementing Linear Algebra Algorithms for Dense Matrices on a Vector Pipeline Machine. SIAM Review, Vol. 26, No. 1, Jan. 1984, pp. 91,112.MathSciNetCrossRefGoogle Scholar
  3. 3.
    E. W. Elmroth and F. G. Gustavson. Applying Recursion to Serial and Parallel QR Factorization Leads to Better Performance. IBM Journal of Research and Development, Vol. 44, No. 4, July 2000, pp. 605,624.CrossRefGoogle Scholar
  4. 4.
    F. G. Gustavson and I. Jonsson. Minimal Storage High Performance Cholesky via Blocking and Recursion IBM Journal of Research and Development, Vol. 44, No. 6, Nov. 2000, pp. 823,849CrossRefGoogle Scholar
  5. 5.
    G. Golub, C. VanLoan. Matrix Computations, Book. Johns Hopkins Press Baltimore and London.Google Scholar
  6. 6.
    F. G. Gustavson. Recursion Leads to Automatic Variable Blocking for Dense Linear-Algebra Algorithms. IBM Journal of Research and Development, Vol. 41, No. 6, Nov. 1997, pp. 737,755.CrossRefGoogle Scholar
  7. 7.
    F. G. Gustavson. Notes sent to Umea on data formats and descriptors. March 1998.Google Scholar
  8. 8.
    F. G. Gustavson. Notes on Blocked Packed Format and Square Blocked Format for Symmetric/Triangular Arrays. Sep. 1999Google Scholar
  9. 9.
    F. G. Gustavson, A. Henriksson, I. Jonsson, B. Kagstrom, P. Ling. Recursive Blocked Data Formats and BLAS’s Dense Linear Algebra Algorithms. In B. Kagstrom, editors Applied Parallel Computing, Large Scale Scientific and Industrial Problems, Lecture Notes in Computer Science, No. 1541, 1998, pp. 195,206.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Fred G. Gustavson
    • 1
  1. 1.IBM T.J. Watson Research CenterYorktown Heights

Personalised recommendations