Abstract
In this chapter, we present a generic programming methodology for expressing data structures and algorithms for numerical linear algebra. We also present a high-performance implementation of this approach, the Matrix Template Library (MTL). As with the Standard Template Library, our approach is five-fold, consisting of generic functions, containers, iterators, adapters, and function objects, all developed specifically for high-performance numerical linear algebra. Within our framework, we provide generic functions corresponding to the mathematical operations that define linear algebra. Similarly, the containers, adapters, and iterators are used to represent and to manipulate concrete linear algebra objects such as matrices and vectors. To many scientific computing users, however, the advantages of an elegant programming interface are secondary to issues of performance. There are two aspects to how we achieved high performance in the MTL. The first is the use of static polymorphism (template functions) with modern optimizing compilers, enabling extreme flexibility with no loss in performance. The second is the application of abstraction to the optimization process itself. The Basic Linear Algebra Instruction Set (BLAIS) is presented as an abstract interface to several important performance optimizations. Our experimental results show that MTL with the BLAIS achieves performance that is as good as, or better than, vendor-tuned libraries, even though MTL and the BLAIS are written completely in C++. We therefore conclude that the use of abstraction is not a barrier to performance, contrary to conventional wisdom, and that certain abstractions can in fact facilitate optimization. In addition, MTL requires orders of magnitude fewer lines of code for its implementation, with the concomitant savings in development and maintenance effort.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
E. Anderson, Z. Bai, C. Bischoff, J. Demmel, J. Dongarra, J. DuCroz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen. LAPACK: A portable linear algebra package for high-performance computers. In Proceedings of Supercomputing ’90, pages 1–10. IEEE Press, 1990.
M. H. Austern. Generic Programming and the STL. Professional computing series. Addison-Wesley, 1999.
J. Bilmes, K. Asanovic, J. Demmel, D. Lam, and C.-W. Chin. Optimizing matrix multiply using PHiPAC: A portable, high-performance, ANSI C coding methodology. Technical Report CS-96–326, University of Tennessee, May 1996. Also available as LAPACK working note 111.
S. Carr and Y. Guan. Unroll-and-jam using uniformly generated sets. In Proceedings of the 30th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO-97), pages 349–357, Los Alamitos, Dec.1–3 1997. IEEE Computer Society.
S. Carr and K. Kennedy. Blocking linear algebra codes for memory hierarchies. In D. C. Dongarra, Jack, Messina, Paul, Sorensen and R. G. Voigt, editors, Proceedings of the 4th Conference on Parallel Processing for Scientific Computing, pages 400–405, Philadelphia, PA, USA, Dec. 1989. SIAM Publishers.
L. Carter, J. Ferrante, and S. F. Hummel. Hierarchical tiling for improved superscalar performance. In Proceedings of the 9th International Symposium on Parallel Processing (IPPS’95, pages 239–245, Los Alamitos, CA, USA, Apr. 1995. IEEE Computer Society Press.
K. Czarnecki. Generative Programming: Principles and Techniques of Software Engineering Based on Automated Configuration and Fragment-Based Component Models. PhD thesis, Technische Universitat, Ilmenau, Germany, 1998.
K. Czarnecki and U. Eisenecker. Generative Programming: Methods, Techniques and Applications. Addison-Wesley, 1999. to appear.
K. Czarnecki, U. Eisenecker, R. Glück, D. Vandevoorde, and T. L. Veldhuizen. Generative programming and active libraries. In TBA of Lecture Notes in Computer Science, Dagstuhl-Seminar on Generic Programming, 1998.
J. Dongarra, J. Bunch, C. Moler, and G. Stewart. LINPACK Users Guide. Philadelphia, PA, 1978.
J. Dongarra, J. D. Croz, I. Duff, and S. Hammarling. A set of level 3 basic linear algebra subprograms. ACM Transactions on Mathematical Software, 16(1):1–17, 1990.
J. Dongarra, J. D. Croz, S. Hammarling, and R. Hanson. Algorithm 656: An extended set of basic linear algebra subprograms: Model implementations and test programs. ACM Transactions on Mathematical Software, 14(1):18–32, 1988.
C. Forum. Working paper for draft proposed international standard for information systems - programming language C++. Technical report, American National Standards Institute, 1995.
E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Professional Computing Series. Addison Wesley, 1995.
G. H. Golub and C. F. V. Loan. Matrix Computations. Johns Hopkins, 3rd edition, 1996.
IBM. Engineering and Scientific Subroutine Library, Guide and Reference, 2nd edition, 1992.
Kuck and Associates. Kuck and Associates C++ User’s Guide.
M. S. Lam, E. E. Rothberg, and M. E. Wolf. The cache performance and optimizations of blocked algorithms. In ASPLOS IV, April 1991.
C. Lawson, R. Hanson, D. Kincaid, and F. Krogh. Basic linear algebra subprograms for fortran usage. ACM Transactions on Mathematical Software, 5(3):308–323, 1979.
B. C. McCandless. The role of abstraction in high performance linear algebra. Master’s thesis, University of Notre Dame, 1998.
B. C. McCandless and A. Lumsdaine. The role of abstraction in high-performance computing. In Scientific Computing in Object-Oriented Parallel Environments. ISCOPE, December 1997.
K. Mens, C. Lopes, B. Tekinerdogan, and G. Kiczales. Aspect-oriented programming Lecture Notes in Computer Science, 1357:483–490, 1998.
MTL home page: http://www.lsc.nd.edu/research/mt1
S. Muchnick. Advanced Compiler Design and Implementation. Morgan Kaufmann, 1997.
D. R. Musser and A. Saini. STL tutorial and Reference Guide. Addison-Wesley, Reading, 1996.
Netlib BLAS http://www.netlib.org/blas/index.html.
NIST. MatrixMarket. http://gams.nist.gov/MatrixMarket/.
R. Pozo. Template Numerical Toolkit (TNT) for Linear Algebra. National Insitute of Standards and Technology.
K. A. Remington and R. Pozo. NIST Sparse BLAS User’s Guide. National Institute of Standards and Technology.
Y. Saad. SPARSKIT: a basic tool kit for sparse matrix computations. Technical report, NASA Ames Research Center, 1990.
J. Siek, A. Lumsdaine, and L.-Q. Lee. Generic programming for high perfor-mance numerical linear algebra. In Proceedings of the SIAM Workshop on Ob-ject Oriented Methods for Inter-operable Scientific and Engineering Computing(00’98). SIAM Press, 1999.
B. Smith, J. Boyle, Y. Ikebe, V. Klema, and C. Moler. Matrix Eigensystem Routines: EISPACK Guide. New York, NY, second edition, 1970.
A. A. Stepanov. Generic programming. Lecture Notes in Computer Science, 1181, 1996.
A. A. Stepanov and M. Lee. The Standard Template Library. Technical Report X3J16/94–0095, WG21/N0482, ISO Programming Language C++ Project, May 1994.
B. Stroustrup. The C++ Programming Language. Addison Wesley, 3rd edition, 1997.
T. Veldhuizen. Using C++ template metaprograms. C++ Report, May 1995.
T. L. Veldhuizen. Expression templates. C++ Report, 7(5):26–31, June 1995. Reprinted in C++ Gems, ed. Stanley Lippman.
T. L. Veldhuizen. Arrays in Blitz++. In Proceedings of the 2nd International Scientific Computing in Object-Oriented Parallel Environments (ISCOPE’98), Lecture Notes in Computer Science. Springer-Verlag, 1998.
R. C. Whaley and J. Dongarra. Automatically tuned linear algebra software (ATLAS). Technical report, University of Tennessee and Oak Ridge National Laboratory, 1997.
M. E. Wolf and M. S. Lam. A data locality optimising algorithm. In B. Hailpern, editor, Proceedings of the ACM SIGPLAN ’91 Conference on Programming Language Design and Implementation, pages 30–44, Toronto, ON, Canada, June 1991. ACM Press.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Siek, J., Lumsdaine, A. (2000). A Modern Framework for Portable High-Performance Numerical Linear Algebra. In: Langtangen, H.P., Bruaset, A.M., Quak, E. (eds) Advances in Software Tools for Scientific Computing. Lecture Notes in Computational Science and Engineering, vol 10. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-57172-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-57172-5_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-66557-1
Online ISBN: 978-3-642-57172-5
eBook Packages: Springer Book Archive