Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Generalized utility metrics for supercomputers


The problem of ranking the utility of supercomputer systems arises frequently in situations such as procurements and other types of evaluations of architectures. It is also central for any general ranking of supercomputers such as the Top500. Rankings of computer systems have traditionally solely focused on performance aspects. In recent years restrictions due to power and space requirements of large supercomputers have become very noticeable, which has increased the importance of including these factors in generalized rankings. In this paper we present an overview of the current practice for utility metrics and analyze their shortcomings. We then present and discuss in detail a new concept for a parameterized utility metric for supercomputers, which is based on effective performance, available memory size, actual power consumption, and (if desired) the floor space required for supercomputers. This metric is designed and proposed for augmenting the current Top500 ranking.


  1. 1.

    Top500 supercomputer sites. (2009)

  2. 2.

    Strohmaier E, Dongarra J, Meuer HW, Simon HD (1999) The marketplace of high-performance computing. Parall Comput 25(13–14):1517–1544

  3. 3.

    Dongarra J, Luszczek P, Petitet A (2003) The LINPACK benchmark: past, present and future. Concurr Comput Pract Exper 15:1–18

  4. 4.

    The Green500 List. (2009)

  5. 5.

    HPC Challenge Benchmark. (2009)

  6. 6.

    SPEC: Standard performance evaluation corporation. (2009)

  7. 7.

    Feng W-C, Cameron K (2007) The Green500 List: Encouraging Sustainable Supercomputing. Computer 40(12):50–55

  8. 8.

    Makimoto T, Eguchi K, Yoneyama M (2001) The Cooler the Better: New Directions in the Nomadic Age. Computer 34(4):38–42

  9. 9.

    STREAM: Sustainable memory bandwidth in high performance computers. (2009)

  10. 10.

    Oliker L, Carter J, Wehner M, Canning A, Ethier S, Mirin A, Parks D, Worley PH, Kitawaki S, Tsuda Y (2005) Leading Computational Methods on Scalar and Vector HEC Platforms. Proc SC 2005:62

  11. 11.

    Oliker L, Canning A, Carter J, Iancu C, Lijewski M, Kamil S, Shalf J, Shan H, Strohmaier E, Ethier S, Goodale T (2007) Scientific Application Performance on Candidate PetaScale Platforms. Proc IPDPS 2007:1–12

  12. 12.

    Luszczek P, Dongarra J, Koester D, Rabenseifner R, Lucas B, Kepner J, McCalpin J, Bailey D, Takahashi D (2005) Introduction the the HPC challenge benchmark suite. Available at

  13. 13.

    Kramer W, Shalf J, Strohmaier E (2005) The NERSC Sustained System Performance (SSP) Metric. Lawrence Berkeley National Laboratory. Paper LBNL-58868.

Download references

Author information

Correspondence to Erich Strohmaier.

Additional information

This work was supported by the ASCR Office in the DOE Office of Science under contract number DE-AC02-05CH11231.

Rights and permissions

Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Reprints and Permissions

About this article

Cite this article

Strohmaier, E. Generalized utility metrics for supercomputers . Comp. Sci. Res. Dev. 23, 185–193 (2009).

Download citation


  • Computer performance
  • Utility metrics
  • Power efficiency
  • High performance computing market analysis