Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Flexible workload generation for HPC cluster efficiency benchmarking

  • 214 Accesses

  • 10 Citations


The High Performance Computing (HPC) community is well-accustomed to the general idea of benchmarking. In particular, the TOP500 ranking as well as its foundation—the Linpack benchmark—have shaped the field since the early 1990s. Other benchmarks with a larger workload variety such as SPEC MPI2007 are also well-accepted and often used to compare and rate a system’s computational capability. However, in a petascale and soon-to-be exascale computing environment, the power consumption of HPC systems and consequently their energy efficiency have been and continue to be of growing importance, often outrivaling all aspects that focus narrowly on raw compute performance. The Green500 list is the first major attempt to rank the energy efficiency of HPC systems. However, its main weakness is again the focus on a single, highly compute bound algorithm. Moreover, its method of extrapolating a system’s power consumption from a single node is inherently error-prone. So far, no benchmark is available that has been developed from ground up with the explicit focus on measuring the energy efficiency of HPC clusters. We therefore introduce such a benchmark that includes transparent energy measurements with professional power analyzers. Our efforts are based on well-established standards (C, POSIX-IO and MPI) to ensure a broad applicability. Our well-defined and comprehensible workloads can be used to, e.g. compare the efficiency of HPC systems or to track the effects of power saving mechanisms that can hardly be understood by running regular applications due to their overwhelming complexity.

This is a preview of subscription content, log in to check access.


  1. 1.

    Barroso LA, Holzle U (2007) The case for energy-proportional computing. Computer 40:33–37. http://doi.ieeecomputersociety.org/10.1109/MC.2007.443

  2. 2.

    Brunst H, Hackenberg D, Juckeland G, Rohling H (2010) Comprehensive performance tracking with Vampir 7. In: Müller MS, Resch MM, Schulz A, Nagel WE (eds) Tools for high performance computing 2009. Springer, Berlin, pp 17–29. doi:10.1007/978-3-642-11261-4_2

  3. 3.

    Feng X, Ge R, Cameron KW (2005) Power and energy profiling of scientific applications on distributed systems. In: Proceedings of the 19th IEEE international parallel and distributed processing symposium (IPDPS’05)—papers, vol 01, IPDPS ’05. IEEE Comput Soc, Washington, p 34. doi:10.1109/IPDPS.2005.346

  4. 4.

    Hackenberg D, Schöne R, Molka D, Müller MS, Knüpfer A (2010) Quantifying power consumption variations of HPC systems using SPEC MPI benchmarks. Comput Sci Res Dev 25:155–163. doi:10.1007/s00450-010-0118-0

  5. 5.

    Intel (2010) Intel®energy checker—SDK device driver kit user guide, 2.0 edn. 2011/05/06 online at http://software.intel.com/file/32935

  6. 6.

    Intel (2010) Intel®energy checker—software developer kit user guide, 2.0 edn

  7. 7.

    Lange KD (2009) Identifying shades of green: the SPECpower benchmarks. Computer 42:95–97. http://doi.ieeecomputersociety.org/10.1109/MC.2009.84

  8. 8.

    Minartz T, Kunkel J, Ludwig T (2010) Simulation of power consumption of energy efficient cluster hardware. Comput Sci Res Dev 25:165–175. doi:10.1007/s00450-010-0120-6

  9. 9.

    Molka D, Hackenberg D, Schöne R, Müller MS (2010) Characterizing the energy consumption of data transfers and arithmetic operations on x86-64 processors. In: Proceedings of the 1st international green computing conference. IEEE Press, New York, pp 123–133. doi:10.1109/GREENCOMP.2010.5598316

  10. 10.

    Ryckbosch F, Polfliet S, Eeckhout L (2011) Trends in server energy-proportionality. Computer 99(PrePrints). http://doi.ieeecomputersociety.org/10.1109/MC.2011.130

  11. 11.

    Schöne R, Hackenberg D (2011) On-line analysis of hardware performance events for workload characterization and processor frequency scaling decisions. In: Proceeding of the second joint WOSP/SIPEW international conference on performance engineering, ICPE ’11. ACM, New York, pp 481–486. doi:10.1145/1958746.1958819

  12. 12.

    Schöne R, Tschüter R, Hackenberg D, Ilsche T (2011) The VampirTrace plugin counter interface: introduction and examples. In: Proceedings of the EuroPar 2010—workshops, pp 501–511

  13. 13.

    Shuaiwen S, Rong G, Xizhou F, Cameron KW (2009) Energy profiling and analysis of the hpc challenge benchmarks. Int J High Perform Comput Appl 23:265–276. doi:10.1177/1094342009106193. http://portal.acm.org/citation.cfm?id=1572226.1572228

Download references

Author information

Correspondence to Daniel Molka.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Molka, D., Hackenberg, D., Schöne, R. et al. Flexible workload generation for HPC cluster efficiency benchmarking. Comput Sci Res Dev 27, 235–243 (2012). https://doi.org/10.1007/s00450-011-0194-9

Download citation


  • Power consumption
  • Energy efficiency
  • Benchmark
  • High performance computing
  • Workload generation