Analyzing the power consumption behavior of a large scale data center
- 128 Downloads
The aim of this paper is to illustrate the use of application and system level logs to better understand scientific data center behavior and energy-spending. Analyzing a data center log of 900 nodes (Sandy Bridge and Haswell), we study node power consumption and describe approaches to estimate and forecast it. Our results include methods to cluster nodes based on different vmstat and RAPL measurements as well as Gaussian and GAM models for estimating the plug power consumption. We also analyze failed jobs and find that non-successfully terminated jobs consume around 40% of computing time. While the actual numbers are likely to vary in different data centers at different times, the purpose of the paper is to share ideas of what can be found by statistical and machine learning analysis of large amount of log data.
KeywordsRAPL Energy modeling Energy efficiency Data center log analysis
Author Kashif Nizam Khan would like to thank Nokia Foundation for a grant which helped to carry out this work.
- 1.Taito supercluster. https://research.csc.fi/csc-s-servers/taito. Accessed 17 Marc 2017
- 3.Borghesi A, Bartolini A, Lombardi M, Milano M, Benini L (2016) Predictive modeling for job power consumption in HPC systems. Springer, Cham, pp 181–199Google Scholar
- 6.Economou D, Rivoire S, Kozyrakis C, Ranganathan P (2006) Full-system power analysis and modeling for server environments. In: International symposium on computer architecture-IEEEGoogle Scholar
- 8.Hackenberg D, Schöne R, Ilsche T, Molka D, Schuchart J, Geyer R (2015) An energy efficiency feature survey of the Intel Haswell processor. In: 2015 IEEE international parallel and distributed processing symposium workshop, pp. 896–904. https://doi.org/10.1109/IPDPSW.2015.70
- 9.Hirki M, Ou Z, Khan KN, Nurminen JK, Niemi T (2016) Empirical study of the power consumption of the x86-64 instruction decoder. In: USENIX workshop on cool topics on sustainable data centers (CoolDC 16). USENIX Association, Santa Clara, CAGoogle Scholar
- 10.Intel: Intel 64 and IA-32 Architectures Software Developer’s Manual Volume 3 (3A, 3B & 3C): System Programming Guide (2014)Google Scholar
- 14.Molka D, Hackenberg D, Schöne R, Müller MS (2010) Characterizing the energy consumption of data transfers and arithmetic operations on x86-64 processors. In: International conference on green computing, pp 123–133Google Scholar
- 15.Podzimek A, Bulej L, Chen LY, Binder W, Tuma P (2015) Analyzing the impact of cpu pinning and partial cpu loads on performance and energy efficiency. In: 2015 15th IEEE/ACM international symposium on cluster, cloud and grid computing, pp 1–10. https://doi.org/10.1109/CCGrid.2015.164
- 16.Shehabi A, Smith S, Horner N, Azevedo I, Brown R, Koomey J, Masanet E, Sartor D, Herrlin M, Lintner W (2016) United states data center energy usage report. Lawrence Berkeley National Laboratory, Berkeley, California. LBNL-1005775, p 4Google Scholar
- 17.Zhai Y, Zhang X, Eranian S, Tang L, Mars J (2014) HaPPy: hyperthread-aware power profiling dynamically. In: 2014 USENIX annual technical conference (USENIX ATC 14), pp 211–217. USENIX Association, Philadelphia, PAGoogle Scholar