Abstract
This paper evaluates the performance of the HPC Challenge benchmarks in several virtual environments, including VMware, KVM and VirtualBox. The HPC Challenge benchmarks consist of a suite of tests that examine the performance of HPC architectures using kernels with memory access patterns more challenging than those of the High Performance LINPACK (HPL) benchmark used in the TOP500 list. The tests include four local (matrix-matrix multiply, STREAM, RandomAccess and FFT) and four global (High Performance Linpack – HPL, parallel matrix transpose – PTRANS, RandomAccess and FFT) kernel benchmarks.
The purpose of our experiments is to evaluate the overheads of the different virtual environments and investigate how different aspects of the system are affected by virtualization. We ran the benchmarks on an 8-core system with Core i7 processors using Open MPI. We did runs on the bare hardware and in each of the virtual environments for a range of problem sizes. As expected, the HPL results had some overhead in all the virtual environments, with the overhead becoming less significant with larger problem sizes. The RandomAccess results show drastically different behavior and we attempt to explain it with pertinent experiments. We show the cause of variability of performance results as well as major causes of measurement error.
This material is based upon work supported in part by the National Science Foundation under Grant No. 0910812 to Indiana University for “FutureGrid: An Experimental, High-Performance Grid Test-bed.” Partners in the FutureGrid project include U. Chicago, U. Florida, San Diego Supercomputer Center - UC San Diego, U. Southern California, U. Texas at Austin, U. Tennessee at Knoxville, U. of Virginia, Purdue I., and T-U. Dresden.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Danciu, V.A., gentschen Felde, N., Kranzlmüller, D., Lindinger, T.: High-performance aspects in virtualized infrastructures. In: 4th International DMTF Academic Alliance Workshop on Systems and Virtualization Management, pp. 25–32 (October 2010)
Han, J., Ahn, J., Kim, C., Kwon, Y., Choi, Y.-r., Huh, J.: The Effect of Multi-core on HPC Applications in Virtualized Systems. In: Guarracino, M.R., Vivien, F., Träff, J.L., Cannatoro, M., Danelutto, M., Hast, A., Perla, F., Knüpfer, A., Di Martino, B., Alexander, M. (eds.) Euro-Par-Workshop 2010. LNCS, vol. 6586, pp. 615–623. Springer, Heidelberg (2011)
Hines, S., Wyatt, B., Chang, J.M.: Increasing timing resolution for processes and threads in Linux (2000) (unpublished)
Huang, W., Liu, J., Abali, B., Panda, D.: A case for high performance computing with virtual machines. In: Proceedings of the 20th Annual International Conference on Supercomputing (2006)
Li, A., Yang, X., Kandula, S., Zhang, M.: CloudCmp: comparing public cloud providers. In: 10th Annual Conference on Internet Measurement (2010)
Luszczek, P., Bailey, D., Dongarra, J., Kepner, J., Lucas, R., Rabenseifner, R., Takahashi, D.: The HPC challenge HPCC benchmark suite. In: SuperComputing 2006 Conference Tutorial (2006)
Mei, Y., Liu, L., Pu, X., Sivathanu, S.: Performance measurements and analysis of network I/O applications in virtualized cloud. In: IEEE 3rd International Conference on Cloud Computing, pp. 59–66 (August 2010)
Smith, K.A., Selzter, M.: File layout and file system performance. Computer Science Technical Report TR-35-94, Harvard University (1994)
Tao, J., Fürlinger, K., Marten, H.: Performance Evaluation of OpenMP Applications on Virtualized Multicore Machines. In: Chapman, B.M., Gropp, W.D., Kumaran, K., Müller, M.S. (eds.) IWOMP 2011. LNCS, vol. 6665, pp. 138–150. Springer, Heidelberg (2011)
Tsugawa, M., Fortes, J.A.B.: Characterizing user-level network virtualization: performance, overheads and limits. International Journal of Network Management (2009), doi:10.1002/nem.733
Timekeeping in VMware Virtual Machines: VMware ESX 4.0/ESXi 4.0, VMware workstation 7.0 information guide
Walters, J., Chaudhary, V., Cha, M., Guercio, S.J., Gallo, S.: A comparison of virtualization technologies for HPC. In: 22nd International Conference on Advanced Information Networking and Applications, pp. 861–868 (March 2008)
Younge, A.J., Henschel, R., Brown, J.T., von Laszewski, G., Qiu, J., Fox, G.C.: Analysis of virtualization technologies for High Performance Computing environments. In: Proceedings of The Fourth IEEE International Conference on Cloud Computing (CLOUD 2011), Washington Marriott, Washington DC, USA, July 4-9 (2011); technical Report (February 15, 2011), updated (April 2011)
Youseff, L., Wolski, R., Gorda, B., Krintz, C.: Paravirtualization for HPC Systems. In: Min, G., Di Martino, B., Yang, L.T., Guo, M., Rünger, G. (eds.) ISPA Workshops 2006. LNCS, vol. 4331, pp. 474–486. Springer, Heidelberg (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Luszczek, P., Meek, E., Moore, S., Terpstra, D., Weaver, V.M., Dongarra, J. (2012). Evaluation of the HPC Challenge Benchmarks in Virtualized Environments. In: Alexander, M., et al. Euro-Par 2011: Parallel Processing Workshops. Euro-Par 2011. Lecture Notes in Computer Science, vol 7156. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29740-3_49
Download citation
DOI: https://doi.org/10.1007/978-3-642-29740-3_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29739-7
Online ISBN: 978-3-642-29740-3
eBook Packages: Computer ScienceComputer Science (R0)