Performance Evaluation of HPC Benchmarks on VMware’s ESXi Server
A major obstacle to virtualizing HPC workloads is a concern about the performance loss due to virtualization. We will demonstrate that new features significantly enhance the performance and scalability of virtualized HPC workloads on VMware’s virtualization platform. Specifically, we will discuss VMware’s ESXi Server performance for virtual machines with up to 64 virtual CPUs as well as support for exposing virtual NUMA topology to guest operating systems, enabling the operating system and applications to make intelligent NUMA aware decisions about memory allocation and process/thread placement. NUMA support is especially important for large VMs which necessarily span host NUMA nodes on all modern hardware. We will show how the virtual NUMA topology is chosen to closely match physical host topology, while preserving the now expected virtualization benefits of portability and load balancing. We show that the benefit of exposing the virtual NUMA topology can lead to performance gains of up to 167%. Overall, we will show close to native performance on applications from SPEC MPI V2.0 and SPEC OMP V3.2 benchmarks virtualized on our prototype VMware’s ESXi Server.
KeywordsNon Uniform Memory Architecture (NUMA) ESXi High Performance Computing (HPC) virtual NUMA (vNUMA) virtualization
Unable to display preview. Download preview PDF.
- 1.Advanced Configuration and Power Interface specification, rev 4.0a (2009), http://www.acpi.info/spec40.htm
- 2.Agesen, O., Garthwaite, A., Sheldon, J., Subrahmanyam, P.: The evolution of an x86 virtual machine monitor. Operating Systems Review 44(4) (2010)Google Scholar
- 3.Scalable Infrastructure with the CPU scheduler in VMware ESX 4.1, http://www.vmware.com/files/pdf/techpaper/VMW_vSphere41_cpu_schedule_ESX.pdf
- 4.Gavrilovska, A., Kumar, S., et al.: High-Performance Hypervisor Architectures: Virtualization in HPC Systems. In: HPCVirt 2007: 1st Workshop on System-level Virtualization for High Performance Computing (2007)Google Scholar
- 7.Nagarajan, A.B., Mueller, F.: Proactive fault tolerance for HPC with Xen Virtualization. In: Proceedings of the 21st Annual International Conference on Supercomputing (ICS 2007), pp. 23–32. ACM Press (2007)Google Scholar
- 8.Open MPI: Open Source High Performance Computing (2011), http://www.open-mpi.org
- 10.Rao, D., Schwan, K.: vNUMA-mgr: Managing VM memory on NUMA platforms. In: 2010 International Conference on High Performance Computing (HiPC), pp. 1–10 (December 2010)Google Scholar
- 12.Valle, G., Engelmann, C., Scott, S.L., Naughton, T., Ong, H.: System-Level Virtualization for High Performance Computing, February 13-15 (2008)Google Scholar
- 14.Youseff, L., Seymour, K., You, H., Dongarra, J., Wolski, R.: The impact of paravirtualized memory hierarchy on linear algebra computational kernels and software. In: Proceedings of the 17th International Symposium on High Performance Distributed Computing, HPDC 2008, pp. 141–152. ACM, New York (2008)CrossRefGoogle Scholar