vPlacer: A Co-scheduler for Optimizing the Performance of Parallel Jobs in Xen
Xen, a popular virtualization platform which enables multiple operating systems sharing one physical host, has been widely used in various fields nowadays. Currently, the existing schedulers of Xen are initially targeting at serial jobs, which achieves a remarkable utilization of computer hardware and impressive overall performance. However, the virtualized systems are expected to accommodate both parallel jobs and serial jobs in practice, and resource contention between virtual machines results in severe performance degradation of the parallel jobs. Moreover, the physical resource is vastly wasted during the communication process due to the ineffective scheduling of parallel jobs.
This paper aims to optimize the performance of the parallel jobs in Xen using the co-scheduling mechanism. In this paper, we statistically analyze the process of scheduling parallel jobs in Xen, which points out that the credit scheduler is not capable of properly scheduling a parallel job. Moreover, we propose vPlacer, a conservative co-scheduler to improve the performance of the parallel job in Xen. Our co-scheduler is able to identify the parallel jobs and optimize the scheduling process to satisfy the particularity of the parallel job. The prototype of our vPlacer is implemented, and the experimental results show that the performance of the parallel job is significantly improved and the utilization of the hardware resource is optimized.
KeywordsXen Virtualization Parallel job Scheduler
This work is partially supported by the National Key R&D Program of China 2018YFB1003201 and Guangdong Pre-national Project 2014GKXM054.
- 1.Ackaouy, E.: The Xen credit Cpu scheduler. In: Proceedings of (2006)Google Scholar
- 2.Amazon EC2: Amazon web services (2015). http://aws.amazon.com/es/ec2/. Accessed Nov 2012
- 4.Chen, H., Jin, H., Hu, K., Huang, J.: Scheduling overcommitted VM: behavior monitoring and dynamic switching-frequency scaling. Futur. Gener. Comput. Syst. 29(1), 341–351 (2013). https://doi.org/10.1016/j.future.2011.08.006, http://www.sciencedirect.com/science/article/pii/S0167739X11001452, including Special section: AIRCC-NetCoM 2009 and Special section: Clouds and Service-Oriented Architectures
- 5.Chen, T.Y., Wei, H.W., Wei, M.F., Chen, Y.J., Hsu, T., Shih, W.K.: LaSA: a locality-aware scheduling algorithm for hadoop-mapreduce resource assignment. In: 2013 International Conference on Collaboration Technologies and Systems (CTS), pp. 342–346, May 2013. https://doi.org/10.1109/CTS.2013.6567252
- 6.Gorda, B., Brooks, E.I.: Gang scheduling a parallel machine, p. 3 (1991)Google Scholar
- 7.Huang, W., Koop, M.J., Gao, Q., Panda, D.K.: Virtual machine aware communication libraries for high performance computing. In: Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, SC 2007, pp. 9:1–9:12. ACM, New York (2007). https://doi.org/10.1145/1362622.1362635
- 8.Huang, W., Liu, J., Abali, B., Panda, D.K.: A case for high performance computing with virtual machines. In: Proceedings of the 20th Annual International Conference on Supercomputing, ICS 2006, pp. 125–134. ACM, New York (2006). https://doi.org/10.1145/1183401.1183421
- 9.Kang, H., Chen, Y., Wong, J.L., Sion, R., Wu, J.: Enhancement of Xen’s scheduler for mapreduce workloads. In: Proceedings of the 20th International Symposium on High Performance Distributed Computing, HPDC 2011, pp. 251–262. ACM, New York (2011). https://doi.org/10.1145/1996130.1996164
- 11.Shao, Z., Wang, Q., Xie, X., Jin, H., He, L.: Analyzing and improving MPI communication performance in overcommitted virtualized systems. In: 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems, pp. 381–389, July 2011. https://doi.org/10.1109/MASCOTS.2011.27
- 12.Vallee, G., Naughton, T., Engelmann, C., Ong, H., Scott, S.L.: System-level virtualization for high performance computing. In: 16th Euromicro Conference on Parallel, Distributed and Network-Based Processing (PDP 2008), pp. 636–643, February 2008. https://doi.org/10.1109/PDP.2008.85
- 13.Xi, S., Wilson, J., Lu, C., Gill, C.: RT-Xen: towards real-time hypervisor scheduling in Xen. In: Proceedings of the Ninth ACM International Conference on Embedded Software, EMSOFT 2011, pp. 39–48. ACM, New York (2011). https://doi.org/10.1145/2038642.2038651
- 14.Ye, K., Jiang, X., Chen, S., Huang, D., Wang, B.: Analyzing and modeling the performance in Xen-based virtual cluster environment. In: 2010 IEEE 12th International Conference on High Performance Computing and Communications (HPCC), pp. 273–280, September 2010. https://doi.org/10.1109/HPCC.2010.79
- 15.Youseff, L., Wolski, R., Gorda, B., Krintz, C.: Evaluating the performance impact of Xen on mpi and process execution for HPC systems. In: Proceedings of the 2nd International Workshop on Virtualization Technology in Distributed Computing, p. 1. IEEE Computer Society, Washington, D.C. (2006). https://doi.org/10.1109/VTDC.2006.4