Abstract
Current production resource management and scheduling systems often use some mechanism to guarantee fair sharing of computational resources among different users of the system. For example, the user who so far consumed small amount of CPU time gets higher priority and vice versa. However, different users may have highly heterogeneous demands concerning system resources, including CPUs, RAM, HDD storage capacity or, e.g., GPU cores. Therefore, it may not be fair to prioritize them only with respect to the consumed CPU time. Still, applied mechanisms often do not reflect other consumed resources or they use rather simplified and “ad hoc” solutions to approach these issues. We show that such solutions may be (highly) unfair and unsuitable for heterogeneous systems. We provide a survey of existing works that try to deal with this situation, analyzing and evaluating their characteristics. Next, we present new enhanced approach that supports multi-resource aware user prioritization mechanism. Importantly, this approach is capable of dealing with the heterogeneity of both jobs and resources. A working implementation of this new prioritization scheme is currently applied in the Czech National Grid Infrastructure MetaCentrum.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Walltime is the time a job spends executing on a machine(s). It is an important parameter used in the fairshare algorithm as we explain in Sect. 2.
- 2.
A fairshare usage represents the metric of utilization measurement [11]. Typically, it is the amount of consumed CPU time of a given user.
- 3.
In MetaCentrum, resources allocated (i.e., reserved) to a given job cannot be used by other jobs even if those resources are not fully used. Therefore, when speaking about CPU, RAM, etc., requirements we mean the amount of a given resource that has been allocated for a job, even if actual job requirements were smaller. Similarly, a job CPU time is the number of allocated CPUs multiplied by that job walltime.
- 4.
PBS Works is a division of Altair which is responsible for PBS-Pro development. The meeting took place at the Supercomputing 2013 conference in Denver, CO, USA.
- 5.
The additional parameter \(node\_cost_i\) is optional and can be used to express (real) cost and/or importance of machine \(i\), e.g., GPU-equipped nodes are less common (i.e., more valuable) in MetaCentrum. By default, \(node\_cost_i = 1.0\).
- 6.
It is obtained by parsing node specification requests obtained by qsub command.
- 7.
This enhanced TORQUE can be obtained at: https://github.com/CESNET/torque.
- 8.
Detailed description is available at: https://wiki.metacentrum.cz/wiki/Running_jobs_in_scheduler.
References
Adaptive Computing Enterprises, Inc. Maui Scheduler Administrator’s Guide, version 3.2, January 2014. http://docs.adaptivecomputing.com
Adaptive Computing Enterprises, Inc. Moab workload manager administrator’s guide, version 7.2.6, January 2014. http://docs.adaptivecomputing.com
Adaptive Computing Enterprises, Inc. TORQUE Admininstrator Guide, version 4.2.6, January 2014. http://docs.adaptivecomputing.com
Apache.org. Hadoop Capacity Scheduler, January 2014. http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
Apache.org. Hadoop Fair Scheduler, January 2014. http://hadoop.apache.org/docs/r1.2.1/fair_scheduler.html
Apache.org. Hadoop Next Generation Fair Scheduler, January 2014. http://hadoop.apache.org/docs/r2.2.0/hadoop-yarn/hadoop-yarn-site/FairScheduler.html
Blazewicz, J., Drozdowski, M., Markiewicz, M.: Divisible task scheduling - concept and verification. Parallel Comput. 25(1), 87–98 (1999)
Dolev, D., Feitelson, D.G., Halpern, J.Y., Kupferman, R., Linial, N.: No justified complaints: on fair sharing of multiple resources. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS 2012, pp. 68–75. ACM, New York (2012)
Ghodsi, A., Zaharia, M., Hindman, B., Konwinski, A., Shenker, S., Stoica, I.: Dominant resource fairness: fair allocation of multiple resource types. In: 8th USENIX Symposium on Networked Systems Design and Implementation (2011)
Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., Goldberg, A.: Quincy: fair scheduling for distributed computing clusters. In: ACM SIGOPS 22nd Symposium on Operating Systems Principles, pp. 261–276 (2009)
Jackson, D.B., Snell, Q.O., Clement, M.J.: Core algorithms of the maui scheduler. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 87–102. Springer, Heidelberg (2001)
Jain, R., Chiu, D.-M., Hawe, W.: A quantitative measure of fairness and discrimination for resource allocation in shared computer systems. Technical report TR-301, Digital Equipment Corporation (1984)
Joe-Wong, C., Sen, S., Lan, T., Chiang, M.: Multi-resource allocation: fairness-efficiency tradeoffs in a unifying framework. In: 31st Annual International Conference on Computer Communications (IEEE INFOCOM), pp. 1206–1214 (2012)
Kleban, S.D., Clearwater, S.H.: Fair share on high performance computing systems: what does fair really mean? In: Third IEEE International Symposium on Cluster Computing and the Grid (CCGrid2003), pp. 146–153. IEEE Computer Society (2003)
Klusáček, D., Rudová, H.: Alea 2 - job scheduling simulator. In: Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques (SIMUTools 2010). ICST (2010)
Klusáček, D., Rudová, H.: New multi-resource fairshare prioritization mechanisms for heterogeneous computing platforms. In: Cracow Grid Workshop, pp. 89–90. ACC Cyfronet AGH (2013)
Klusáček, D., Rudová, H., Jaroš, M.: Multi resource fairness: problems and challenges. In: Desai, N., Cirne, W. (eds.) JSSPP 2013. LNCS, vol. 8429, pp. 81–95. Springer, Heidelberg (2014)
MetaCentrum, January 2014. http://www.metacentrum.cz/
PBS Works. PBS Professional 12.1, Administrator’s Guide, January 2014. http://www.pbsworks.com
Acknowledgments
We highly appreciate the support of the Grant Agency of the Czech Republic under the grant No. P202/12/0306. The support provided under the programme “Projects of Large Infrastructure for Research, Development, and Innovations” LM2010005 funded by the Ministry of Education, Youth, and Sports of the Czech Republic is highly appreciated. The access to the MetaCentrum computing facilities and workloads is kindly acknowledged.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Klusáček, D., Rudová, H. (2015). Multi-resource Aware Fairsharing for Heterogeneous Systems. In: Cirne, W., Desai, N. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2014. Lecture Notes in Computer Science(), vol 8828. Springer, Cham. https://doi.org/10.1007/978-3-319-15789-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-15789-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15788-7
Online ISBN: 978-3-319-15789-4
eBook Packages: Computer ScienceComputer Science (R0)