Abstract
The article is focused on user estimates inaccuracy study in high performance computing scheduler. The main goal of this survey is to study historical job submission data from the real supercomputer cenre and investigate accuracy of resources estimated by user on job submission. Based on this study 68.94% of jobs used less than 25% of their requested walltime, which indicates a high estimation inaccuracy. In case of estimation of amount of memory only 31.46% of jobs uses more than 75% of requested memory. Analysis of historical jobs from the further prediction of resources accuracy has been made. Similarities of the new submitted jobs with historical data were described by keys based on user names, software name, walltime and number of processors. Also mathematical model for jobs scheduling in HPC system, with a goal functions, has been proposed and verified.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bożejko, W.: Solving the flow shop problem by parallel programming. J. Parallel Distrib. Comput. 69(5), 470–481 (2009)
Bożejko, W.: On single-walk parallelization of the job shop problem solving algorithms. Comput. Oper. Res. 39(9), 2258–2264 (2012)
Bożejko, W., Uchroński, M., Wodecki, M.: Block approach to the cyclic flow shop scheduling. Comput. Ind. Eng. 81, 158–166 (2015)
Bożejko, W., Wodecki, M.: Parallel genetic algorithm for minimizing total weighted completion time. LNAI (L NCS), vol. 3070. Springer, Heidelberg (2004)
Chiang, S.H., Arpaci-Dusseau, A., Vernon, M.K.: The Impact of More Accurate Requested Runtimes on Production Job Scheduling Performance. In: Job scheduling strategies for parallel processing, 8th International Workshop, JSSPP 2002, Edinburgh, Scotland, UK, 24 July 2002 Revised Papers, pp. 103–127. Springer, Heidelberg (2002)
Cirne, W., Berman, F.: A comprehensive model of the supercomputer workload. In: 2001 IEEE International Workshop on Workload Characterization 2001, WWC-4, pp. 140–148, December 2001
Lan, Z., Buettner, D., Desai, N., Tang, W.: Job scheduling with adjusted runtime estimates on production supercomputers. J. Parallel Distrib. Comput. 73, 926–938 (2013)
Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans. Parallel Distrib. Syst. 12(6), 529–543 (2001)
Rodrigo, G.P., Östberg, P.O., Elmroth, E., Antypas, K., Gerber, R., Ramakrishnan, L.: Towards understanding HPC users and systems: a NERSC case study. J. Parallel Distrib. Comput. 111, 206–221 (2018)
Smith, W., Taylor, V., Foster, I.: Using run-time predictions to estimate queue wait times and improve scheduler performance. In: Job Scheduling Strategies for Parallel Processing: IPPS/SPDP’99 Workshop, JSSPP 1999, San Juan, Puerto Rico, 16 April 1999 Proceedings, pp. 202–219. Springer, Heidelberg (1999)
Srinivasan, S., Kettimuthu, R., Subramani, V., Sadayappan, P.: Characterization of backfilling strategies for parallel job scheduling. In: International Conference on Parallel Processing Workshops 2002 Proceedings, pp. 514–519 (2002)
Ward, W.A., Mahood, C.L., West, J.E.: Scheduling jobs on parallel systems using a relaxed backfill strategy. In: Job Scheduling Strategies for Parallel Processing: 8th International Workshop, JSSPP 2002, Edinburgh, Scotland, UK, 24 July 2002, Revised Papers, pp. 88–102. Springer, Heidelberg (2002)
Acknowledgement
The paper was partially supported by the National Science Centre of Poland, grant OPUS no. DEC 2017/25/B/ST7/02181.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Uchroński, M., Bożejko, W., Krajewski, Z., Tykierko, M., Wodecki, M. (2019). User Estimates Inaccuracy Study in HPC Scheduler. In: Zamojski, W., Mazurkiewicz, J., Sugier, J., Walkowiak, T., Kacprzyk, J. (eds) Contemporary Complex Systems and Their Dependability. DepCoS-RELCOMEX 2018. Advances in Intelligent Systems and Computing, vol 761. Springer, Cham. https://doi.org/10.1007/978-3-319-91446-6_47
Download citation
DOI: https://doi.org/10.1007/978-3-319-91446-6_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91445-9
Online ISBN: 978-3-319-91446-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)