Abstract
Understanding user behavior is great helpful for assessing HPC system job scheduling, promoting allocation efficiency and improving user satisfaction. Current research on user behavior is mainly focused on think time (i.e. time between two consecutive jobs) of non-commercial supercomputer systems. In this paper, we present a methodology to characterize workloads of the commercial supercomputer. We use it to analyze the 2.7 million jobs of different users in various fields of Tianhe-1A from 2016.01 to 2017.12 and 0.89 million jobs of Sugon 5000A from 2015.09 to 2017.03.
In order to identify the main factors affecting the user’s job submission behavior on commercial supercomputers, this paper analyzed the correlation between user’s job submission behavior and various factors such as job characteristics and quota constraint. The result shows that, on the commercial supercomputer, user s job submission behavior is not obviously affected by the previous job’s runtime and waiting time. It is affected by the number of processors the job uses, the previous job’s status and the size of the total resources that users can submit jobs. We also find that, there are three job submission peaks on each day. In the time window of 8 h, 86% jobs of a same user have the same number of processors and nearly 40% of them have little difference in runtime.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Geist, A., et al.: A survey of high-performance computing scaling challenges. Int. J. High Perform. Comput. Appl. 33(1), 104–113 (2017)
Reed, D.A., Dongarra, J.: Exascale computing and big data. Commun. ACM 58(7), 56–68 (2015)
Shmueli, E., Feitelson, D.G.: Uncovering the effect of system performance on user behavior from traces of parallel systems. In: International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 274–280 (2007)
Feitelson, D.G.: Looking at data. In: IEEE International Symposium on Parallel and Distributed Processing (IPDPS), pp. 1–9 (2008)
Schlagkamp, S. et al.: Consecutive job submission behavior at mira supercomputer. In: International Symposium on High-Performance Parallel and Distributed Computing (HPDC), pp. 93–96 (2016)
Sun, N., et al.: High-performance computing in China: research and applications. Int. J. High Perform. Comput. Appl. 24(4), 363–409 (2010)
Rodrigo, G.P., et al.: Towards understanding job heterogeneity in HPC: a NERSC case study. In: IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 521–526 (2016)
Rodrigo, G.P., et al.: Towards understanding HPC users and systems: a NERSC case study. J. Parallel Distrib. Comput. 111, 206–221 (2017)
Luu, H., et al.: A multiplatform study of I/O behavior on petascale supercomputers. In: International Symposium on High-Performance Parallel and Distributed Computing (HPDC), pp. 33–44 (2015)
Schlagkamp, S., et al.: Analyzing users in parallel computing: a user-oriented study. In: International Conference on High Performance Computing and Simulation, pp. 395–402 (2016)
Zakay, N., Feitelson, Dror G.: On identifying user session boundaries in parallel workload logs. In: Cirne, W., Desai, N., Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2012. LNCS, vol. 7698, pp. 216–234. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35867-8_12
Schlagkamp, S., et al.: Understanding user behavior: from HPC to HTC. Procedia Comput. Sci. 80, 2241–2245 (2016)
http://www.ssc.net.cn/resources_1.aspx, 2018/04/28
Yoo, Andy B., Jette, Morris A., Grondona, M.: SLURM: Simple Linux Utility for Resource Management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003). https://doi.org/10.1007/10968987_3
https://git.ustclug.org/yshen/CSWA/tree/master/ssc. Accessed 28 Apr 2018
http://www.cs.huji.ac.il/labs/parallel/workload/. Accessed 26 Apr 2018
Acknowledgments
This research was supported by the National Key R&D Program of China (NO.2016YFB0201404) and Tianjin Binhai Industrial Cloud Public Service Platform and Application Promotion Project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Feng, J., Liu, G., Zhang, Z., Li, T., Li, Y., Sun, F. (2018). Quota-constrained Job Submission Behavior at Commercial Supercomputer. In: Li, C., Wu, J. (eds) Advanced Computer Architecture. ACA 2018. Communications in Computer and Information Science, vol 908. Springer, Singapore. https://doi.org/10.1007/978-981-13-2423-9_17
Download citation
DOI: https://doi.org/10.1007/978-981-13-2423-9_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-2422-2
Online ISBN: 978-981-13-2423-9
eBook Packages: Computer ScienceComputer Science (R0)
