Abstract
As cloud computing emerges and gains acceptance, more and more software applications of various domains are transforming into the SaaS model. Recently, the concept of HPC as a Service (HPCaaS) was proposed to bring the traditional high performance computing field into the era of cloud computing. One of its goals aims to allow users to get easier access to HPC facilities and applications. This paper deals with related job submission and scheduling issues to achieve such goal. Traditional HPC users in supercomputing centers are required to specify the amount of processors to use upon job submission. However, we think this requirement might not be necessary for HPCaaS users since most modern parallel jobs are moldable and they usually could not know how to choose an appropriate amount of processors to allow their jobs to finish earlier. Therefore, we propose a moldable job scheduling approach which relieves HPC users’ burden of selecting an appropriate number of processors and can achieve even better system performance than existing job scheduling methods. The experimental results indicate that our approach can achieve up to 75% performance improvement than the traditional rigid processor allocation method and 3% improvement than previous moldable job scheduling methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
AbdelBaky, M., Parashar, M., Kim, H., JordanKirk, E.J., Sachdeva, V., Sexton, J., Jamjoom, H., Shae, Z.Y., Pencheva, G., Tavakoli, R., Wheeler, M.F.: Enabling High Performance Computing as a Service. IEEE Computer 45, 72–80 (2012)
Feitelson, D.G., Weil, A.M.: Utilization and Predictability in Scheduling the IBM SP2 with Backfilling. In: 12th Int’l Parallel Processing Symp., pp. 542–546 (April 1998)
Gibbons, R.: A Historical Application Profiler for Use by Parallel Schedulers. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1997 and JSSPP 1997. LNCS, vol. 1291, pp. 58–77. Springer, Heidelberg (1997)
Lifka, D.: The ANL/IBM SP Scheduling System. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1995 and JSSPP 1995. LNCS, vol. 949, pp. 295–303. Springer, Heidelberg (1995)
Mu’alem, A.W., Feitelson, D.G.: Utilization, Predictability, Workloads, and User Runtime Estimate in Scheduling the IBM SP2 with Backfilling. IEEE Transactions on Parallel and Distributed Systems 12(6), 529–543 (2001)
Downey, A.B.: A Model for Speedup of Parallel Programs. UC Berkeley EECS Technical Report, No. UCB/CSD-97-933 (January 1997)
Downey, A.B.: A Parallel Workload Model and Its Implications for Processor Allocation. In: The 6th International Symposium on High Performance Distributed Computing (1997)
Srinivasan, S., Krishnamoorthy, S., Sadayappan, P.: A Robust Scheduling Strategy for Moldable Scheduling of Parallel Jobs. In: 5th IEEE International Conference on Cluster Computing, pp. 92–99 (2003)
Srinivasan, S., Subramani, V., Kettimuthu, R., Holenarsipur, P., Sadayappan, P.: Effective Selection of Partition Sizes for Moldable Scheduling of Parallel Jobs. In: Sahni, S.K., Prasanna, V.K., Shukla, U. (eds.) HiPC 2002. LNCS, vol. 2552, pp. 174–183. Springer, Heidelberg (2002)
Sun, H., Cao, Y., Hsu, W.J.: Efficient Adaptive Scheduling of Multiprocessors with Stable Parallelism Feedback. IEEE Transactions on Parallel and Distributed System 22(4) (April 2011)
Huang, K.C.: Performance Evaluation of Adaptive Processor Allocation Policies for Moldable Parallel Batch Jobs. In: 3th Workshop on Grid Technologies and Applications (2006)
Parallel Workloads Archive, http://www.cs.huji.ac.il/labs/parallel/workload/
Feitelson, D.G.: A Survey of Scheduling in Multiprogrammed Parallel Systems, Research Report RC 19790 (87657), IBM T. J. Watson Research Center (October 1994)
Feitelson, D.G., Rudolph, L., Schweigelshohn, U., Sevcik, K., Wong, P.: Theory and Practice in Parallel Job Scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1997 and JSSPP 1997. LNCS, vol. 1291, pp. 1–34. Springer, Heidelberg (1997)
Kleinrock, L., Huang, J.H.: On parallel processing systems: Amdahl’s law generalized and some results on optimal design. IEEE Trans. Softw. Eng. 18(5) (1992)
Huang, K.C., Huang, T.C., Tung, T.H., Shih, P.Z.: Effective Processor Allocation for Moldable Jobs with Application Speedup Model. In: Proceedings of the International Computer Symposium, ICS 2012, Taiwan (2012)
The Message Passing Interface (MPI) standard, http://www.mcs.anl.gov/research/projects/mpi/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Huang, KC., Huang, TC., Tsai, MJ., Chang, HY. (2014). Moldable Job Scheduling for HPC as a Service. In: Park, J., Stojmenovic, I., Choi, M., Xhafa, F. (eds) Future Information Technology. Lecture Notes in Electrical Engineering, vol 276. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40861-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-40861-8_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40860-1
Online ISBN: 978-3-642-40861-8
eBook Packages: EngineeringEngineering (R0)