Abstract
Backfilling is a scheduling optimization that requires information about job runtimes to be known. Such information can come from either of two sources: estimates provided by users when the jobs are submitted, or predictions made by the system based on historical data regarding previous executions of jobs. In both cases, each job is assigned a precise prediction of how long it will run. We suggest that instead the whole distribution of the historical data be used. As a result, the whole backfilling framework shifts from a concrete plan for the future schedule to a probabilistic plan where jobs are backfilled based on the probability that they will terminate in time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Lifka, D.: The ANL/IBM SP scheduling system. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1995. LNCS, vol. 949, pp. 295–303. Springer, Heidelberg (1995)
Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans. Parallel and Distributed systems 12(6), 529–543 (2001)
Feitelson, D.G., Mu’alem Weil, A.: Utilization and predictability in scheduling the IBM SP2 with backfilling. In: International Parallel Processing Symposium, Number 12, pp. 542–546 (1998)
Zotkin, D., Keleher, P.J.: Job-length estimation and performance in backfilling schedulers. In: International Symposium on High Performance Distributed Computing, Number 8 (1999)
Chiang, S.H., Arpaci-Dusseau, A., Vernon, M.K.: The impact of more accurate requested runtimes on production job scheduling performance. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2002. LNCS, vol. 2537, pp. 103–127. Springer, Heidelberg (2002)
Tsafrir, D., Feitelson, D.G.: The dynamics of backfilling: solving the mystery of why increased inaccuracy may help. In: IEEE International Symposium on Workload Characterization, pp. 131–141 (2006)
Tsafrir, D., Etsion, Y., Feitelson, D.G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans. Parallel and Distributed systems 18(6), 789–803 (2007)
Gibbons, R.: A historical application profiler for use by parallel schedulers. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1997. LNCS, vol. 1291, pp. 58–77. Springer, Heidelberg (1997)
Smith, W., Foster, I., Taylor, V.: Predicting application run times using historical information. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1998. LNCS, vol. 1459, pp. 122–142. Springer, Heidelberg (1998)
Parallel workloads archive, http://www.cs.huji.ac.il/labs/parallel/workload/
Feitelson, D.G.: Locality of sampling and diversity in parallel system workloads. In: 21st International Conference on Supercomputing, pp. 53–63 (2007)
Nissimov, A.: Locality and its usage in parallel job runtime distribution modeling using HMM. Master’s thesis, The Hebrew University (2006)
Feitelson, D.G., Rudolph, L., Schwiegelshohn, U., Sevcik, K.C., Wong, P.: Theory and practice in parallel job scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1997. LNCS, vol. 1291, pp. 1–34. Springer, Heidelberg (1997)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nissimov, A., Feitelson, D.G. (2008). Probabilistic Backfilling. In: Frachtenberg, E., Schwiegelshohn, U. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2007. Lecture Notes in Computer Science, vol 4942. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78699-3_6
Download citation
DOI: https://doi.org/10.1007/978-3-540-78699-3_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-78698-6
Online ISBN: 978-3-540-78699-3
eBook Packages: Computer ScienceComputer Science (R0)