Skip to main content
SpringerLink
Log in

Don’t Hurry Be Happy: A Deadline-Based Backfilling Approach

  • Conference paper
  • First Online:
Book cover Job Scheduling Strategies for Parallel Processing (JSSPP 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10773))

Included in the following conference series:

Abstract

Computing resources in data centers are usually managed by a Resource and Job Management System whose main objective is to complete submitted jobs as soon as possible while maximizing resource usage and ensuring fairness among users. However, some users might not be as hurried as the job scheduler but only interested in their jobs to complete before a given deadline.

In this paper, we derive from this initial hypothesis a low-complexity scheduling algorithm, called Deadline-Based Backfilling (DBF), that distinguishes regular jobs that have to complete as early as possible from deadline-driven jobs that come with a deadline before when they have to finish. We also investigate a scenario in which deadline-driven jobs are submitted and evaluate the impact of the proposed algorithm on classical performance metrics with regard to state-of-the-art scheduling algorithms. Experiments conducted on four different workloads show that the proposed algorithm significantly reduces the average wait time and average stretch when compared to Conservative Backfilling.

Work partially supported by the MOEBUS ANR project (13-ANR-INFR- 01).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Alea web site: https://github.com/aleasimulator/alea.

  2. 2.

    Batsim web site: https://github.com/oar-team/batsim.

References

  1. Capit, N., Da Costa, G., Georgiou, Y., Huard, G., Martin, C., Mounié, G., Neyron, P., Richard, O.: A batch scheduler with high level components. In: Proceedings of the 5th International Symposium on Cluster Computing and the Grid (CCGrid), Cardiff, UK, May 2005, pp. 776–783 (2005)

    Google Scholar 

  2. Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple Linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003). https://doi.org/10.1007/10968987_3

    Chapter  Google Scholar 

  3. Staples, G.: TORQUE - TORQUE resource manager. In: Proceedings of the ACM/IEEE SC2006 Conference on High Performance Networking and Computing, Tampa, FL, p. 8, November 2006

    Google Scholar 

  4. Boutin, E., Ekanayake, J., Lin, W., Shi, B., Zhou, J., Qian, Z., Wu, M., Zhou, L.: Apollo: scalable and coordinated scheduling for cloud-scale computing. In: Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation, (OSDI), Broomfield, CO, pp. 285–300, October 2014

    Google Scholar 

  5. Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., Wilkes, J.: Large-scale cluster management at Google with Borg. In: Proceedings of the 10th European Conference on Computer Systems (EuroSys), Bordeaux, France, April 2015

    Google Scholar 

  6. Hindman, B., Konwinski, A., Zaharia, M., Ghodsi, A., Joseph, A.D., Katz, R.H., Shenker, S., Stoica, I.: Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI), Boston, MA (2011)

    Google Scholar 

  7. Schwiegelshohn, U., Yahyapour, R.: Analysis of first-come-first-serve parallel job scheduling. In: Proceedings of the 9th Annual ACM-SIAM Symposium on Discrete Algorithms, San Francisco, CA, 629–638, January 1998

    Google Scholar 

  8. Feitelson, D., Tsafrir, D., Krakov, D.: Experience with using the parallel workloads archive. J. Parallel Distrib. Comput. 74(10), 2967–2982 (2014)

    Article  Google Scholar 

  9. Feitelson, D.G., Weil, A.M.: Utilization and predictability in scheduling the IBM SP2 with backfilling. In: Proceedings of the 12th International Parallel Processing Symposium (IPPS), pp. 542–546 (1998)

    Google Scholar 

  10. Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans. Parallel Distrib. Syst. 12(6), 529–543 (2001)

    Article  Google Scholar 

  11. Klusáček, D., Tóth, Š.: On interactions among scheduling policies: finding efficient queue setup using high-resolution simulations. In: Silva, F., Dutra, I., Santos Costa, V. (eds.) Euro-Par 2014. LNCS, vol. 8632, pp. 138–149. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09873-9_12

    Google Scholar 

  12. Lifka, D.A.: The ANL/IBM SP scheduling system. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1995. LNCS, vol. 949, pp. 295–303. Springer, Heidelberg (1995). https://doi.org/10.1007/3-540-60153-8_35

    Chapter  Google Scholar 

  13. Klusáček, D., Rudová, H.: Alea 2 - job scheduling simulator. In: Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques (SIMUTools 2010), Malaga, Spain (2010)

    Google Scholar 

  14. Caniou, Y., Gay, J.-S.: Simbatch: an API for simulating and predicting the performance of parallel resources managed by batch systems. In: César, E., Alexander, M., Streit, A., Träff, J.L., Cérin, C., Knüpfer, A., Kranzlmüller, D., Jha, S. (eds.) Euro-Par 2008. LNCS, vol. 5415, pp. 223–234. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00955-6_27

    Chapter  Google Scholar 

  15. Casanova, H., Giersch, A., Legrand, A., Quinson, M., Suter, F.: Versatile, scalable, and accurate simulation of distributed applications and platforms. J. Parallel Distrib. Comput. 74(10), 2899–2917 (2014)

    Article  Google Scholar 

  16. Dutot, P.-F., Mercier, M., Poquet, M., Richard, O.: Batsim: a realistic language-independent resources and jobs management systems simulator. In: Desai, N., Cirne, W. (eds.) JSSPP 2015-2016. LNCS, vol. 10353, pp. 178–197. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61756-5_10

    Chapter  Google Scholar 

  17. N’takpé, T., Suter, F.: Companion of the don’t hurry be happy: a deadline-based backfilling approach article (2017). https://doi.org/10.6084/m9.figshare.4644466

  18. Liu, C.L., Layland, J.: Scheduling algorithms for multiprogramming in a hard-real-time environment. J. ACM 20(1), 46–61 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  19. Jyothi, S.A., Curino, C., Menache, I., Narayanamurthy, S.M., Tumanov, A., Yaniv, J., Mavlyutov, R., Goiri, I., Krishnan, S., Kulkarni, J., Rao, S.: Morpheus: towards automated SLOs for enterprise clusters. In: Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI), Savannah, GA, pp. 117–134, November 2016

    Google Scholar 

  20. Lucier, B., Menache, I., Naor, J., Yaniv, J.: Efficient online scheduling for deadline-sensitive jobs. In: Proceedings of the 25th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA), Montreal, Canada, pp. 305–314, July 2013

    Google Scholar 

  21. Baraglia, R., Capannini, G., Pasquali, M., Puppin, D., Ricci, L., Techiouba, A.: Backfilling strategies for scheduling streams of jobs on computational farms. In: Danelutto, M., Fragopoulou, P., Getov, V. (eds.) Making Grids Work, pp. 103–115. Springer, Boston (2008). https://doi.org/10.1007/978-0-387-78448-9_8

    Chapter  Google Scholar 

  22. Klusác̆ek, D., Rudová, H.: Performance and fairness for users in parallel job scheduling. In: Cirne, W., Desai, N., Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2012. LNCS, vol. 7698, pp. 235–252. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35867-8_13

    Chapter  Google Scholar 

  23. Klusàček, D., Chlumský, V.: Planning and metaheuristic optimization in production job scheduler. In: Proceedings of the 20th Workshop on Job Scheduling Strategies for Parallel Processing, Chicago, IL, May 2016. https://doi.org/10.1007/978-3-319-61756-5_11

  24. Lindsay, A., Galloway-Carson, M., Johnson, C., Bunde, D., Leung, V.: Backfilling with guarantees made as jobs arrive. Concurr. Computat. Pract. Exp. 25(4), 513–523 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Université Nangui Abrogoua, Abidjan, Côte d’Ivoire

    Tchimou N’takpé

  2. Centre de Calcul de l’IN2P3/CNRS, Lyon, Villeurbanne, France

    Frédéric Suter

  3. Inria, Lyon, France

    Frédéric Suter

Authors
  1. Tchimou N’takpé
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frédéric Suter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frédéric Suter .

Editor information

Editors and Affiliations

  1. CESNET, Prague, Czech Republic

    Dalibor Klusáček

  2. Google, Mountain View, California, USA

    Walfredo Cirne

  3. Google, Seattle, Washington, USA

    Narayan Desai

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

N’takpé, T., Suter, F. (2018). Don’t Hurry Be Happy: A Deadline-Based Backfilling Approach. In: Klusáček, D., Cirne, W., Desai, N. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2017. Lecture Notes in Computer Science(), vol 10773. Springer, Cham. https://doi.org/10.1007/978-3-319-77398-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-77398-8_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-77397-1

  • Online ISBN: 978-3-319-77398-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation