Skip to main content
SpringerLink
Log in

Batsim: A Realistic Language-Independent Resources and Jobs Management Systems Simulator

  • Conference paper
  • First Online:
Job Scheduling Strategies for Parallel Processing (JSSPP 2015, JSSPP 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10353))

Abstract

As large scale computation systems are growing to exascale, Resources and Jobs Management Systems (RJMS) need to evolve to manage this scale modification. However, their study is problematic since they are critical production systems, where experimenting is extremely costly due to downtime and energy costs. Meanwhile, many scheduling algorithms emerging from theoretical studies have not been transferred to production tools for lack of realistic experimental validation. To tackle these problems we propose Batsim, an extendable, language-independent and scalable RJMS simulator. It allows researchers and engineers to test and compare any scheduling algorithm, using a simple event-based communication interface, which allows different levels of realism. In this paper we show that Batsim’s behaviour matches the one of the real RJMS OAR. Our evaluation process was made with reproducibility in mind and all the experiment material is freely available.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    NPB 3.3.1 available here [25].

References

  1. Balouek, D., et al.: Adding virtualization capabilities to the grid’5000 testbed. In: Ivanov, I.I., Sinderen, M., Leymann, F., Shan, T. (eds.) CLOSER 2012. CCIS, vol. 367, pp. 3–20. Springer, Cham (2013). doi:10.1007/978-3-319-04519-1_1

    Chapter  Google Scholar 

  2. Barcelona Supercomputing Center: Extrae, February 2016. https://www.bsc.es/computer-sciences/extrae

  3. Bedaride, P., Degomme, A., Genaud, S., Legrand, A., Markomanolis, G., Quinson, M., Stillwell, M., Suter, F., Videau, B.: Toward better simulation of MPI applications on ethernet/TCP networks, November 2013. https://hal.inria.fr/hal-00919507/document

  4. Bell, W.H., Cameron, D.G., Millar, A.P., Capozza, L., Stockinger, K., Zini, F.: Optorsim: a grid simulator for studying dynamic data replication strategies. Int. J. High Perform. Comput. Appl. 17(4), 403–416 (2003)

    Article  MATH  Google Scholar 

  5. Caniou, Y., Gay, J.-S.: Simbatch: an API for simulating and predicting the performance of parallel resources managed by batch systems. In: César, E., et al. (eds.) Euro-Par 2008. LNCS, vol. 5415, pp. 223–234. Springer, Heidelberg (2009). doi:10.1007/978-3-642-00955-6_27

    Chapter  Google Scholar 

  6. Capit, N., Da Costa, G., Georgiou, Y., Huard, G., Martin, C., Mounié, G., Neyron, P., Richard, O.: A batch scheduler with high level components. In: IEEE International Symposium on Cluster Computing and the Grid, 2005. CCGrid 2005, vol. 2, pp. 776–783. IEEE (2005)

    Google Scholar 

  7. Casanova, H., Giersch, A., Legrand, A., Quinson, M., Suter, F.: Versatile, scalable, and accurate simulation of distributed applications and platforms. J. Parallel Distrib. Comput. 74(10), 2899–2917 (2014). http://hal.inria.fr/hal-01017319

    Article  Google Scholar 

  8. Clauss, P.N., Stillwell, M., Genaud, S., Suter, F., Casanova, H., Quinson, M.: Single node on-line simulation of MPI applications with SMPI, May 2011. https://hal.inria.fr/inria-00527150/document

  9. Diaz, A., Batista, R., Castro, O.: Realtss: a real-time scheduling simulator. In: 4th International Conference on Electrical and Electronics Engineering, 2007. ICEEE 2007, pp. 165–168. IEEE (2007)

    Google Scholar 

  10. Dutot, P.-F., Poquet, M., Trystram, D.: Communication models insights meet simulations. In: Hunold, S., et al. (eds.) Euro-Par 2015. LNCS, vol. 9523, pp. 258–269. Springer, Cham (2015). doi:10.1007/978-3-319-27308-2_22

    Chapter  Google Scholar 

  11. Estrada, T., Flores, D., Taufer, M., Teller, P.J., Kerstens, A., Anderson, D.P., et al.: The effectiveness of threshold-based scheduling policies in BOINC projects. In: Second IEEE International Conference on e-Science and Grid Computing, 2006. e-Science 2006, p. 88. IEEE (2006)

    Google Scholar 

  12. Feitelson, D.G.: Workload Modeling for Computer Systems Performance Evaluation. Cambridge University Press, Cambridge (2015). https://cds.cern.ch/record/2005898

    Book  MATH  Google Scholar 

  13. Grid5000: Nancy: Home - Grid5000, February 2016. https://www.grid5000.fr/mediawiki/index.php/Nancy:Home

  14. Imbert, M., Pouilloux, L., Rouzaud-Cornabas, J., Lébre, A., Hirofuchi, T.: Using the EXECO toolbox to perform automatic and reproducible cloud experiments, December 2013. https://hal.inria.fr/hal-00861886

  15. Inria: InriaForge: Evalys: Projet Home. https://gforge.inria.fr/projects/evalys

  16. Inria: BatSim Homepage, February 2016. http://batsim.gforge.inria.fr/

  17. Inria: InriaForge: Batsimctn: Project Home, February 2016. https://gforge.inria.fr/projects/simctn/

  18. Inria: InriaForge:expe_batsim: Project Home, February 2016. https://gforge.inria.fr/projects/expe-batsim

  19. Inria: Welcome to execo–execo v2.5.3, February 2016. http://execo.gforge.inria.fr/doc/latest-stable/

  20. Jones, W.M., Ligon III, W.B., Pang, L.W., Stanzione, D.: Characterization of bandwidth-aware meta-schedulers for co-allocating jobs across multiple clusters. J. Supercomput. 34(2), 135–163 (2005)

    Article  Google Scholar 

  21. Klusáček, D., Rudová, H.: Alea 2: job scheduling simulator. In: Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques, p. 61. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering) (2010)

    Google Scholar 

  22. Legrand, A.: Simgrid Usages, January 2016. http://simgrid.gforge.inria.fr/Usages.php

  23. Lucarelli, G., Mendonca, F., Trystram, D., Wagner, F.: Contiguity and locality in backfilling scheduling. In: 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 586–595. IEEE (2015)

    Google Scholar 

  24. Mercier, M.: MPI+PRV+TIT-traces_nas-Benchmarks_2016-02-08-10-10-44, February 2016. http://academictorrents.com/details/53b46a4ff43a8ae91f674b26c65c5cc6187f4f8e

  25. NASA: NAS Parallel Benchmarks, February 2016. https://www.nas.nasa.gov/publications/npb.html

  26. Pascual, J.A., Miguel-Alonso, J., Lozano, J.A.: Locality-aware policies to improve job scheduling on 3D tori. J. Supercomput. 71(3), 966–994 (2015)

    Article  Google Scholar 

  27. Ridruejo Perez, F.J., Miguel-Alonso, J.: INSEE: an interconnection network simulation and evaluation environment. In: Cunha, J.C., Medeiros, P.D. (eds.) Euro-Par 2005. LNCS, vol. 3648, pp. 1014–1023. Springer, Heidelberg (2005). doi:10.1007/11549468_111

    Chapter  Google Scholar 

  28. Phatanapherom, S., Uthayopas, P., Kachitvichyanukul, V.: Dynamic scheduling II: fast simulation model for grid scheduling using HyperSim. In: Proceedings of the 35th Conference on Winter Simulation: Driving Innovation, pp. 1494–1500. Winter Simulation Conference (2003)

    Google Scholar 

  29. Proebsting, T., Warren, A.M.: Repeatability and benefaction in computer systems research. Technical report, The university of Arizona (2015). http://reproducibility.cs.arizona.edu/v2/RepeatabilityTR.pdf

  30. Ruiz, C., Harrache, S., Mercier, M., Richard, O.: Reconstructable software appliances with kameleon. SIGOPS Oper. Syst. Rev. 49(1), 80–89 (2015)

    Article  Google Scholar 

  31. Stanisic, L., Legrand, A.: Effective reproducible research with org-mode and Git. In: Lopes, L., et al. (eds.) Euro-Par 2014. LNCS, vol. 8805, pp. 475–486. Springer, Cham (2014). doi:10.1007/978-3-319-14325-5_41

    Google Scholar 

  32. Takefusa, A., Matsuoka, S., Nakada, H., Aida, K., Nagashima, U.: Overview of a performance evaluation system for global computing scheduling algorithms. In: Proceedings of the Eighth International Symposium on High Performance Distributed Computing, pp. 97–104. IEEE (1999)

    Google Scholar 

  33. tcbozzetti: tcbozzetti/trabalhoconclusao, February 2016. https://github.com/tcbozzetti/trabalhoconclusao

  34. oar team: Batsim protocol description (2016), https://github.com/oar-team/batsim/blob/master/doc/proto_description.md

  35. oar team: Kamelot (2016). https://github.com/oar-team/oar3/blob/master/oar/kao/kamelot.py

  36. Xia, H., Dail, H., Casanova, H., Chien, A.: The microgrid: using emulation to predict application performance in diverse grid network environments. In: Proceedings of the Workshop on Challenges of Large Applications in Distributed Environments (2004)

    Google Scholar 

  37. Yu, J., Buyya, R.: A taxonomy of scientific workflow systems for grid computing. SIGMOD Rec. 34(3), 44–49 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Univ. Grenoble Alpes, LIG, 38000, Grenoble, France

    Pierre-François Dutot, Michael Mercier, Millian Poquet & Olivier Richard

  2. Inria, CNRS, LIG, 38000, Grenoble, France

    Pierre-François Dutot, Michael Mercier, Millian Poquet & Olivier Richard

  3. Atos, Bezons, France

    Michael Mercier

Authors
  1. Pierre-François Dutot
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Mercier
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Millian Poquet
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olivier Richard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Millian Poquet .

Editor information

Editors and Affiliations

  1. Google, Seattle, USA

    Narayan Desai

  2. Google, Mountain View, USA

    Walfredo Cirne

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Dutot, PF., Mercier, M., Poquet, M., Richard, O. (2017). Batsim: A Realistic Language-Independent Resources and Jobs Management Systems Simulator. In: Desai, N., Cirne, W. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP JSSPP 2015 2016. Lecture Notes in Computer Science(), vol 10353. Springer, Cham. https://doi.org/10.1007/978-3-319-61756-5_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-61756-5_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-61755-8

  • Online ISBN: 978-3-319-61756-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation