The Journal of Supercomputing

, Volume 75, Issue 12, pp 7857–7871 | Cite as

An experimental study of scheduling algorithms for many-task applications

  • Oleg SukhoroslovEmail author
  • Alexey Nazarenko
  • Roman Aleksandrov


The paper studies the performance of algorithms for scheduling of many-task applications in distributed computing systems. Two important classes of such applications are considered: bags-of-tasks and workflows. The comparison of algorithms is performed on the basis of discrete-event simulation for various application cases and system configurations. The developed simulation framework based on SimGrid toolkit provides the necessary tools for implementation of scheduling algorithms, generation of synthetic systems and applications, execution of simulation experiments and analysis of results. This allowed to perform a large number of experiments in a reasonable amount of time and to ensure reproducible results. The conducted experiments demonstrate the dependence of the performance of studied algorithms on various application and system characteristics. While confirming the performance advantage of advanced static algorithms, the presented results reveal some interesting insights. In particular, the accuracy of the used network model helped to demonstrate the limitations of simple analytical models for scheduling of data-intensive parallel applications with static algorithms.


Distributed computing Scheduling Many-task applications Bag-of-tasks Workflow DAG Discrete-event simulation 


  1. 1.
    Arabnejad H, Barbosa JG (2014) List scheduling algorithm for heterogeneous systems by an optimistic cost table. IEEE Trans Parallel Distrib Syst 25(3):682–694CrossRefGoogle Scholar
  2. 2.
    Arabnejad H, Barbosa JG, Prodan R (2016) Low-time complexity budget-deadline constrained workflow scheduling on heterogeneous resources. Future Gener Comput Syst 55:29–40CrossRefGoogle Scholar
  3. 3.
    Armstrong R, Hensgen D, Kidd T (1998) The relative performance of various mapping algorithms is independent of sizable variances in run-time predictions. In: 1998 Seventh Heterogeneous Computing Workshop. (HCW 98) Proceedings. IEEE, pp 79–87Google Scholar
  4. 4.
    Bharathi S, Chervenak A, Deelman E, Mehta G, Su MH, Vahi K (2008) Characterization of scientific workflows. In: 2008 Third Workshop on Workflows in Support of Large-Scale Science, pp 1–10Google Scholar
  5. 5.
    Bittencourt LF, Sakellariou R, Madeira ERM (2010) Dag scheduling using a lookahead variant of the heterogeneous earliest finish time algorithm. In: 2010 18th Euromicro Conference on Parallel, Distributed and Network-Based Processing, pp 27–34.
  6. 6.
    Casanova H, Giersch A, Legrand A, Quinson M, Suter F (2014) Versatile, scalable, and accurate simulation of distributed applications and platforms. J Parallel Distrib Comput 74(10):2899–2917CrossRefGoogle Scholar
  7. 7.
    Chen W, Deelman E (2012) Workflowsim: a toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th International Conference on E-science (e-science). IEEE, pp 1–8Google Scholar
  8. 8.
    Freund RF, Gherrity M, Ambrosius S, Campbell M, Halderman M, Hensgen D, Keith E, Kidd T, Kussow M, Lima JD et al (1998) Scheduling resources in multi-user, heterogeneous, computing environments with smartnet. In: 1998 Seventh Heterogeneous Computing Workshop. (HCW 98) Proceedings. IEEE, pp 184–199Google Scholar
  9. 9.
    Graham RL, Lawler EL, Lenstra JK, Kan AR (1979) Optimization and approximation in deterministic sequencing and scheduling: a survey. Ann Discrete Math 5:287–326MathSciNetCrossRefGoogle Scholar
  10. 10.
    Hunold S, Rauber T, Suter F (2008) Scheduling dynamic workflows onto clusters of clusters using postponing. In: 8th IEEE International Symposium on Cluster Computing and the Grid. CCGRID’08. IEEE, pp 669–674Google Scholar
  11. 11.
    Maheswaran M, Ali S, Siegal HJ, Hensgen D, Freund RF (1999) Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems. In: Eighth Heterogeneous Computing Workshop. (HCW’99) Proceedings. IEEE, pp 30–44Google Scholar
  12. 12.
    Nazarenko A, Sukhoroslov O (2017) An experimental study of workflow scheduling algorithms for heterogeneous systems. In: Malyshkin V (ed) Parallel computing technologies. Springer, Cham, pp 327–341CrossRefGoogle Scholar
  13. 13.
    Raicu I, Foster IT, Zhao Y (2008) Many-task computing for grids and supercomputers. In: Workshop on Many-Task Computing on Grids and Supercomputers. MTAGS 2008. IEEE, pp 1–11Google Scholar
  14. 14.
    Sih GC, Lee EA (1993) A compile-time scheduling heuristic for interconnection-constrained heterogeneous processor architectures. IEEE Trans Parallel Distrib Syst 4(2):175–187CrossRefGoogle Scholar
  15. 15.
    Taylor IJ, Deelman E, Gannon DB, Shields M (2014) Workflows for e-Science: scientific workflows for grids. Springer, IncorporatedGoogle Scholar
  16. 16.
    Tobita T, Kasahara H (2002) A standard task graph set for fair evaluation of multiprocessor scheduling algorithms. J Sched 5(5):379–394MathSciNetCrossRefGoogle Scholar
  17. 17.
    Topcuoglu H, Hariri S, Wu MY (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274. CrossRefGoogle Scholar
  18. 18.
    Velho P, Legrand A (2009) Accuracy study and improvement of network simulation in the simgrid framework. In: Proceedings of the 2nd International Conference on Simulation Tools and Techniques, p 13. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering)Google Scholar
  19. 19.
    Velho P, Schnorr LM, Casanova H, Legrand A (2013) On the validity of flow-level tcp network models for grid and cloud simulations. ACM Trans Model Comput Simul (TOMACS) 23(4):23MathSciNetCrossRefGoogle Scholar
  20. 20.
    Yu J, Buyya R, Ramamohanarao K (2008) Workflow scheduling algorithms for grid computing. In: Xhafa F, Abraham A (eds) Metaheuristics for scheduling in distributed computing environments. Studies in computational intelligence, vol 146. Springer, Berlin, Heidelberg, pp 173–214Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute for Information Transmission Problems of the Russian Academy of SciencesMoscowRussia
  2. 2.National Research University Higher School of EconomicsMoscowRussia

Personalised recommendations