Skip to main content
SpringerLink
Log in
Book cover

Intelligent Decision Systems in Large-Scale Distributed Environments pp 25–47Cite as

Efficient Hierarchical Task Scheduling on GRIDS Accounting for Computation and Communications

  • Chapter

  • 617 Accesses

Part of the book series: Studies in Computational Intelligence ((SCI,volume 362))

Abstract

This chapter proposes a novel Grid-based scheduling algorithm that optimizes both computation and communications costs of workflow applications. Based on a hierarchical two-steps optimization process, a super scheduler first applies a Recursive Convex Clustering Algorithm (RCCA) that efficiently clusters tasks while minimizing communication costs. In the second step, a resource-broker assigns the generated convex sets to resources clusters. Local schedulers then optimize the makespan for the group of tasks assigned to their cluster, using a graphic processing unit (GPU)-based parallel cellular Genetic Algorithm(cGA). The performance improvement brought by this novel two-step scheduling algorithm compared to a hierarchical list-scheduling approach is empirically demonstrated on different real-world workflow applications.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alba, E., Dorronsoro, B.: Cellular Genetic Algorithms. In: Operations Research/Compuer Science Interfaces. Springer, Heidelberg (2008)

    Google Scholar 

  2. Alba, E., Tomassini, M.: Parallelism and Evolutionary Algorithms. IEEE Transactions on Evolutionary Computation 6, 443–462 (2002)

    Article  Google Scholar 

  3. Bampis, E., Giroudeau, R., König, J.C.: An approximation algorithm for the precedence constrained scheduling problem with hierarchical communications. Theor. Comput. Sci. 290(3), 1883–1895 (2003)

    Article  Google Scholar 

  4. Blachot, F., Huard, G., Pecero, J.E., Saule, E., Trystram, D.: Scheduling instructions on hierarchical machines. In: IEEE IPDPS-PDSEC 2010, USA (2010), doi:10.1109/IPDPSW.2010.5470711

    Google Scholar 

  5. Bolze, R., Cappello, F., Caron, E., Daydé, M., Desprez, F., Jeannot, E., Jégou, Y., Lanteri, S., Leduc, J., Melab, N., Mornet, G., Namyst, R., Primet, P., Quetier, B., Richard, O., Talbi, E.-G., Irena, T.: Scheduling on large scale distributed platforms: from models to implementations. Int. J. Found. Comput. Sci. 16(2), 217–237 (2005)

    Article  Google Scholar 

  6. Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G.B., Good, J., Laity, A., Jacob, J.C., Katz, D.S.: Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems. Scientific Programming Journal 13(3), 219–237 (2005)

    Google Scholar 

  7. Dong, F., Akl, S.: An Adaptive Double-layer Workflow Scheduling Approach for Grid Computing. In: Proc. of the High Performance Computing Symposium (HPCS) 2007, Canada (2007)

    Google Scholar 

  8. Dorronsoro, B., Bouvry, P., Cañero, J.A., Maciejewski, A.A., Siegel, H.J.: Multi-objective robust static mapping of independent tasks on grids. In: International Conference on Evolutionary Computation (CEC), part of the IEEE World Congress on Computational Intelligence (WCCI), pp. 3389–3396 (2010)

    Google Scholar 

  9. Dutot, P.-F., Eyraud, L., Mounié, G., Trystram, D.: Scheduling on large scale distributed platforms: from models to implementations. Int. J. Found. Comput. Sci. 16(2), 217–237 (2005)

    Article  Google Scholar 

  10. Dutot, P.-F., N’Takpé, T., Suter, F., Casanova, H.: Scheduling Parallel Task Graphs on (Almost) Homogeneous Multi-cluster Platforms. IEEE Trans on Parallel and Distributed Systems 20(7), 940–952 (2009)

    Article  Google Scholar 

  11. El-Rewini, H., Ali, H., Lewis, T.: Task Scheduling in Parallel and Distributed Systems. PTR Prentice Hall, Englewood Cliffs (1994)

    Google Scholar 

  12. Garg, S., Buyya, R., Siegel, H.J.: Time and cost trade-off management for scheduling parallel applications on utility grids. Future Generation Computer Systems 26(8), 1344–1355 (2010)

    Article  Google Scholar 

  13. Gauja, B., Huard, G., Pecero, J., Thierry, E., Trystram, D.: Convex Scheduling for Grid Computing. In: WASC 2004 - 1st Workshop on Algorithms for Scheduling and Communication, Bertinoro, Italy (2004)

    Google Scholar 

  14. Grid5000 (2009), http://www.grid5000.org

  15. Guzek, M., Pecero, J., Dorronsoro, B., Bouvry, P.: A cellular genetic algorithm for scheduling applications and energy-aware communication optimization. In: Workshop on Optimization Issues in Energy Efficient Distributed Systems (OPTIM), part of the International Conference on High Performance Computing & Simulation (HPCS), Caen, France, pp. 241–248 (2010)

    Google Scholar 

  16. He, L., Jarvis, S.A., Spooner, D.P., Bacigalupo, D., Tan, G., Nudd, G.R.: Mapping DAG-based applications to multiclusters with background workload. In: IEEE International Symposium on Cluster Computing and the Grid, vol. 2, pp. 855–862 (2005), doi:10.1109/CCGRID.2005.1558651

    Google Scholar 

  17. Hwang, J.J., Chow, Y.C., Angers, F.D., Lee, C.Y.: Scheduling precedence graphs in systems with interprocessor communication times. SIAM Journal on Computing 18(2), 244–257 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  18. Krauter, K., Buyya, R., Maheswaran, M.: A taxonomy and survey of Grid resource management systems for distributed computing. Int. J. of Software: Practice and Experience 32(2), 135–164 (2002)

    Article  MATH  Google Scholar 

  19. Lee, Y.C., Subrata, R., Zomaya, A.Y.: On the performance of a dual-objective optimization model for workflow applications on Grid platforms. IEEE Trans on Parallel and Distributed Systems 20(9), 1273–1284 (2009)

    Article  Google Scholar 

  20. Lepére, R., Trystram, D.: A new clustering algorithm for scheduling with large communication delays. In: 16th IEEE-ACM annual International Symposium on Parallel and Distributed Processing (IPDPS 2002), USA (2002)

    Google Scholar 

  21. Mahjoub, A., Pecero, J.E., Trystram, D.: Scheduling with uncertainties on new computing platforms. Journal Comput Optim Appl. (2010)

    Google Scholar 

  22. Manderick, B., Spiessens, P.R.: Fine-grained parallel genetic algorithm. In: Schaffer, J. (ed.) Third International Conference on Genetic Algorithms (ICGA), pp. 428–433. Morgan Kaufmann, San Francisco (1989)

    Google Scholar 

  23. Martino, B.D., Dongarra, J., Hoisie, A., Yang, L.T., Zima, H.: Engineering the Grid: Status and Perspective. American Scientific Publishers (2006)

    Google Scholar 

  24. Nasri, W., Steffenel, L.A., Trystram, D.: Adaptive approaches for efficient parallel algorithms on cluster-based systems. International Journal of Grid and Utility Computing (IJGUC) 1(2), 98–108 (2009)

    Article  Google Scholar 

  25. Pecero, J.E., Bouvry, P.: An improved genetic algorithm for efficient scheduling on distributed memory parallel systems. In: IEEE/ACS International Conference on Computer Systems and Applications, AICCSA 2010 (2010), doi:10.1109/AICCSA.2010.5587030

    Google Scholar 

  26. Pecero, J.E., Trystram, D., Zomaya, A.Y.: A new genetic algorithm for scheduling for large communication delays. In: Sips, H., Epema, D., Lin, H.-X. (eds.) Euro-Par 2009. LNCS, vol. 5704, pp. 241–252. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  27. Radulescu, A., van Gemund, A.J.C.: Flb: Fast load balancing for distributed-memory machines. In: Proc. Int. Conf. on Parallel Processing (1999)

    Google Scholar 

  28. Radulescu, A., van Gemund, A.J.C.: Fast and effective task scheduling in heterogeneous systems. In: Proc. 9th Heterogeneous Computing Workshop, HCW (2000)

    Google Scholar 

  29. Sánchez, J.E.P., Trystram, D.: A new genetic convex clustering algorithm for parallel time minimization with large communication delays. In: Joubert, G.R., Nagel, W.E., Peters, F.J., Plata, O., Tirado, P., Zapata, E. (eds.) Parallel Computing: Current & Future Issues of High-End Computing, vol. 33, pp. 709–716. John von Newmann (2006)

    Google Scholar 

  30. Shivle, S., Siegel, H.J., Maciejewski, A.A., Sugavanam, P., Banka, T., Castain, R., Chindam, K., Dussinger, S., Pichumani, P., Satyasekaran, P., Saylor, W., Sendek, D., Sousa, J., Sridharan, J., Velazco, J.: Static allocation of resources to communicating subtasks in a heterogeneous ad hoc grid environment. Journal of Parallel and Distributed Computing, Special Issue on Algorithms for Wireless and Ad-hoc Networks 66(4), 600–611 (2006)

    MATH  Google Scholar 

  31. Tchernykh, A., Schwiegelson, U., Yahyapour, R., Kuzjurin, N.: On-line hierarchical job scheduling on grids with admisible allocation. J. Sched. 13(5), 545–552 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  32. Topcuoglu, H., Hariri, S., Wu, M.: Performance-Effective and Low- Complexity Task Scheduling for Heterogeneous Computing. IEEE Trans. Parallel and Distributed Systems 13(3), 260–274 (2002)

    Article  Google Scholar 

  33. Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bulletin 1(6), 80–83 (1945)

    Article  Google Scholar 

  34. Whitley, D.: Cellular genetic algorithms. In: Forrest, S. (ed.) Fifth International Conference on Genetic Algorithms (ICGA), p. 658. Morgan Kaufmann, California (1993)

    Google Scholar 

  35. Yu, J., Buyya, R.: A taxonomy of workflow management systems for grid computing. Journal of Grid Computing 3(3-4), 171–200 (2006), doi:10.1007/s10723-005-9010-8

    Article  Google Scholar 

  36. Yu, J., Buyya, R., Ramamohanarao, K.: Workflow scheduling algorithms for Grid computing. In: Xhafa, F., Abraham, A. (eds.) Metaheuristics for Scheduling in Distributed Computing Environments. Studies in Computational Intelligence, vol. 146, pp. 173–214. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  37. Zomaya, A.Y., Chan, G.: Efficient clustering for parallel tasks execution in distributed systems. In: Proc. of Workshop NIDISC 2004, New Mexico, USA, pp. 167–177 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Communications Research Unit, University of Luxembourg, 6 rue R. Coudenhove-Kalergi, L-1359, Luxembourg

    Johnatan E. Pecero, Frederic Pinel, Grégoire Danoy & Pascal Bouvry

  2. Interdisciplinary Centre for Security, Reliability, and Trust, University of Luxembourg, 6 rue R. Coudenhove-Kalergi, L-1359, Luxembourg

    Bernabé Dorronsoro

  3. Advanced Network Research Group, School of Information Technologies, University of Sydney, Sydney, NSW, 2006, Australia

    Albert Y. Zomaya

Authors
  1. Johnatan E. Pecero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frederic Pinel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernabé Dorronsoro
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Grégoire Danoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pascal Bouvry
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Albert Y. Zomaya
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computer Science and Communications Research Unit, University of Luxembourg, 6 rue R. Coudenhove-Kalergi, L-1359, Luxembourg

    Pascal Bouvry

  2. School of Computing and IDEAS Research Institute, Robert Gordon University, AB25 1HG, Aberdeen, Scotland, United Kingdom

    Horacio González-Vélez

  3. Department of Mathematics and Computer Science, University of Bielsko-Biala, ul.Willowa 2, 43-309, Bielsko-Biala, Poland

    Joanna Kołodziej

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Pecero, J.E., Pinel, F., Dorronsoro, B., Danoy, G., Bouvry, P., Zomaya, A.Y. (2011). Efficient Hierarchical Task Scheduling on GRIDS Accounting for Computation and Communications. In: Bouvry, P., González-Vélez, H., Kołodziej, J. (eds) Intelligent Decision Systems in Large-Scale Distributed Environments. Studies in Computational Intelligence, vol 362. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21271-0_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-21271-0_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-21270-3

  • Online ISBN: 978-3-642-21271-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation