Abstract
Cloud computing has emerged as a new approach to bring computing as a service, in both academia and industry. One of the challenging issues is scientific workflow execution, where the job scheduling problem becomes more complex, especially when communication processes are taken into account. To provide good performance, many algorithms have been designed for distributed environments. However, these algorithms are not adapted to the uncertain and dynamic nature of cloud computing. In this paper, we present a general view on scheduling problems in cloud computing with communication, and compare existed solutions based on three models of cloud applications named CU-DAG, EB-DAG and CA-DAG. We formulate the problem and review several workflow scheduling algorithms. We discuss the main difficulties of using existed application models in the domain of computations on clouds. Finally, we show that our CA-DAG approach, based on separate vertices for computing and communications, and introducing communication awareness, allows us to mitigate uncertainty in a more efficient way.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Robison, S.: HP Shane Robison Executive Viewpoint: The Next Wave: Everything as a Service. http://www.hp.com/hpinfo/execteam/articles/robison. Accessed 30 January 2014
CSC: CSC cloud usage index latest report, Computer Sciences Corporation. http://www.csc.com/au/ds/39454/75790-csc_cloud_usage_index_latest_report. Accessed 20 January 2014
Zhang, Q., Cheng, L., Boutaba, R.: Cloud computing: state-of-the-art and research challenges. J. Internet Serv. Appl. 1(1), 7–18 (2010)
N. US Department of Commerce, Final Version of NIST Cloud Computing Definition Published. http://www.nist.gov/itl/csd/cloud-102511.cfm. Accessed 20 January 2014
Hollinsworth, D.: The workflow reference model. In: Workflow Management Coalition, vol. TC00–1003 (1995)
Pandey, S., Wu, L., Guru, S.M., Buyya, R.: A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments. In: 2010 24th IEEE International Conference on Advanced Information Networking and Applications (AINA), pp. 400–407 (2010)
Kandula, S., Sengupta, S., Greenberg, A., Patel, P., Chaiken, R.: The nature of data center traffic: measurements & analysis. In: Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement Conference, New York, NY, USA, pp. 202–208 (2009)
AbdelBaky, M., Parashar, M., Kim, H., Jordan, K.E., Sachdeva, V., Sexton, J., Jamjoom, H., Shae, Z.Y., Pencheva, G., Tavakoli, T., Wheeler, M.F.: Enabling high-performance computing as a service. Computer 45(10), 72–80 (2012)
Tchernykh, A., Schwiegelsohn, U., Alexandrov, V., Talbi, E.: Towards understanding uncertainty in cloud computing resource provisioning. SPU 2015 - solving problems with uncertainties (3rd Workshop). In: Conjunction with the 15th International Conference on Computational Science (ICCS 2015), ReykjavÃk, Iceland, 1–3 June 2015. Procedia Computer Science, Elsevier, vol. 51, pp. 1772–1781 (2015)
Tychinsky A.: Innovation Management of Companies: Modern Approaches, Algorithms, Experience. Taganrog Institute of Technology, Taganrog (2006). http://www.aup.ru/books/m87/
Kliazovich, D., Pecero, J., Tchernykh, A., Bouvry, P., Khan, S., Zomaya, A.: CA-DAG: modeling communication-aware applications for scheduling in cloud computing. J. Grid Comput., 1–17 (2015). Springer, Netherlands
Tsafrir, D., Etsion, Y., Feitelson, D.G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans. Parallel Distrib. Syst. 18(6), 789–803 (2007)
RamÃrez-Alcaraz, J.M., Tchernykh, A., Yahyapour, R., Schwiegelshohn, U., Quezada-Pina, A., González-GarcÃa, J.L., Hirales-Carbajal, A.: Job allocation strategies with user run time estimates for online scheduling in hierarchical Grids. J. Grid Comput. 9(1), 95–116 (2011)
Zitzler, E.: Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications, vol. 63. Shaker, Ithaca (1999)
Kliazovich, D., Bouvry, P., Khan, S.U.: DENS: data center energy-efficient network-aware scheduling. Cluster Comput. 16(1), 65–75 (2013)
Liu, K., Jin, H., Chen, J., Liu, X., Yuan, D., Yang, Y.: A compromised-time-cost scheduling algorithm in swindew-c for instance-intensive cost-constrained workflows on a cloud computing platform. Int. J. High Perform. Comput. Appl. 24(4), 445–456 (2010)
Jin, J., Luo, J., Song, A., Dong, F., Xiong, R.: BAR: an efficient data locality driven task scheduling algorithm for cloud computing. In: 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2011), pp. 295–304 (2011)
Sonnek, J., Greensky, J., Reutiman, R., Chandra, A.: Starling: minimizing communication overhead in virtualized computing platforms using decentralized affinity-aware migration. In: 39th International Conference on Parallel Processing (ICPP 2010), pp. 228–237 (2010)
Pecero, J.E., Trystram, D., Zomaya, A.Y.: A new genetic algorithm for scheduling for large communication delays. In: Sips, H., Epema, D., Lin, H.-X. (eds.) Euro-Par 2009. LNCS, vol. 5704, pp. 241–252. Springer, Heidelberg (2009)
Stage, A., Setzer, T.: Network-aware migration control and scheduling of differentiated virtual machine workloads. In: Proceedings of the 2009 ICSE Workshop on Software Engineering Challenges of Cloud Computing, pp. 9–14. IEEE Computer Society (2009)
Sinnen, O., Sousa, L.A.: Communication contention in task scheduling. IEEE Trans. Parallel Distrib. Syst. 16(6), 503–515 (2005)
Volckaert, B., Thysebaert, P., De Leenheer, M., De Turck, F., Dhoedt, B., Demeester, P.: Network aware scheduling in grids. In: Proceedings of the 9th European Conference on Networks and Optical Communifications, p. 9 (2004)
Malawski, M., Juve, G., Deelman, E., Nabrzyski, J.: Cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, p. 22 (2012)
Yu, J., Buyya, R.: A taxonomy of workflow management systems for grid computing. J. Grid Comput. 3(3–4), 171–200 (2005)
Tchernykh, A., Pecero, J., Barrondo, A., Schaeffer, E.: Adaptive energy efficient scheduling in peer-to-peer desktop grids. Future Gener. Comput. Systems 36, 209–220 (2014)
Tchernykh, A., Lozano, L., Schwiegelshohn, U., Bouvry, P., Pecero, J.E., Nesmachnow, S., Drozdov, A.Y.: Online bi-objective scheduling for IaaS clouds with ensuring quality of service. J. Grid Comput., 1–18 (2015). Springer
Carbajal, A.H., Tchernykh, A., Yahyapour, R., Röblitz, T., RamÃrez-Alcaraz, J.M., González-GarcÃa, J.L.: Multiple workflow scheduling strategies with user run time estimates on a grid. J. Grid Comput. 10(2), 325–346 (2012). Springer-Verlag, New York, USA
Quezada, A., Tchernykh, A., González, J., Hirales, A., RamÃrez, J.-M., Schwiegelshohn, U., Yahyapour, R., Miranda, V.: Adaptive parallel job scheduling with resource admissible allocation on two level hierarchical grids. Future Gener. Comput. Syst. 28(7), 965–976 (2012)
Rodriguez, A., Tchernykh, A., Ecker, K.: Algorithms for dynamic scheduling of unit execution time tasks. Eur. J. Oper. Res. 146(2), 403–416 (2003). Elsevier Science, North-Holland
Kianpisheh, S., Jalili, S., Charkari, N.M.: Predicting job wait time in grid environment by applying machine learning methods on historical information. Int. J. Grid Distrib. Comput. 5(3) (2012)
Iverson, M.A., Ozguner, F.; Follen, G.J.: Run-time statistical estimation of task execution times for heterogeneous distributed computing. In: Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing, 1996, pp. 263–270 (1996)
Ramirez-Velarde, R.V., Rodriguez-Dagnino, R.M.: From commodity computers to high-performance environments: scalability analysis using self-similarity, large deviations and heavy-tails. Concurrency Comput. Pract. Exp. 22, 1494–1515 (2010)
Acknowledgment
This work is partially supported by CONACYT (Consejo Nacional de Ciencia y TecnologÃa, México), grant no. 178415. The work of D. Dzmitry Kliazovich is partly funded by National Research Fund, Luxembourg in the framework of ECO-CLOUD (C12/IS/3977641) project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Miranda, V., Tchernykh, A., Kliazovich, D. (2016). Dynamic Communication-Aware Scheduling with Uncertainty of Workflow Applications in Clouds. In: Gitler, I., Klapp, J. (eds) High Performance Computer Applications. ISUM 2015. Communications in Computer and Information Science, vol 595. Springer, Cham. https://doi.org/10.1007/978-3-319-32243-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-32243-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-32242-1
Online ISBN: 978-3-319-32243-8
eBook Packages: Computer ScienceComputer Science (R0)