Advertisement

Data-Oriented Scheduling with Dynamic-Clustering Fault-Tolerant Technique for Scientific Workflows in Clouds

  • 23 Accesses

Abstract

Cloud computing is one of the most prominent parallel and distributed computing paradigm. It is used for providing solution to a huge number of scientific and business applications. Large scale scientific applications which are structured as scientific workflows are evaluated through cloud computing. Scientific workflows are data-intensive applications, as a single scientific workflow may consist of hundred thousands of tasks. Task failures, deadline constraints, budget constraints and improper management of tasks can also instigate inconvenience. Therefore, provision of fault-tolerant techniques with data-oriented scheduling is an important approach for execution of scientific workflows in Cloud computing. Accordingly, we have presented enhanced data-oriented scheduling with Dynamic-clustering fault-tolerant technique (EDS-DC) for execution of scientific workflows in cloud computing. We have presented data-oriented scheduling as a proposed scheduling technique. We have also equipped EDS-DC with Dynamic-clustering fault-tolerant technique. To know the effectiveness of EDS-DC, we compared its results with three well-known enhanced heuristic scheduling policies referred to as: (a) MCT-DC, (b) Max-min-DC, and (c) Min-min-DC. We considered scientific workflow of CyberShake as a case study, because it contains most of the characteristics of scientific workflows such as integration, disintegration, parallelism, and pipelining. The results show that EDS-DC reduced make-span of 10.9% as compared to MCT-DC, 13.7% as compared to Max-min-DC, and 6.4% as compared to Min-min-DC scheduling policies. Similarly, EDS-DC reduced the cost of 4% as compared to MCT-DC, 5.6% as compared to Max-min-DC, and 1.5% as compared to Min-min-DC scheduling policies. These results in respect of make-span and cost are highly significant for EDS-DC as compared with above referred three scheduling policies. The SLA is not violated for EDS-DC in respect of time and cost constraints, while it is violated number of times for MCT-DC, Max-min-DC, and Min-min-DC scheduling techniques.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

REFERENCES

  1. 1

    Shi, J., Luo, J., Dong, F., Zhang, J., and Zhang, J., Elastic resource provisioning for scientific workflow scheduling in cloud under budget and deadline constraints, Cluster Comput., 2016, vol. 19, no. 1, pp. 167–182.

  2. 2

    Sun, D., Chang, G., Miao, C., and Wang, X., “Analyzing, modeling and evaluating dynamic adaptive fault tolerance strategies in cloud computing environments, J. Supercomput., 2013, vol. 66, no. 1, pp. 193–228.

  3. 3

    Lifka, D., et al., XSEDE Cloud Survey Report, Urbana, IL: Natl. Center Supercomput. Appl., 2013.

  4. 4

    Li, X., Song, J., and Huang, B., A scientific workflow management system architecture and its scheduling based on cloud service platform for manufacturing big data analytics, Int. J. Adv. Manuf. Technol., 2016, vol. 84, nos. 1–4, pp. 119–131.

  5. 5

    Abbott, B.P., et al., LIGO: the Laser Interferometer Gravitational-Wave Observatory, Rep. Prog. Phys., 2009 vol. 72, no. 7, p. 76901.

  6. 6

    Bharathi, S., Deelman, E., Mehta, G., Vahi, K., Chervenak, A., and Su, M., Characterization of scientific workflows, Proc. 3rd Workshop on Workflows in Support of Large Scale Science, Austin, TX, 2008.

  7. 7

    Callaghan, S., et al., Metrics for heterogeneous scientific workflows : a case study of an earthquake science application, Int. J. High Perform. Comput. Appl., 2011, vol. 25, no. 3, pp. 274-285.

  8. 8

    Callaghan, S., et al., Reducing time-to-solution using distributed high-throughput mega-workflows—experiences from SCEC CyberShake, Proc. 4th Int. Conf. on e-Science, e-Science, December 7–12, 2008, Indianapolis, 2008, pp. 151–158.

  9. 9

    Abrishami, S., Naghibzadeh, M., and Epema, D.H.J., Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds, Future Gener. Comput. Syst., 2013, vol. 29, no. 1, pp. 158–169.

  10. 10

    Chakraborty, D., Mankar, V.V., and Nanavati, A.A., Enabling runtime adaptation of workflows to external events in enterprise environments, Proc. IEEE Int. Conf. on Web Service ICWS 2007, Salt Lake City, 2007, pp. 1112–1119.

  11. 11

    Deelman, E., Singh, G., Livny, M., Berriman, B., and Good, J., “The cost of doing science on the cloud: the montage example, Proc. Int. Conf. High Performance Computing Networking, Storage and Analysis SC 2008, Austin, TX, 2008.

  12. 12

    Serrano, D., et al., “SLA guarantees for cloud services, Future Gener. Comput. Syst., 2016, vol. 54, pp. 233–246.

  13. 13

    Choi, J., Adufu, T., and Kim, Y., Data-locality aware scientific workflow scheduling methods in HPC cloud environments, Int. J. Parallel Program., 2017, vol. 45, no. 5, pp. 1128–1141.

  14. 14

    Tang, W., et al., Data-aware resource scheduling for multicloud workflows: a fine-grained simulation approach, Proc. Int. Conf. on Cloud Computer Technologies and Science CloudCom, Marrakesh, 2015, pp. 887–892.

  15. 15

    Chen, W. and Deelman, E., “Fault tolerant clustering in scientific workflows, Proc. 8th IEEE World Congr. on Service, Honolulu, 2012, pp. 9–16.

  16. 16

    Harshitha, S.B., Kaneria, P., and Manjaiah, D.H., Comparative study of workflow scheduling algorithms in cloud computing, Int. J. Innovation Res. Comput. Commun. Eng., 2014, special issue 2, pp. 31–37.

  17. 17

    Mathew, T., et al., Study and analysis of various task scheduling algorithms in the cloud computing environment, Proc. Int. Conf. on Advances in Computing, Communications and Informatics (ICACCI), September 24–27, 2014, New Delhi, 2014, pp. 658–664.

  18. 18

    Chen, W. and Deelman, E., WorkflowSim: A toolkit for simulating scientific workflows in distributed environments, Proc. 8th IEEE Int. Conf. E-Science, Chicago, 2012.

  19. 19

    Kosar, T. and Balman, M., A new paradigm: data-aware scheduling in grid computing, Future Gener. Comput. Syst., 2009, vol. 25, no. 4, pp. 406–413.

  20. 20

    Zeng, L., Veeravalli, B., and Zomaya, A.Y., An integrated task computation and data management scheduling strategy for workflow applications in cloud environments, J. Network Comput. Appl., 2015, vol. 50, pp. 39–48.

  21. 21

    Poola, D., Ramamohanarao, K., and Buyya, R., Fault-tolerant workflow scheduling using spot instances on clouds, Procedia Comput. Sci., 2014, vol. 29, pp. 523–533.

  22. 22

    Kumar, D., Baranwal, G., Raza, Z., and Vidyarthi, D.P., A systematic study of double auction mechanisms in cloud computing, J. Syst. Software, 2017, vol. 125, pp. 234–255.

  23. 23

    Malawski, M., Juve, G., Deelman, E., and Nabrzyski, J., Algorithms for cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds, Future Gener. Comput. Syst., 2015, vol. 48, pp. 1–18.

  24. 24

    He, X., Sun, X., and Laszewski, G., A QoS guided scheduling algorithm for grid computing, Office, 2002, vol. 18, no. 4, pp. 1–15.

  25. 25

    Madureira, A.M. and Definitions, A.B., Ordered minimum completion time heuristic for unrelated parallel-machines problems, Proc. 9th Iberian Conf. on Information Systems and Technologies (CISTI), Barcelona, 2014.

  26. 26

    Priyadarsini, R.J., Performance evaluation of min-min and max-min algorithms for job scheduling in federated cloud, Int. J. Comput. Appl., 2014, vol. 99, no. 18, pp. 47–54.

  27. 27

    Qureshi, K., Khan, F.G., Manuel, P., and Nazir, B., A hybrid fault tolerance technique in grid computing system, J. Supercomput., 2011, vol. 56, no. 1, pp. 106–128.

  28. 28

    Bala, A. and Chana, I., Fault tolerance-challenges, techniques and implementation in cloud computing, Int. J. Comput. Sci., 2012, vol. 9, no. 1, pp. 288–293.

  29. 29

    Deelman, E., et al., “Pegasus, a workflow management system for science automation, Future Gener. Comput. Syst., 2015, vol. 46, pp. 17–35.

  30. 30

    Tolosana-Calasanz, R., Bañares, J.Á., Pham, C., and Rana, O.F., “Enforcing QoS in scientific workflow systems enacted over Cloud infrastructures, J. Comput. Syst. Sci., 2012, vol. 78, no. 5, pp. 1300–1315.

  31. 31

    Chen, W., Ferreira, R., Deelman, E., and Sakellariou, R., Balanced task clustering in scientific workflows, Proc. 9th IEEE Int. Conf. on e-Science, Beijing, 2013, pp. 1–8.

  32. 32

    Antonescu, A.F. and Braun, T., Simulation of SLA-based VM-scaling algorithms for cloud-distributed applications, Future Gener. Comput. Syst., 2016, vol. 54, pp. 260–273.

  33. 33

    Mustafa, S., Nazir, B., Hayat, A., ur Rehman Khan, A., and Madani, S.A., “Resource management in cloud computing: taxonomy, prospects, and challenges, Comput. Electron. Eng., 2015, vol. 47, pp. 186–203.

  34. 34

    Barladian, B.Kh., et al., An efficient mulithreading algorithm for the simulation of global illumination, Program. Comput. Software, 2017, vol. 43, no. 4, pp. 217–223.

  35. 35

    Fursova, N.I., et al., A lightweight method for virtual machine introspection, Program. Comput. Software, 2017, vol. 43, no. 5, pp. 307–313.

  36. 36

    Gusev, A.D., Nasonov, A.V., and Krylov, A.S., Fast parallel grid warping-based image sharpening method, Program. Comput. Software, 2017, vol. 43, no. 4, pp. 230–233.

  37. 37

    Kuplyakov, D., Shalnov, E., and Konushin, A., Markov chain Monte Carlo based video tracking algorithm, Program. Comput. Software, 2017, vol. 43, no. 4, pp. 224–229.

  38. 38

    Massobrio, R., et al., Towards a cloud computing paradigm for big data analysis in smart cities, Program. Comput. Software, 2018, vol. 44, no. 3, pp. 181–189.

  39. 39

    Muruganantham, R. and Ganeshkumar, P., Quality of service enhancement in wireless sensor network using flower pollination algorithm, Program. Comput. Software, 2018, vol. 44, no. 6, pp. 398–406.

  40. 40

    Raja, R. and Ganeshkumar, P., QoSTRP: a trusted clustering based routing protocol for mobile ad-hoc networks, Program. Comput. Software, 2018, vol. 44, no. 6, pp. 407–416.

  41. 41

    Pashchenko, N.F., Zipa, K.S., and Ignatenko, A.V., An algorithm for the visualization of stereo images simultaneously captured with different exposures, Program. Comput. Software, 2017, vol. 43, no. 4, pp. 250–257.

  42. 42

    Varnovskiy, N.P., et al., Secure cloud computing based on threshold homomorphic encryption, Program. Comput. Software, 2015, vol. 41, no. 4, pp. 215–218.

  43. 43

    Zelenova, S.A., and Zelenov, S.V., Schedulability analysis for strictly periodic tasks in RTOS, Program. Comput. Software, 2018, vol. 44, no. 3, pp. 159–169.

  44. 44

    Zipa, K.S. and Ignatenko, A.V., Algorithms for the analysis and visualization of high dynamic range images based on human perception, Program. Comput. Software, 2016, vol. 42, no. 6, pp. 367–374.

Download references

Author information

Correspondence to Z. Ahmad or A. I. Jehangiri or M. Iftikhar or A. I. Umer or I. Afzal.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ahmad, Z., Jehangiri, A.I., Iftikhar, M. et al. Data-Oriented Scheduling with Dynamic-Clustering Fault-Tolerant Technique for Scientific Workflows in Clouds. Program Comput Soft 45, 506–516 (2019). https://doi.org/10.1134/S0361768819080097

Download citation