Abstract
Cloud environment provides high performance computing services to process massive data for data-intensive workflows. Due to the different functional requirements, tasks in a workflow might be allocated to multiple cloud servers. The massive data among these tasks have to be transferred and this greatly increases the execution cost. To decrease the transferred data size during the workflow execution, this paper proposes a dynamic task allocation method based on the data dependencies. The workflow with data dependencies and typical control logic, i.e., sequential, parallel, and exclusive choice, is described based on process algebra. The data size relevant to a data dependency can be obtained only after the task is executed. Each task is allocated to a certain server according to relevant data size and maximal data paths. A case study is presented to illustrate the feasibility and effect of the proposed method and the related work is discussed based on the case study.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Rimal, B.P., Choi, E.: A service-oriented taxonomical spectrum, cloudy challenges and opportunities of cloud computing. Int. J. Commun Syst 25(6), 796–819 (2012)
Diaz-Montes, J., Diaz-Granados, M., Zou, M., Tao, S., Parashar, M.: Supporting data-intensive workflows in software-defined federated multi-clouds. IEEE Trans. Cloud Comput. 6(1), 250–263 (2018)
Alkhanaka, E.N., Leea, S.P., Rezaeia, R., Parizi, R.M.: Cost optimization approaches for scientific workflow scheduling in cloud and grid computing: a review, classifications, and open issues. J. Syst. Softw. 113(3), 1–26 (2016)
Lenhard, J., Ferme, V., Harrer, S., Geiger, M., Pautasso, C.: Lessons learned from evaluating workflow management systems. In: Braubach, L., et al. (eds.) ICSOC 2017. LNCS, vol. 10797, pp. 215–227. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91764-1_17
Masdari, M., ValiKardan, S., Shahi, Z., Azar, S.I.: Towards workflow scheduling in cloud computing: a comprehensive analysis. J. Netw. Comput. Appl. 66, 64–82 (2016)
Moghadam, M.H., Babamir, S.M., Mirabi, M.: A multi-objective optimization model for data-intensive workflow scheduling in data grids. In: IEEE 41st Conference on Local Computer Networks Workshops, pp. 25–33 (2016)
Kumar, M.S., Gupta, I., Jana, P.K.: Forward load aware scheduling for data-intensive workflow applications in cloud system. In: International Conference on Information Technology, pp. 93–97 (2016)
Rodriguez, M.A., Buyya, R.: Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. IEEE Trans. Cloud Comput. 2(2), 222–235 (2014)
Choi, J., Adufu, T., Kim, Y.: Data-locality aware scientific workflow scheduling methods in HPC cloud environments. Int. J. Parallel Prog. 45(5), 1128–1141 (2017)
Smanchat, S., Viriyapant, K.: Taxonomies of workflow scheduling problem and techniques in the cloud. Future Gener. Comput. Syst. 52, 1–12 (2015)
Gupta, M., Jain, A.: A survey on cost aware task allocation algorithm for cloud environment. In. 4th IEEE International Conference on Signal Processing, Computing and Control, pp. 642–646 (2017)
Yuan, D., Yang, Y., Liu, X., Zhang, G., Chen, J.: A data dependency based strategy for intermediate data storage in scientific cloud workflow systems. Concurr. Comput. Pract. Exp. 24(9), 956–976 (2012)
Bilgaiyan, S., Sagnika, S., Das M.: Workflow scheduling in cloud computing environment using cat swarm optimization. In: IEEE International Advance Computing Conference (IACC), pp. 680–685 (2014)
Xie, Y., Chen, S., Ni, Q., Hanqing, W.: Integration of resource allocation and task assignment for optimizing the cost and maximum throughput of business processes. J. Intell. Manuf. (2017). https://doi.org/10.1007/s10845-017-1329-z
Guerfel, R., Sbaï, Z., Ayed, R.B.: Model checking of cost-effective elasticity strategies in cloud computing. In: Braubach, L., et al. (eds.) ICSOC 2017. LNCS, vol. 10797, pp. 80–92. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91764-1_7
Baeten, J.C.M., Middelburg, C.A.: Process Algebra with Timing. Springer, New York (2002). https://doi.org/10.1007/978-3-662-04995-2
Bousselmi, K., Brahmi, Z., Gammoudi, M.M.: QoS-aware scheduling of workflows in cloud computing environments. In: IEEE 30th International Conference on Advanced Information Networking and Applications, pp. 737–745 (2016)
Mishra, S.K., Puthal, D., Sahoo1, B., Jena, S.K., Obaidat, M.S.: An adaptive task allocation technique for green cloud computing. J. Supercomput. 74(1), 370–385 (2018)
Bessai, K., Youcef, S., Oulamara, A., Godart, C., Nurcan, S.: Bi-criteria workflow tasks allocation and scheduling in cloud computing environments. In: IEEE Fifth International Conference on Cloud Computing, pp. 638–645 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, X., Zheng, L., Junyu, C., Shang, L. (2019). Dynamic Task Allocation for Data-Intensive Workflows in Cloud Environment. In: Liu, X., et al. Service-Oriented Computing – ICSOC 2018 Workshops. ICSOC 2018. Lecture Notes in Computer Science(), vol 11434. Springer, Cham. https://doi.org/10.1007/978-3-030-17642-6_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-17642-6_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-17641-9
Online ISBN: 978-3-030-17642-6
eBook Packages: Computer ScienceComputer Science (R0)