Abstract
Workflows can orchestrate multiple applications that need resources to execute. The cloud computing has emerged as an on-demand resource provisioning paradigm, which can support workflow execution. In recent years, Amazon offers a new service option, i.e., EC2 spot instances, whose price is on average more than 75 % lower than the one of on-demand instances. Therefore, we can make use of spot instances to execute workflows in a cost-efficient way. However, the spot instances is cut off when their price increases and exceeds the customer’s bid, which will make the task failed and the execution time becomes unpredictable. We propose a deadline driven scheduling (DDS) algorithm which is able to use both on-demand and spot instances to reduce the cost while the deadline of workflows can also be guaranteed with a high probability. Especially, we use an attribute, called global weight, to represent the interdependency relations of tasks and schedule the tasks whose interdependent tasks need longer time first to reduce the whole execution time. The experimental results demonstrate that DDS algorithm is effective in reducing cost while satisfying the deadline constraints of workflows.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Case Studies of Amazon AWS service. http://aws.amazon.com/solutions/case-studies
Yu, J., Buyya, R., Tham, C.K.: A cost-based scheduling of scientific workflow applications on utility grids. In: Proceedings of the 1st IEEE International Conference on e-Science and Grid Computing, vol. 8 p. 147 (2005)
Duan, R., Prodan, R., Fahringer, T.: Performance and cost optimization for multiple large-scale grid workflow applications. In: SC 2007 Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, 2007, p. 12. IEEE (2007)
Mao, M., Humphrey, M.: Auto-Scaling to minimize cost and meet application deadlines in cloud workflows. In: Proceedings of International Conference on High Performance Computing, Networking, Storage and Analysis (SC), pp. 49:1–49:12 (2011)
Byun, E.K., Kee, Y.S., Kim, J.S., et al.: Cost optimized provisioning of elastic resources for application workflows. Future Gener. Comput. Syst. 27(8), 1011–1026 (2011)
Yi, S., Andrzejak, A., Kondo, D.: Monetary cost-aware checkpointing and migration on amazon cloud spot instances. IEEE Trans. Serv. Comput. 5, 512–524 (2011)
Voorsluys, W., Buyya, R.: Reliable provisioning of spot instances for compute-intensive applications, pp. 542–549 (2012)
Zhong, H., Tao, K., Zhang, X.: An approach to optimized resource scheduling algorithm for open-source cloud systems. In: Fifth Annual China Grid Conference (2010)
Lin, C., Lu, S.: Scheduling scientific workflows elastically for cloud computing. In: IEEE 4th International Conference on Cloud Computing (2011)
Liu, H., Xu, D., Miao, H.: Ant colony optimization based service flow scheduling with various QoS requirements in cloud computing. In: 2011 First ACIS International Symposium on Software and Network Engineering (SSNE), pp. 53–58. IEEE (2011)
Yu, J., Buyya, R., Tham, C.K.: Cost-based scheduling of scientific workflow application on utility grids. In: First International Conference on e-Science and Grid Computing (2005)
Kllapi, H., Sitaridi, E., Tsangaris, M.M., Ioannidis, Y.: Schedule optimization for data processing flows on the cloud. In: Proceedings of the ACM SIGMOD International Conference on Management of Data (2011)
Wang, H., Jing, Q., Chen, R., He, B., Qian, Z., Zhou, L.: Distributed systems meet economics: pricing in the cloud. In: Proceedings of the Second USENIX Conference on Hot Topics in Cloud Computing (HotCloud), pp. 6–6 (2010)
Herodotou, H., Babu, S.: Profiling, what-if analysis, and cost-based optimization of mapreduce programs. Proc. VLDB Endowment 4(11), 1111–1122 (2011)
Zhou, A.C., He, B., Liu, C.: Probabilistic scheduling of scientific workflows in dynamic cloud environments. In: CoRR (2013)
Juve, G., Chervenak, A., Deelman, E., et al.: Characterizing and profiling scientific workflows. Future Gener. Comput. Syst. 29(3), 682–692 (2013)
Garey M.R., Johnson, D.S.: Computers and intractability: a guide to the theory of NP-completeness. W.H. Freeman & Co., (2003)
Kwok, Y.K., Ahmad, I.: Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Comput. Surv. 31(4), 406–471 (1999)
Sakellariou, R., Zhao, H.: A hybrid heuristic for DAG scheduling on heterogeneous systems. In: The 13th Heterogeneous Computing Workshop (HCW 2004), Santa Fe, New, Mexico, USA, 26 April 2004
Topcuoglu, H., Hariri, S., Wu, M.-Y.: Task scheduling algorithms for heterogeneous processors. In: 8th Proceedings of Heterogeneous Computing Workshop (1999)
Javadi, B., Thulasiramy, R.K., Buyya, R.: Statistical modeling of spot instance prices in public cloud environments. In: 2011 Fourth IEEE International Conference on Utility and Cloud Computing, pp. 219–228. IEEE Computer Society (2011)
Acknowledgements
This work is partially supported by China National Science Foundation (Granted Number 61272438, 61472253), Research Funds of Science and Technology Commission of Shanghai Municipality (Granted Number 15411952502, 12511502704).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Ma, Z., Cao, J., Qian, S. (2015). DDS: A Deadline Driven Workflow Scheduling Algorithm for Hybrid Amazon Instances. In: Yao, L., Xie, X., Zhang, Q., Yang, L., Zomaya, A., Jin, H. (eds) Advances in Services Computing. APSCC 2015. Lecture Notes in Computer Science(), vol 9464. Springer, Cham. https://doi.org/10.1007/978-3-319-26979-5_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-26979-5_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26978-8
Online ISBN: 978-3-319-26979-5
eBook Packages: Computer ScienceComputer Science (R0)