Abstract
With the rapid development of cloud environment, the capabilities of systems have been promoted with powerful computing and storage. But for the characteristic of “pay-as-you-go” of cloud resources, it is necessary to consider the different data storage cost. Especially for processing of “old data” in long-term storage, an appropriate strategy is needed to reduce users’ cost. Considering the characteristics of price stratification in the current commercial cloud environment, a three-level price stratified storage strategy is proposed based on the CTT-SP algorithm, which stores part of the “old data” on relatively inexpensive secondary and tertiary storage, and ensures that the time delay caused by three-level storage does not exceed the deadline. Compared with other storage methods, the experimental result shows the strategy proposed can guarantee the time delay while reducing the cost of users significantly in long-term storage.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Deelman, E., et al.: Pegasus: mapping scientific workflows onto the grid. In: Grid Computing, Second European Across Grids Conference, Axgrids, Nicosia, Cyprus, January, Revised Papers (2004)
Ludäscher, B., et al.: Scientific workflow management and the Kepler system: research articles. Concurr. Comput.: Pract. Exp. 18, 1039–1065 (2006)
Oinn, T., et al.: Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics 20, 3045–3054 (2004)
Li, X., et al.: A novel workflow-level data placement strategy for data-sharing scientific cloud workflows. IEEE Trans. Serv. Comput. (1939)
Ikken, S., Renault, E., Barkat, A., Kechadi, M.T., Tari, A.: Efficient intermediate data placement in federated cloud data centers storage. In: Boumerdassi, S., Renault, É., Bouzefrane, S. (eds.) MSPN 2016. LNCS, vol. 10026, pp. 1–15. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-50463-6_1
Zhao, Q., Xiong, C., Zhao, X., Yu, C., Xiao, J.: A data placement strategy for data-intensive scientific workflows in cloud. In: IEEE/ACM International Symposium on Cluster (2015)
Xu, R., Zhao, K., Zhang, P., Dong, Y., Yun, Y.: A novel data set importance based cost-effective and computation-efficient storage strategy in the cloud. In: IEEE International Conference on Web Services (2017)
Dong, Y., Cui, L., Li, W., Xiao, L., Yun, Y.: An algorithm for finding the minimum cost of storing and regenerating datasets in multiple clouds. IEEE Trans. Cloud Comput. (2016)
Dong, Y., Yun, Y., Xiao, L., Chen, J.: A cost-effective strategy for intermediate data storage in scientific cloud workflows, pp. 1–12 (2010)
Dong, Y., Yun, Y., Xiao, L., Chen, J.: On-demand minimum cost benchmarking for intermediate dataset storage in scientific cloud workflow systems. J. Parallel Distrib. Comput. 71, 316–332 (2011)
Lei, F., Sha, M., Liu, X., Liang, Y.: Improved CTT-SP algorithm with critical path method for massive data storage in scientific workflow systems. Int. J. Pattern Recognit. Artif. Intell. 30 (2016)
Lei, F., Sha, M., Liang, Y., Liu, X.: Experimental analysis on CTT-SP algorithm for intermediate data storage in scientific workflow systems. In: International Conference on Computational Intelligence & Security (2016)
Amazon EC2 Pricing. https://amazonaws-china.com/cn/s3/pricing/. Accessed 1 Mar 2019
Baidu Cloud Pricing. https://cloud.baidu.com/product/bos.html. Accessed 16 Feb 2019
Tencent Cloud Pricing. https://cloud.tencent.com/product/cos. Accessed 16 Feb 2019
Acknowledgment
This work is supported by Anhui Natural Science Foundation 1908085MF206 and National Natural Science Foundation of China (NO. 61402007, 61573022), the Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Lv, Z., Zhang, C., Wang, F. (2019). Cost Optimization Strategy for Long-Term Storage of Scientific Workflow. In: Cheng, X., Jing, W., Song, X., Lu, Z. (eds) Data Science. ICPCSEE 2019. Communications in Computer and Information Science, vol 1058. Springer, Singapore. https://doi.org/10.1007/978-981-15-0118-0_12
Download citation
DOI: https://doi.org/10.1007/978-981-15-0118-0_12
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-0117-3
Online ISBN: 978-981-15-0118-0
eBook Packages: Computer ScienceComputer Science (R0)