Advertisement

Efficient Intermediate Data Placement in Federated Cloud Data Centers Storage

  • Sonia IkkenEmail author
  • Eric Renault
  • Amine Barkat
  • M. Tahar Kechadi
  • Abdelkamel Tari
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10026)

Abstract

The goal of cloud federation strategies is to define a mechanism for resources sharing among federation collaborators. Those mechanisms must be fair to guaranty the common benefits of all the federation members. This paper focuses on intermediate data allocation cost in federated cloud storage. Through a federation mechanism, we propose a mixed integer linear programming model (MILP) to assist multiple data centers hosting intermediate data generated from a scientific community. Under the constraints of the problem, an exact algorithm is proposed to minimize intermediate data allocation cost over the federated data centers storage, taking into account scientific users requirements, intermediate data dependency and data size. Experimental results show the cost-efficiency and scalability of the proposed federated cloud storage model.

Keywords

Big data workflow system Intermediate data Storage federation MILP 

References

  1. 1.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  2. 2.
    Google Cloud Platform. https://cloud.google.com/bigtable/
  3. 3.
  4. 4.
    Apache Hadoop Core. http://hadoop.apache.org/core
  5. 5.
  6. 6.
    Amazon Web Services. https://aws.amazon.com/fr/
  7. 7.
    Kyriazis, D. (ed.): Data Intensive Storage Services for Cloud Environments. IGI Global, Hershey (2013)Google Scholar
  8. 8.
    Yuan, D., Yang, Y., Liu, X., Zhang, G., Chen, J.: A data dependency based strategy for intermediate data storage in scientific cloud workflow systems. Concurrency Comput. Pract. Experience 24(9), 956–976 (2012)CrossRefGoogle Scholar
  9. 9.
    Yuan, D., Yang, Y., Liu, X., Chen, J.: A data placement strategy in scientific cloud workflows. Future Gener. Comput. Syst. 26(8), 1200–1214 (2010)CrossRefGoogle Scholar
  10. 10.
    Zhao, Q., Xiong, C., Zhao, X., Yu, C., Xiao, J.: A data placement strategy for data-intensive scientific workflows in cloud. In: 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 928–934. IEEE, May 2015Google Scholar
  11. 11.
    Ruiz-Alvarez, A., Humphrey, M.: A model and decision procedure for data storage in cloud computing. In: 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 572–579. IEEE, May 2012Google Scholar
  12. 12.
    Agarwala, S., Jadav, D., Bathen, L.A.: iCostale: adaptive cost optimization for storage clouds. In: IEEE International Conference on Cloud Computing (CLOUD), pp. 436–443. IEEE (2011)Google Scholar
  13. 13.
    Negru, C., Pop, F., Cristea, V.: Cost optimization for data storage in public clouds: a user perspective. In: Proceedings of 13th International Conference on Informatics in Economy (2014)Google Scholar
  14. 14.
    Toosi, A.N., Calheiros, R.N., Thulasiram, R.K., Buyya, R.: Resource provisioning policies to increase IaaS provider’s profit in a federated cloud environment. In: 2011 IEEE 13th International Conference on High Performance Computing and Communications (HPCC), pp. 279–287. IEEE, September 2011Google Scholar
  15. 15.
    Tarification Amazon S3. https://aws.amazon.com/fr/s3/pricing/
  16. 16.
  17. 17.
    B2 Cloud Storage Tarification. https://www.backblaze.com/b2/cloud-storage.html
  18. 18.
    Google Cloud Storage Pricing. https://cloud.google.com/storage/pricing
  19. 19.

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Sonia Ikken
    • 1
    • 2
    Email author
  • Eric Renault
    • 1
  • Amine Barkat
    • 2
    • 3
  • M. Tahar Kechadi
    • 4
  • Abdelkamel Tari
    • 2
  1. 1.Telecom SudParisÉvryFrance
  2. 2.Faculty of Exact SciencesUniversity of BejaiaBejaïaAlgeria
  3. 3.Politecnico di MilanoMilanoItaly
  4. 4.UCD School of Computer Science and InformaticsBelfieldIreland

Personalised recommendations