Abstract
Cloud service providers and enterprises usually deploy high performance storage to manage the unrelenting growth of data. In this paper, we focus on performance optimization and evaluation by using optimal regeneration codes in such cloud storage systems. We present an efficient data maintenance management framework to reduce network repair traffic with the minimum data movement while keeping the desired fault-tolerance in storage systems. In the management framework, it has two phases including the traditional erasure coding process and the optimal placement process. We formally represent the optimal placement as a variant of the bin packing problem by bipartite graphs. Then, we model the placement transform by the interchange graph and propose an efficient heuristic algorithm to find the optimal solution. All feasible solutions are linked together by interchange operation and thus the search space can be taken as an interchange graph. Finally, we evaluate the performance of the optimal placement during data maintenance with different practical settings in our experiments. The experimental results show that the amount of network repair traffic can be reduced by about \(10\,\%\) than the initial placement and by about 2X than traditional erasure coding placement.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
IDC. The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things. http://www.emc.com/leadership/digital-universe/2014iview/executive-summary.htm
Amazon S3. http://aws.amazon.com/s3/
OpenStack Storage. http://www.openstack.org/software/openstack-storage/
Khan, O., Burns, R., Plank, J., et al.: Rethinking erasure codes for cloud file systems: miniminzing I/O for recovery and degraded reads. In: Proceedings of the FAST, February 2012
Huang, C., Simitci, H., Xu, Y., Ogus, A., Calder, B., Gopalan, P., Li, J., Yekhanin, S.: Erasure coding in windows azure storage. In: Proceedings of the 2012 USENIX Conference on Annual Technical Conference, USENIX ATC12, Berkeley, CA, USA (2012)
Rashmi, K., Shah, N.B., Kumar, P.V., Ramchandran, K.: Explicit construction of optimal exact regenerating codes for distributed storage. In: Proceedings of the 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1243–1249 (2009)
El Rouayheb, S., Ramchandran, K.: Fractional repetition codes for repair in distributed storage systems. In: Proceedings of the 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1510–1517 (2010)
Dimakis, A.G., Godfrey, P., Wainwright, M., Ramachandran, K.: Network coding for distributed storage system. IEEE Trans. Inform. Theory 56(9), 4539–4551 (2010)
Dimakis, A.G., Ramchandran, K., Wu, Y., Suh, C.: A survey on network codes for distributed storage. In: Proceedings of the IEEE, pp. 476–489 (2011)
Koo, J.C., Gill III, J.T.: Scalable constructions of fractional repetition codes in distributed storage systems. In: Proceedings of the 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1366–1373 (2011)
Olmez, O., Ramamoorthy, A.: Repairable replication-based storage systems using resolvable designs. In: Proceedings of the 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1174–1181 (2012)
Silberstein, N., Etzion, T.: Optimal Fractional Repetition Codes, January 2014. arXiv:1401.4734
Goparaju, S., El Rouayheb, S., Calderbank, R.: New Codes and Inner Bounds for Exact Repair in Distributed Storage Systems, February 2014. arXiv:1402.2343
Ryser, H.J.: Combinatorial properties of matrices of zeros and ones. Canad. J. Math. 9, 371–377 (1957)
Brualdi, R.A.: Matrices of zeros and ones with fixed row and column sum vectors. Linear Algebra Appl. 33, 159–231 (1980)
Chen, R., Huang, H., Xiao, G.: Relation between parity-check matrixes and cycles of associated tanner graphs. IEEE Commun. Lett. 11(8), 674–676 (2007)
Anil, S., Gupta, M.K., Gulliver, T.A.: Enumerating some fractional repetition codes, March 2013. arXiv:1303.6801
Toni, E.: The Existence of Fractional Repetition Codes (2012). arXiv: 1201.3547
Acknowledgments
This work was supported in part the National Science Foundation of China Projects under Grant 61201234, 61202381, 61202168, 61572357 and 61170301, and also Tianjin Science Foundation Project under Grant 13JCQNJC00400.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Xu, G., Li, H., Mao, Q., Lin, S., Zhang, H. (2015). Performance Optimization and Evaluation of Space Management in Cloud Storage Systems. In: Wang, G., Zomaya, A., Martinez, G., Li, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2015. Lecture Notes in Computer Science(), vol 9531. Springer, Cham. https://doi.org/10.1007/978-3-319-27140-8_42
Download citation
DOI: https://doi.org/10.1007/978-3-319-27140-8_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27139-2
Online ISBN: 978-3-319-27140-8
eBook Packages: Computer ScienceComputer Science (R0)