Abstract
The default model and the dynamic model of data placement based on the need of fault tolerance in HDFS cloud storage cluster system were built. On this basis, the \( Rack_{i} \) selection algorithm based on the weight to process the problems of data transmission was designed and the \( DN_{ik} \) selection algorithm based on the gray evaluation criteria of entropy weight to evaluate each DataNode in the \( Rack_{i} \) comprehensively were proposed, that come to a DataNode with the best Map/Reduce performance eventually. Analysis and experiments showed that the \( Rack_{i} \) selection algorithm and \( DN_{ik} \) selection algorithm can provide the data replicas pre-written into HDFS reasonable DataNodes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Korupolu M, Plaxton G, Rajaraman R (2001) Placement algorithms for hierarchical cooperative caching. In: Proceedings of the 10th annual symposium on discrete algorithms vol 38 No 1 Jan pp 260–302
Rashedur RM, Barker K, Alhajj R (2008) Replica placement strategies in data grid. J Grid Comput 6:103–123
Buyya R (2002) Economic-based distributed resource management and scheduling for grid computer. Monash University, Melbourne
Ranganathan K, Foster I (2001) Identifying dynamic replication strategies for a high- performance data grid. In: Proceeding of the second international workshop on grid computing Denver, Nov pp 75–86
Ranganathan K, Foster I (2001) Design and evaluation of dynamic replication strategies for a high-perfomance data grid. International conference on computing in high energy and nuclear physics, Beijing, Sep
Xiong Fu, Wang R, Song D (2010) Heuristic replica placement algorithm in data grids. Syst Eng Electron , 32(7):1513–1516 (in Chinese)
Julong D (1996) Basic methods of grey system. Huazhong University of Science and Technology Press, Wuhan (in Chinese)
Yannian Z, Zhu Yi’ (2011) The grey correlation evaluation algorithm of comprehensive performance about embedded computer based on the combination weight. J Northwestern Polytechnical Univ 29(1):12–16 (in Chinese)
Yi XQ (2010) Application of entropy weight coefficient method to the evaluation of the profitability of listed firms. 2010 international conference on computer application and system modeling 22–24 pp 108–110
Xiong R, Junzhou L, Song A, Jin J-H (2011) QoS preference-aware replica selection strategy in cloud computing. J Commun 32(7):93–103
Acknowledgments
This work is financially supported by the national natural science foundation of China under Grant NO. 61262016 and the natural science foundation of GanSu province under Grant NO. 1010RJZA050 & 14-0220.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bao, G., Yu, C., Zhao, H., Luan, Y. (2013). Researching on the Placement of Data Replicas in the System of HDFS Cloud Storage Cluster. In: Sun, Z., Deng, Z. (eds) Proceedings of 2013 Chinese Intelligent Automation Conference. Lecture Notes in Electrical Engineering, vol 255. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38460-8_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-38460-8_29
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38459-2
Online ISBN: 978-3-642-38460-8
eBook Packages: EngineeringEngineering (R0)