Abstract
Nowadays, different end devices with different computation and bandwidth capabilities acquire data from Internet. To improve efficiency of data storage and retrieve, in this paper, we study how to use random linear network coding to construct an efficient distributed storage system to reduce the traffic cost in a dynamic network. In order to balance the success ratio of recovery traffic cost and traffic speed, we firstly introduce a random network coding scheme and implement a practically available distributed storage system in the actual environment. We then adjust different parameters, e.g., finite fields, link bandwidth, node computing capabilities, etc., to evaluate the proposed system. Finally, experiment results show the efficiency of the proposed designs.
S. Zhang—This work was supported in part by the National Natural Science Foundation of China (No.61672370, 61572310), Natural Science Foundation of the Higher Education Institutions of Jiangsu Province (No. 16KJB520040), Suzhou Key Laboratory of Converged Communication (No. SZS0805), Prospective Application Foundation Research of Suzhou of China (No. SYG201730), Six Talent Peak high-level personnel selection and training foundation of Jiangsu of China (No. 2014-WLW-010), Shanghai Key Laboratory of Intelligent Information Processing, Fudan University (No. IIPL-2016-008) and Postgraduate Research & Practice Innovation Program of Jiangsu Province (No. SJCX17_0661).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Buyya, R., Cortes, T., Jin, H.: A Case for Redundant Arrays of Inexpensive Disks (RAID), vol. 17, no. 3, pp. 109–116. ACM (1988)
Ahlswede, R., Cai, N., Li, S.Y.R., Yeung, R.W.: Network information flow. Proc. IEEE Trans. Inf. Theory 46(4), 1204–1216 (2000)
Dimakis, A.G., Godfrey, P.B., Wainwright, M.J., Ramchandran, K.: Network coding for distributed storage systems. In: Proceedings of IEEE International Conference on Computer Communications (INFOCOM), vol. 56, pp. 2000–2008 (2007)
Dimakis, A.G., Ramchandran, K., Wu, Y., Suh, C.: A survey on network codes for distributed storage. Proc. IEEE 99(3), 476–489 (2011)
Liu, F., Shen, S., Li, B., Li, B., Jin, H.: Cinematic-quality vod in a p2p storage cloud: design, implementation and measurements. IEEE J. Sel. Areas Commun. (JSAC) 31(9), 214–226 (2013)
Zhang, Y., Dragga, C., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H.: Viewbox: integrating local file systems with cloud storage services. In: Proceedings of File and Storage Technologies, vol. 49, pp. 119–132 (2014)
Weil, S.A., Brandt, S.A., Miller, E.L., Long, D.D., Maltzahn, C.: Ceph: a scalable, high-performance distributed file system. In: Proceedings of Operating Systems Design and Implementation, pp. 307–320 (2006)
Hu, W., Yang, T., Matthews, J.N.: The good, the bad and the ugly of consumer cloud storage. ACM SIGOPS Oper. Syst. Rev. 44(3), 110–115 (2010)
Muthitacharoen, A., Chen, B.: A low-bandwidth network file system. In: Proceedings of Symposium on Operating Systems Principles, vol. 35, no. 5, pp. 174–187 (2001)
Bowers, K.D., Juels, A., Oprea, A.: A high-availability and integrity layer for cloud storage. In: Proceedings of Computer and Communications Security (CCS), pp. 187–198 (2009)
Shvachko, K., Kuang, H., Radia, S., Chansler, R., Shvachko, K.: The hadoop distributed file system mass storage systems and technologies. In: Proceedings of Mass Storage Systems and Technology (MSST) (2010)
Fitzek, F.H.P., Toth, T., Szabados, A., Pedersen, M.V.: Implementation and performance evaluation of distributed cloud storage solutions using random linear network coding. In: IEEE International Conference on Proceedings of Communications Workshops (ICC), pp. 249–254. IEEE (2014)
Sipos, M., Fitzek, F.H.P., Lucani, D.E., Pedersen, M.V.: Distributed cloud storage using network coding. In: Proceedings of Consumer Communications and Networking Conference (CCNC), pp. 127–132 (2014)
Dimakis, A.G., Godfrey, P.B., Wainwright, M.J., Ramchandran, K.: Network coding for distributed storage systems. IEEE Trans. Inf. Theory 56(9), 4539–4551 (2010)
Chen, H.C.H., Hu, Y., Lee, P.P.C., Tang, Y.: NCCloud: a network-coding-based storage system in a cloud-of-clouds. IEEE Trans. Comput. 63(1), 31–44 (2014)
Shwe, H.Y., Chong, P.H.J.: Scalable distributed cloud data storage service for internet of things. In: Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People, and Smart World Congress, pp. 869–873 (2016)
Wang, J., Wang, J., Lu, K., Xiao, B., Gu, N.: Optimal linear network coding design for secure unicast with multiple streams. In: Proceedings of Information Communications (INFOCOM), pp. 2240–2248 (2010)
Wang, J., Wang, J., Lu, K., Xiao, B., Gu, N.: Modeling and optimal design of linear network coding for secure unicast with multiple streams. IEEE Trans. Parallel Distrib. Syst. 24(10), 2025–2035 (2013)
Wang, J., Wang, X., Zhang, S., Zhu, Y., Jia, J.: An efficient reliable communication scheme in wireless sensor networks using linear network coding. Int. J. Distrib. Sensor Netw. pp. 487–513 (2012)
Chuang, C., Yung, K.: Network dynamics, network resources and innovation. In: Proceedings of Academy of Management Annual Meeting (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
He, B., Wang, J., Zhou, J., Lu, K., Li, L., Zhang, S. (2018). The Design and Implementation of Random Linear Network Coding Based Distributed Storage System in Dynamic Networks. In: Vaidya, J., Li, J. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2018. Lecture Notes in Computer Science(), vol 11337. Springer, Cham. https://doi.org/10.1007/978-3-030-05063-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-05063-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05062-7
Online ISBN: 978-3-030-05063-4
eBook Packages: Computer ScienceComputer Science (R0)