Abstract
A file synchronization program has to exchange lots of metadata information between a client and a server. As data have been growing rapidly in file synchronization systems, the metadata exchanging overhead is more critical now than before. In this paper, we propose a practical approach for minimizing metadata exchange overhead on the file synchronization system using variable-length chunking. The key idea is to minimize metadata by adapting file similarity information and generating only one hash key for several metadata. In the experiment, traditional file synchronization program sends 4.3 Mbyte over 500 MByte data file. However, the proposed algorithm only sends 37 Kbyte metadata.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tridgell, A.: Efficient algorithms for sorting and synchronization. PhD thesis, The Australian National University (1999)
Skovhede, K., Stach, R.: Duplicati. http://www.duplicati.com (2011)
Meunier, P., Nystrom, S., Kamara, S., Yost, S., Alexander, K., Noland, D., Crane, J.: WETICE 2002. In: IEEE International Workshops on Activesync, TCP/IP and 802.11b wireless vulnerabilities of WinCE-based PDAs, pp. 145–150 (2002)
HotSync, P.: Palm developer online documentation (2007)
Starobinski, D., Trachtenberg, A., Agarwal, S.: Efficient PDA synchronization. IEEE Trans. Mobile Comput. 2(1), 40–51 (2003)
Yan, H., Irmak, U., Suel, T.: Algorithms for low-latency remote synchronization. In: INFOCOM 2008. The 27th Conference on Computer Communications, IEEE 2008, pp. 156–160
Xu, D., Sheng, Y., Ju, D., Wu, J., Wang, D.: High effective two-round remote file fast synchronization algorithm. Jisuanji Kexue yu Tansuo 5(1), 38–49 (2011)
Quinlan, S., Dorward, S.: Venti: A new approach to archival storage. In: Proceedings of the FAST 2002 Conference on File and Storage Technologies, Volume 4 (2002)
Muthitacharoen, A., Chen, B., Mazieres, D.: A low-bandwidth network file system. ACM SIGOPS Oper. Syst. Rev. 35(5), 174–187 (2001)
Jung, H., Park, W., Lee, W., Lee, J., Ko, Y.: Data deduplication system for supporting multi-mode. In: Proceedings of the Third International Conference on Intelligent Information and Database Systems-Volume Part I, Springer, pp. 78–87 (2011)
Acknowledgements
This research was supported by Hallym University Research Fund (HRF-201109-053) and by the Ministry of Education, Science Technology (MEST) and National Research Foundation of Korea (NRF) through the Human Resource Training Project for Regional Innovation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Science+Business Media Dortdrecht
About this paper
Cite this paper
Jung, H.M., Park, J.M., Song, C.G., Kim, S.J., Chun, S.H., Ko, Y.W. (2012). Minimizing Metadata Size for File Synchronization Using Variable-Length Chunking. In: J. (Jong Hyuk) Park, J., Leung, V., Wang, CL., Shon, T. (eds) Future Information Technology, Application, and Service. Lecture Notes in Electrical Engineering, vol 164. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-4516-2_35
Download citation
DOI: https://doi.org/10.1007/978-94-007-4516-2_35
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-4515-5
Online ISBN: 978-94-007-4516-2
eBook Packages: EngineeringEngineering (R0)