Abstract
Platform for Distance Education of China will manage large amounts of educational resource files and large numbers of users, both of which are geographically distributed all over China. In such a grid environment, a high scalable, distributed file sharing approach with dynamic replication is needed. In this paper, we review existing related activities in file sharing and propose a web-based distributed file management architecture called Grid File System (GridFS). We describe its two fundamental services (namely, file access service and metadata service) followed with query processing and dynamic replication algorithm. Our query processing algorithm eliminates the duplication of query messages, and the simulation results show that our approach eliminate exponentially increase of the number of average per-site messages as the TTL (Time To Live) of a query increases to get higher query success rate. We believe GridFS would benefit most grid applications, web information systems and hierarchical web cache systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chervenak, A., Foster, I., Kesselman, C., Salisbury, C., Tuecke, S.: The Data Grid: Towards an architecture for the distributed management and analysis of large scientific datasets. Journal of Network and Computer Applications 23, 187–200 (2001)
Sandberg, R.: The Sun Network File System: Design, Implementation and Experience, Tech. Report, Sun Microsystems, Mountain View CA (1987)
Kazar, M.L., Leverett, B.W., Anderson, O.T., Apostolides, V., Bottos, B.A., Chutani, S., Everhart, C.F., Mason, W.A., Tu, S., Zayas, R.: DEcorum file system architectural overview. In: Proceedings of the Summer USENIX Conference (June 1990)
Morris, J., et al.: Andrew: A Distributed Personal Computing Evironment. Comms. ACM 29(3), 184–201 (1996)
Rajasekar, A.K., Moore, R.W.: Data and Metadata Collections for Scientific Applications. In: Hertzberger, B., Hoekstra, A.G., Williams, R. (eds.) HPCN-Europe 2001. LNCS, vol. 2110, p. 72. Springer, Heidelberg (2001)
Watson, R.W., Coyne, R.A.: The parallel I/O architecture of the High-Performance Storage System (HPSS). In: IEEE MSS Symposium (1995)
Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the grid: Enabling scalable virtual organizations. International Journal on Supercomputing Applications (2001)
European Data Grid (EDG), http://www.eu-datagrid.org/
Grid Physics Network (GriPhyN), http://www.griphyn.org/
Wahl, M., Howes, T., Kille, S.: Lightweight directory access protocol. RFC 2251, vol. 3, Internet Engineering Task Force (1997)
Ranganathan, K., Foster, I.: Identifying Dynamic Replication Strategies for a High Performance Data Grid. In: Proceedings of the International Workshop on Grid Computing, Denver, Colorado (November 2001)
Parsec home page, http://pcl.cs.ucla.edu/projects/parsec
Almeida, V., Bestavros, A., Crovella, M., de Oliveira, A.: Characterizing reference locality in the www. In: Proceedings of 1996 International Conference on parallel and Distributed Information Systems (PDIS 1996) (1996)
Marchioro II, T.L., Landau, R.H.: Web-based Education in Computational Science and Engineering. In: Computational Science Engineering, April-June 1997, pp. 19–26. IEEE Computer Society, Los Alamitos (1997)
Clarke, I., Sandberg, O., Wiley, B., Hong, T.: Freenet: A Distributed Anonymous Information Storage and Retrieval System. In: ICSI Workshop on Design Issues in Anonymity and Unobservability (July 2000)
Gnutella, http://gnutella.wego.com
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Qinghu, L., Jianmin, W., Lam, K.Y., Jiaguang, S. (2003). GridFS: A Web-Based Data Grid for the Distributed Sharing of Educational Resource Files. In: Zhou, W., Nicholson, P., Corbitt, B., Fong, J. (eds) Advances in Web-Based Learning - ICWL 2003. ICWL 2003. Lecture Notes in Computer Science, vol 2783. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45200-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-45200-3_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40772-0
Online ISBN: 978-3-540-45200-3
eBook Packages: Springer Book Archive