Skip to main content
SpringerLink
Log in

Towards a scalable metacomputing storage service

  • Track C2: Computational Science
  • Conference paper
  • First Online:
High-Performance Computing and Networking (HPCN-Europe 1999)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1593))

Included in the following conference series:

Abstract

We describe a prototypical storage service through which we are addressing some of the open storage issues in wide-area distributed high-performance computing. We discuss some of the relevant topics such as latency-tolerance, hierarchical storage integration, and legacy and commercial application support. Existing high-performance computing environments are either ad-hoc or focus narrowly on the simple clienserver case. The storage service which we are developing as part of the DISCWorld metacomputing infrastructure, will provide high-performance access to a global “cloud”, of storage resources in a manner which is scalable, secure, adaptive and portable requiring no application or operating system modifications. Our system design provides flexible, modular and user-extensible access to arbitrary storage mechanisms and on-demand data generation and transformations. We describe our current prototype's status, some performance analysis, other related research and our future plans for the system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Matt Blaze, A Cryptographic File System for Unix, Proc. First ACM Conference on Communications and Computing Security, Fairfax, Virginia, November 1993.

    Google Scholar 

  2. A. L. Brown, Utilising NFS to Expose Persistent Object Store I/O, Proc. Sixth IDEA Workshop, Rutherglen, Australia, January 1998.

    Google Scholar 

  3. Vincent Cate and Thomas Gross, Combining the Concepts of Compression and Caching for a Two-Level Filesystem, Proc. Fourth ASPLOS, Santa Clara, April 1992, pp. 200–211.

    Google Scholar 

  4. C. Catlett, and L. Smarr, Metacomputing, Comm. ACM, 35 (1992), pp. 44–52.

    Article  Google Scholar 

  5. B. Christiansen, P. Cappello, M. F. Ionescu, M. O. Neary, K. E. Schauser, and D. Wu, Javelin: Internet-Based Parallel Computing Using Java, Proc. ACM Workshop on Java for Science and Engineering Computation, June 1997.

    Google Scholar 

  6. I. Foster, C. Kesselman, Globus: A Metacomputing Infrastructure Toolkit, Intl. Journal of Supercomputer Applications, 11 (2):115–128, 1997.

    Article  Google Scholar 

  7. David K. Gifford, Pierre Jouvelot, Mark A. Sheldon, James W. O'Toole, Jr., Semantic File Systems, Proc. Thirteenth Symposium on Operating Systems Principles, 1991.

    Google Scholar 

  8. Japanese Meteorological Satellite Center, The GMS User's Guide, 2nd Ed., 1989.

    Google Scholar 

  9. K.A. Hawick, H.A. James, K.J. Maciunas, F.A. Vaughan, A.L. Wendelborn, M. Buchhorn, M. Rezny, S.R. Taylor and M.D. Wilson, Geographic Information Systems Applications on an ATM-Based Distributed High Performance Computing System, Proc. HPCN Europe '97, Vienna, April 1997.

    Google Scholar 

  10. K.A. Hawick, P.D. Coddington, D.A. Grove, J.F. Hercus, H.A. James, K.E. Kerry, J.A. Mathew, C.J. Patten, A.J. Silis F.A. Vaughan, DISCWorld: An Environment for Service-Based Metacomputing. Future Generations of Computer Science Special Issue on Metacomputing. Also DHPC Technical Report DHPC-042, April 1998.

    Google Scholar 

  11. K.A. Hawick, H.A. James, Craig J. Patten and F.A. Vaughan, DISCWorld: A Distributed High Performance Computing Environment, Proc. HPCN Europe '98, Amsterdam, April 1998.

    Google Scholar 

  12. H.A. James and K.A. Hawick, A Web-based, Interface for On-Demand Processing of Satellite Imagery Archives, Proc. Australian Computer Science, Conference (ACSC) '98, Perth, February 1998.

    Google Scholar 

  13. Craig J. Patten, F.A. Vaughan, K.A. Hawick and A.L. Brown, DWorFS: File System Support for Legacy Applications in DISCWorld, Proc. Fifth IDEA Workshop, Fremantle, February 1998.

    Google Scholar 

  14. Brian Pawlowski, Chet Juszczak, Peter Staubach, Carl Smith Diane Lebel, David Hitz, NFS Version 3 Design and Implementation, Proc. USENIX 1994 Summer Conference, June 1994.

    Google Scholar 

  15. R. Sandberg, D. Goldberg, S. Kleiman, D. Walsh, and B. Lyon, Design and Implementation of the Sun Network Filesystem, Proc. USENIX 1985 Summer Conference, pp. 119–130, 1985.

    Google Scholar 

  16. M. Satyanaranan, J.H. Howard, D.N. Nichols, R.N. Sidebotham., A.Z. Spector, and M.J. West, The ITC Distributed File System: Principles and Design, Proc. Tenth Symposium on Operating Systems Principles, pp. 35–50, 1985.

    Google Scholar 

  17. StorageTek TimberWolf 9740 Specifications, URL http://www.stortek.com/StorageTek/hardware/tape/9740.

    Google Scholar 

  18. Sun Microsystems, NFS V2 Specification, RFC 1094, March 1989.

    Google Scholar 

  19. Sun Microsystems, NFS V3 Specification, RFC 1813, June 1995.

    Google Scholar 

  20. Sun Microsystems, WebNFS Client Specification, RFC 2054, October 1996.

    Google Scholar 

  21. Sun Microsystems, WebNFS Server Specification, RFC 2055, October 1996.

    Google Scholar 

  22. Sun Microsystems, WebNFS, April 1997.

    Google Scholar 

  23. Amin Vahdat, Thomas Anderson, Michael Dahlin, Eshwar Belani, David Culler, Paul Eastham, and Chad Yoshikawa, WebOS: Operating System Services for Wide Area Applications, Proc. Seventh HPDC Conference, Chicago, July 1998.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Advanced Computational Systems Cooperative Research Centre Department of Computer Science, University of Adelaide, 5005, Adelaide, SA, Australia

    Craig J. Patten, K. A. Hawick & J. F. Hercus

Authors
  1. Craig J. Patten
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. A. Hawick
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. F. Hercus
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Peter Sloot Marian Bubak Alfons Hoekstra Bob Hertzberger

Rights and permissions

Reprints and permissions

Copyright information

© 1999 Springer-Verlag

About this paper

Cite this paper

Patten, C.J., Hawick, K.A., Hercus, J.F. (1999). Towards a scalable metacomputing storage service. In: Sloot, P., Bubak, M., Hoekstra, A., Hertzberger, B. (eds) High-Performance Computing and Networking. HPCN-Europe 1999. Lecture Notes in Computer Science, vol 1593. Springer, Berlin, Heidelberg . https://doi.org/10.1007/BFb0100596

Download citation

  • DOI: https://doi.org/10.1007/BFb0100596

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-65821-4

  • Online ISBN: 978-3-540-48933-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation