Towards a scalable metacomputing storage service

  • Craig J. Patten
  • K. A. Hawick
  • J. F. Hercus
Track C2: Computational Science
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1593)


We describe a prototypical storage service through which we are addressing some of the open storage issues in wide-area distributed high-performance computing. We discuss some of the relevant topics such as latency-tolerance, hierarchical storage integration, and legacy and commercial application support. Existing high-performance computing environments are either ad-hoc or focus narrowly on the simple clienserver case. The storage service which we are developing as part of the DISCWorld metacomputing infrastructure, will provide high-performance access to a global “cloud”, of storage resources in a manner which is scalable, secure, adaptive and portable requiring no application or operating system modifications. Our system design provides flexible, modular and user-extensible access to arbitrary storage mechanisms and on-demand data generation and transformations. We describe our current prototype's status, some performance analysis, other related research and our future plans for the system.


File System Storage Service Storage Resource Storage Module Distribute File System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Matt Blaze, A Cryptographic File System for Unix, Proc. First ACM Conference on Communications and Computing Security, Fairfax, Virginia, November 1993.Google Scholar
  2. 2.
    A. L. Brown, Utilising NFS to Expose Persistent Object Store I/O, Proc. Sixth IDEA Workshop, Rutherglen, Australia, January 1998.Google Scholar
  3. 3.
    Vincent Cate and Thomas Gross, Combining the Concepts of Compression and Caching for a Two-Level Filesystem, Proc. Fourth ASPLOS, Santa Clara, April 1992, pp. 200–211.Google Scholar
  4. 4.
    C. Catlett, and L. Smarr, Metacomputing, Comm. ACM, 35 (1992), pp. 44–52.CrossRefGoogle Scholar
  5. 5.
    B. Christiansen, P. Cappello, M. F. Ionescu, M. O. Neary, K. E. Schauser, and D. Wu, Javelin: Internet-Based Parallel Computing Using Java, Proc. ACM Workshop on Java for Science and Engineering Computation, June 1997.Google Scholar
  6. 6.
    I. Foster, C. Kesselman, Globus: A Metacomputing Infrastructure Toolkit, Intl. Journal of Supercomputer Applications, 11 (2):115–128, 1997.CrossRefGoogle Scholar
  7. 7.
    David K. Gifford, Pierre Jouvelot, Mark A. Sheldon, James W. O'Toole, Jr., Semantic File Systems, Proc. Thirteenth Symposium on Operating Systems Principles, 1991.Google Scholar
  8. 8.
    Japanese Meteorological Satellite Center, The GMS User's Guide, 2nd Ed., 1989.Google Scholar
  9. 9.
    K.A. Hawick, H.A. James, K.J. Maciunas, F.A. Vaughan, A.L. Wendelborn, M. Buchhorn, M. Rezny, S.R. Taylor and M.D. Wilson, Geographic Information Systems Applications on an ATM-Based Distributed High Performance Computing System, Proc. HPCN Europe '97, Vienna, April 1997.Google Scholar
  10. 10.
    K.A. Hawick, P.D. Coddington, D.A. Grove, J.F. Hercus, H.A. James, K.E. Kerry, J.A. Mathew, C.J. Patten, A.J. Silis F.A. Vaughan, DISCWorld: An Environment for Service-Based Metacomputing. Future Generations of Computer Science Special Issue on Metacomputing. Also DHPC Technical Report DHPC-042, April 1998.Google Scholar
  11. 11.
    K.A. Hawick, H.A. James, Craig J. Patten and F.A. Vaughan, DISCWorld: A Distributed High Performance Computing Environment, Proc. HPCN Europe '98, Amsterdam, April 1998.Google Scholar
  12. 12.
    H.A. James and K.A. Hawick, A Web-based, Interface for On-Demand Processing of Satellite Imagery Archives, Proc. Australian Computer Science, Conference (ACSC) '98, Perth, February 1998.Google Scholar
  13. 13.
    Craig J. Patten, F.A. Vaughan, K.A. Hawick and A.L. Brown, DWorFS: File System Support for Legacy Applications in DISCWorld, Proc. Fifth IDEA Workshop, Fremantle, February 1998.Google Scholar
  14. 14.
    Brian Pawlowski, Chet Juszczak, Peter Staubach, Carl Smith Diane Lebel, David Hitz, NFS Version 3 Design and Implementation, Proc. USENIX 1994 Summer Conference, June 1994.Google Scholar
  15. 15.
    R. Sandberg, D. Goldberg, S. Kleiman, D. Walsh, and B. Lyon, Design and Implementation of the Sun Network Filesystem, Proc. USENIX 1985 Summer Conference, pp. 119–130, 1985.Google Scholar
  16. 16.
    M. Satyanaranan, J.H. Howard, D.N. Nichols, R.N. Sidebotham., A.Z. Spector, and M.J. West, The ITC Distributed File System: Principles and Design, Proc. Tenth Symposium on Operating Systems Principles, pp. 35–50, 1985.Google Scholar
  17. 17.
    StorageTek TimberWolf 9740 Specifications, URL Scholar
  18. 18.
    Sun Microsystems, NFS V2 Specification, RFC 1094, March 1989.Google Scholar
  19. 19.
    Sun Microsystems, NFS V3 Specification, RFC 1813, June 1995.Google Scholar
  20. 20.
    Sun Microsystems, WebNFS Client Specification, RFC 2054, October 1996.Google Scholar
  21. 21.
    Sun Microsystems, WebNFS Server Specification, RFC 2055, October 1996.Google Scholar
  22. 22.
    Sun Microsystems, WebNFS, April 1997.Google Scholar
  23. 23.
    Amin Vahdat, Thomas Anderson, Michael Dahlin, Eshwar Belani, David Culler, Paul Eastham, and Chad Yoshikawa, WebOS: Operating System Services for Wide Area Applications, Proc. Seventh HPDC Conference, Chicago, July 1998.Google Scholar

Copyright information

© Springer-Verlag 1999

Authors and Affiliations

  • Craig J. Patten
    • 1
  • K. A. Hawick
    • 1
  • J. F. Hercus
    • 1
  1. 1.Advanced Computational Systems Cooperative Research Centre Department of Computer ScienceUniversity of AdelaideAdelaideAustralia

Personalised recommendations