Skip to main content
SpringerLink
Log in

An Expandable Parallel File System Using NFS Servers

  • Conference paper
  • First Online:
High Performance Computing for Computational Science — VECPAR 2002 (VECPAR 2002)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2565))

  • 680 Accesses

Abstract

This paper describes a new parallel file system, called Expand (Expandable Parallel File System)1, that is based on NFS servers. Expand allows the transparent use of multiple NFS servers as a single file system. The different NFS servers are combined to create a distributed partition where files are declustered. Expand requires no changes to the NFS server and uses RPC operations to provide parallel access to the same file. Expand is also independent of the clients, because all operations are implemented using RPC and NFS protocol. Using this system, we can join heterogeneous servers (Linux, Solaris, Windows 2000, etc.) to provide a parallel and distributed partition. The paper describes the design of Expand and the evaluation of a prototype of Expand. This evaluation has been made in Linux clusters and compares Expand, NFS and PVFS.

This work has been partially support by the Spanish Ministry of Science and Technology under the TIC2000-0469 and TIC2000-0471contracts, and by the Community of Madrid under the 07T/0013/2001 contract.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Carns, P.H., Ligon III, W.B., Ross, R. B., Takhur, R. PVFS: A Parallel File System for Linux Clusters. Technical Report ANL/MCS-P804-0400, 2000. 565, 567, 572, 574

    Google Scholar 

  2. Carretero, J., Perez, F., de Miguel, P., Garcia, F., Alonso, L. Performance Increase Mechanisms for Parallel and Distributed File Systems. Parallel Computing: Special Issue on Parallel I/O Systems. Elsevier, (23):525–542, April 1997. 565, 566, 567

    MATH  Google Scholar 

  3. Corbett, P., Johnson, S., Feitelson, D. Overview of the Vesta Parallel File System. ACM Computer Architecture News, 21(5):7–15, December 1993. 565, 567, 572

    Article  Google Scholar 

  4. Chase, F. S., Anderson, D.C., Vahdat, A.M. Interposed Request Routing for Scalable Network Storage. In Fourth Symposium on Operating System Design and Implementation (OSDI2000), 2000. 568

    Google Scholar 

  5. DeBenedictis, E., del Rosario, J. M. Ncube Parallel I/O Software. In Eleventh Annual IEEE International Phoenix Conference on Computers and Communications (IPCCC), pages 117–124, April 1992. 567

    Google Scholar 

  6. Dibble, P., Scott, M., Ellis, C. BRIDGE: A High Performance File System for Parallel Processors. In Proceedings of the IEEE Eighth ICDCS, pages 154–161. IEEE, June 1988. 567

    Google Scholar 

  7. Wanek, H., Schikuta, E. Parallel I/O. In Cluster Computing White Paper, December 2000. 566

    Google Scholar 

  8. Fuerle, T., Jorns, P., Schikuta, E., Wanek, H. Meta-ViPIOS: Harness Distributed I/O resources with ViPIOS. Journal of Research Computing and Systems, Special Issue on Parallel Computing, 1999. 565, 567

    Google Scholar 

  9. Garcia, F., Carretero, J., Perez, F., de Miguel, P., Alonso, L. High Performance Cache Management for Parallel File Systems. Lecture Notes in Computer Science, Vol. 1573, 1999. 565, 577

    Google Scholar 

  10. Gibson, G. The Scotch Paralell Storage Systems. Technical Report CMU-CS-95-107, Scholl of Computer Science, Carnegie Mellon University, Pittsburbh, Pennsylvania, 1995. 565, 567 578 Félix García et al.

    Google Scholar 

  11. Huber, J., Elford, C. L., et al. PPFS: A High Performance Portable Parallel File System. In Proceedings of the 9th ACM International Conference on Supercomputing, pages 385–394. IEEE, July 1995. 567

    Google Scholar 

  12. Kim, G.H., Minninch, R. G. Bigfoot-NFS: A Parallel File-Striping NFS Server. Technical report, Sun Microsystems Computer Corp., 1994. 568

    Google Scholar 

  13. Krieger, O. HFS: A Flexible File System for Shared-Memory Multiprocessors. PhD thesis, Department of Electrical and Computer Engineering, University of Toronto, 1994. 565, 567

    Google Scholar 

  14. Lo Verso, S., Isman, M., Nanopoulos, A., Nesheim, W., Milne, E., Wheeler, R. sfs: A Parallel File System for the CM-5. In Proceedings of the 1993 Summer Usenix Conference, pages 291–305, 1993. 567

    Google Scholar 

  15. Madhyastha, T. Automatic Classification of Input/Output Access Patterns. PhD thesis, niversidad de Illinois, Urbana-Champaign, 1997. 567

    Google Scholar 

  16. Moyer, S. A., Sunderam, V. S. PIOUS: A Scalable Parallel I/O System for Distributed Computing Environments. In Proceedings of the Scalable High-Performance Computing Conferece, pages 71–78, 1994. 565, 567

    Google Scholar 

  17. Nieuwejaar, N., Kotz, D. The Galley Parallel File System. In Proceedings of the 10th ACM International Conference on Supercomputing, May 1996. 565, 567, 573

    Google Scholar 

  18. Olfield, R., Kotz, D. The Armada Parallel File System, 1998. http://www.cs.dartmouth.edu/ dfk/armada/design.html. 565, 567

  19. Patterson, D., Gibson, G., Katz, R. A Case for Redundant Arrays of Inexpensive Disks (RAID). In Proceedings of ACM SIGMOD, pages 109–116. ACM, June 1988. 566

    Google Scholar 

  20. Pierce, P. A Concurrent File System for a Highly Parallel Mass Storage Subsystem. In John L. Gustafson, editor, Proceedings of the Fourth Conference on Hypercubes Concurrent Computers and Applications, pages 155–161. HCCA, March 1989. 567

    Google Scholar 

  21. Gropp, W., Takhur, R., Lusk, E. An Abstract-Device Interface for Implementing Portable Parallel-I/O Interfaces. In Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computation, pages 180–187, October 1996. 567

    Google Scholar 

  22. Gropp, W., Takhur, R., Lusk, E. On Implementing MPI-IO Portably and with High Performance. In of the Sixth Workshop on I/O in Parallel and Distributed Systems, pages 23–32, 1999. 573, 577

    Google Scholar 

  23. del Rosario, J.M., Bordawekar, R., Choundary, A. Improved Parallel I/O via a Two-phase Run-time Access Strategy. ACM Computer Architecture News, 21(5):31–39, December 1993. 566

    Article  Google Scholar 

  24. Sandberg, R., Goldberg, D., Kleiman, S., Walsh, D., Lyon, B. Design and Implementation of the SUN Network Filesystem. In Proc. of the 1985 USENIX Conference. USENIX, 1985. 567

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Architecture Group, Computer Science Department Universidad Carlos III de Madrid, 28911, Leganés, Madrid, Spain

    Félix García, Alejandro Calderón, Jesus Carretero, Jose M. Pérez & Javier Fernández

Authors
  1. Félix García
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alejandro Calderón
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jesus Carretero
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jose M. Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Javier Fernández
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculdade de Engenharia da, Universidade do Porto, Rua Dr. Roberto Frias, 4200-465, Porto, Portugal

    José M. L. M. Palma  & A. Augusto Sousa  & 

  2. Department of Computer Science, University of Tennessee, 37996-1301, Knoxville, TN, USA

    Jack Dongarra

  3. Departamento de Sistemas Informáticos y Computación, Universidad Politécnica de Valencia, Camino de Vera, s/n, Apartado 22012, 46020, Valencia, Spain

    Vicente Hernández

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

García, F., Calderón, A., Carretero, J., Pérez, J.M., Fernández, J. (2003). An Expandable Parallel File System Using NFS Servers. In: Palma, J.M.L.M., Sousa, A.A., Dongarra, J., Hernández, V. (eds) High Performance Computing for Computational Science — VECPAR 2002. VECPAR 2002. Lecture Notes in Computer Science, vol 2565. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36569-9_38

Download citation

  • DOI: https://doi.org/10.1007/3-540-36569-9_38

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-00852-1

  • Online ISBN: 978-3-540-36569-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation