Skip to main content
SpringerLink
Log in

Overview of the MPI-IO Parallel I/O Interface

  • Chapter
Input/Output in Parallel and Distributed Computer Systems

Part of the book series: The Kluwer International Series in Engineering and Computer Science ((SECS,volume 362))

Abstract

Thanks to MPI, writing portable message passing parallel programs is almost a reality. One of the remaining problems is file I/O. Although parallel file systems support similar interfaces, the lack of a standard makes developing a truly portable program impossible. It is not feasible to develop large scientific applications from scratch for each generation of parallel machine, and, in the scientific world, a program is not considered truly portable unless it not only compiles, but also runs efficiently.

The MPI-IO interface is being proposed as an extension to the MPI standard to fill this need. MPI-IO supports a high-level interface to describe the partitioning of file data among processes, a collective interface describing complete transfers of global data structures between process memories and files, asynchronous I/O operations, allowing computation to be overlapped with I/O, and optimization of physical file layout on storage devices (disks).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Robert Bennett, Kelvin Bryant, Alan Sussman, Raja Das, and Joel Saltz. Jovian: A framework for optimizing parallel I/O. In Proceedings of the Scalable Parallel Libraries Conference, pages 10–20. IEEE Computer Society Press, October 1994.

    Google Scholar 

  2. Michael L. Best, Adam Greenberg, Craig Stanfill, and Lewis W. Tucker. CMMD I/O: A parallel Unix I/O. In Proceedings of the Seventh International Parallel Processing Symposium, pages 489–495, 1993.

    Chapter  Google Scholar 

  3. Alok Choudhary, Rajesh Bordawekar, Michael Harry, Rakesh Krishnaiyer, Ravi Ponnusamy, Tarvinder Singh, and Rajeev Thakur. PASSION: parallel and scalable software for input-output. Technical Report SCCS-636, ECE Dept., NPAC and CASE Center, Syracuse University, September 1994.

    Google Scholar 

  4. Peter F. Corbett, Sandra Johnson Baylor, and Dror G. Feitelson. Overview of the Vesta parallel file system. In IPPS ’93 Workshop on Input/Output in Parallel Computer Systems, pages 1–16, 1993.

    Google Scholar 

  5. Peter F. Corbett, Sandra Johnson Baylor, and Dror G. Feitelson. Also published in Computer Architecture News 21(5), December 1993, pages 7–14.

    Article  Google Scholar 

  6. Peter F. Corbett and Dror G. Feitelson. Design and implementation of the Vesta parallel file system. In Proceedings of the Scalable High-Performance Computing Conference, pages 63–70, 1994.

    Chapter  Google Scholar 

  7. Peter F. Corbett, Dror G. Feitelson, Jean-Pierre Prost, George S. Almasi, Sandra Johnson Baylor, Anthony S. Bolmarcich, Yarsun Hsu, Julian Satran, Marc Snir, Robert Colao, Brian Herr, Joseph Kavaky, Thomas R. Morgan, and Anthony Zlotek. Parallel file systems for the IBM SP computers. IBM Systems Journal, pages 222–248, 1995.

    Google Scholar 

  8. Peter F. Corbett, Dror G. Feitelson, Jean-Pierre Prost, and Sandra Johnson Baylor. Parallel access to files in the Vesta file system. In Proceedings of Supercomputing ’93, pages 472–481, 1993.

    Google Scholar 

  9. Parasoft Corporation. Express version 1.0: A communication environment for parallel computers, 1988.

    Google Scholar 

  10. Erik DeBenedictis and Juan Miguel del Rosario. nCUBE parallel I/O software. In Proceedings of the Eleventh Annual IEEE International Phoenix Conference on Computers and Communications, pages 0117–0124, April 1992.

    Chapter  Google Scholar 

  11. Juan Miguel del Rosario, Rajesh Bordawekar, and Alok Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In IPPS ’93 Workshop on Input/Output in Parallel Computer Systems, pages 56–70, 1993.

    Google Scholar 

  12. Juan Miguel del Rosario, Rajesh Bordawekar, and Alok Choudhary. Also published in Computer Architecture News 21(5), December 1993, pages 31–38.

    Article  Google Scholar 

  13. Juan Miguel del Rosario, Michael Harry, and Alok Choudhary. The design of VIP-FS: A virtual, parallel file system for high performance parallel and distributed computing. Technical Report SCCS-628, NPAC, Syracuse, NY 13244, May 1994.

    Google Scholar 

  14. Intel Supercomputer Systems Division. iPSC/2 and iPSC/860 user’s guide, April 1991. Order number: 311532–007.

    Google Scholar 

  15. Intel Supercomputer Systems Division. Intel Paragon XP/S user’s guide, April 1993. Order number: 312489–01.

    Google Scholar 

  16. Rüdiger Esser and Renate Knecht. Intel Paragon XP/S — architecture and software environment. Technical Report KFA-ZAM-IB-9305, Central Institute for Applied Mathematics, Research Center Jülich, Germany, r. esser@kfa-juelich. de, April 26 1993.

    Google Scholar 

  17. Dror G. Feitelson, Peter F. Corbett, Yarson Hsu, and Jean-Pierre Prost. Parallel I/O systems and interfaces for parallel computers. In C.-L. Wu, editor, Multiprocessor Systems—Design and Integration. World Scientific, 1995. To appear.

    Google Scholar 

  18. Dror G. Feitelson, Peter F. Corbett, and Jean-Pierre Prost. Performance of the Vesta parallel file system. In Proceedings of the Ninth International Parallel Processing Symposium, pages 150–158, April 1995.

    Chapter  Google Scholar 

  19. N. Galbreath, W. Gropp, and D. Levine. Applications-driven parallel I/O. In Proceedings of Supercomputing ’93, pages 462–471, 1993.

    Google Scholar 

  20. Jay Huber, Christopher L. Elford, Daniel A. Reed, Andrew A. Chien, and David S. Blumenthal. PPFS: A high performance portable parallel file system. In Proceedings of the 9th ACM International Conference on Supercomputing,pages 385–394, Barcelona, July 1995.

    Chapter  Google Scholar 

  21. David Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994 Symposium on Operating Systems Design and Implementation, pages 61–74, November 1994. Updated as Dartmouth TR PCS-TR94–226 on November 8, 1994.

    Google Scholar 

  22. David Kotz and Nils Nieuwejaar. Dynamic file-access characteristics of a production parallel scientific workload. In Proceedings of Supercomputing ’94, pages 640–649, November 1994.

    Chapter  Google Scholar 

  23. Susan J. LoVerso, Marshall Isman, Andy Nanopoulos, William Nesheim, Ewan D. Milne, and Richard Wheeler. sfs: A parallel file system for the CM-5. In Proceedings of the 1993 Summer USENIX Conference, pages 291–305, 1993.

    Google Scholar 

  24. Message Passing Interface Forum. MPI: A Message-Passing Interface Standard. University of Tennessee, May 1994.

    Google Scholar 

  25. Steven A. Moyer and V. S. Sunderam. A parallel I/O system for high-performance distributed computing. In Proceedings of the IFIP WG10.3 Working Conference on Programming Environments for Massively Parallel Distributed Systems, 1994.

    Google Scholar 

  26. Steven A. Moyer and V. S. Sunderam. PIOUS: a scalable parallel I/O system for distributed computing environments. In Proceedings of the Scalable High-Performance Computing Conference, pages 71–78, 1994.

    Chapter  Google Scholar 

  27. Nils Nieuwejaar and David Kotz. A multiprocessor extension to the conventional file system interface. Technical Report PCS-TR94–230, Dept. of Computer Science, Dartmouth College, September 1994.

    Google Scholar 

  28. Bill Nitzberg. Performance of the iPSC/860 Concurrent File System. Technical Report RND-92–020, NAS Systems Division, NASA Ames, December 1992.

    Google Scholar 

  29. William J. Nitzberg. Collective Parallel I/O. PhD thesis, Department of Computer and Information Science, University of Oregon, December 1995.

    Google Scholar 

  30. Paul Pierce. A concurrent file system for a highly parallel mass storage system. In Proceedings of the Fourth Conference on Hypercube Concurrent Computers and Applications, pages 155–160. Golden Gate Enterprises, Los Altos, CA, March 1989.

    Google Scholar 

  31. Brad Rullman and David Payne. An efficient file I/O interface for parallel applications. DRAFT presented at the Workshop on Scalable I/O, Frontiers ’95, February 1995.

    Google Scholar 

  32. K. E. Seamons and M. Winslett. An efficient abstract interface for multidimensional array I/O. In Proceedings of Supercomputing ’94, pages 650–659, November 1994.

    Chapter  Google Scholar 

  33. K. E. Seamons and M. Winslett. Physical schemas for large multidimensional arrays in scientific computing applications. In Proceedings of the 7th International Working Conference on Scientific and Statistical Database Management, pages 218–227, September 1994.

    Chapter  Google Scholar 

  34. K. E. Seamons and M. Winslett. A data management approach for handling large compressed arrays in high performance computing. In Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computation, pages 119–128, February 1995.

    Google Scholar 

  35. [ Rajeev Thakur. Runtime Support for In-Core and Out-of-Core Data-Parallel Programs. PhD thesis, Department of Electrical and Computer Engineering, Syracuse University, May 1995.

    Google Scholar 

  36. Rajeev Thakur, Rajesh Bordawekar, Alok Choudhary, Ravi Ponnusamy, and Tarvinder Singh. PASSION runtime library for parallel I/O. In Proceedings of the Scalable Parallel Libraries Conference, pages 119–128, October 1994.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IBM T.J. Watson Research Center, Yorktown Heights, New York, 10598, USA

    Peter Corbett, Dror Feitelson, Yarsun Hsu, Jean-Pierre Prost & Marc Snirt

  2. MRJ, Inc., NASA Ames Research Center, M/S 258-6, Moffett Field, California, 94035, USA

    Sam Fineberg, Bill Nitzberg, Bernard Traversat & Parkson Wong

Authors
  1. Peter Corbett
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dror Feitelson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sam Fineberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yarsun Hsu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bill Nitzberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jean-Pierre Prost
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Marc Snirt
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Bernard Traversat
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Parkson Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Bell Communications Research, Morristown, New Jersey, USA

    Ravi Jain

  2. University of Texas at Austin, Austin, Texas, USA

    John Werth  & James C. Browne  & 

Rights and permissions

Reprints and permissions

Copyright information

© 1996 Kluwer Academic Publishers

About this chapter

Cite this chapter

Corbett, P. et al. (1996). Overview of the MPI-IO Parallel I/O Interface. In: Jain, R., Werth, J., Browne, J.C. (eds) Input/Output in Parallel and Distributed Computer Systems. The Kluwer International Series in Engineering and Computer Science, vol 362. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-1401-1_5

Download citation

  • DOI: https://doi.org/10.1007/978-1-4613-1401-1_5

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4612-8607-3

  • Online ISBN: 978-1-4613-1401-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation