Implementing Byte-Range Locks Using MPI One-Sided Communication

  • Rajeev Thakur
  • Robert Ross
  • Robert Latham
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3666)


We present an algorithm for implementing byte-range locks using MPI passive-target one-sided communication. This algorithm is useful in any scenario in which multiple processes of a parallel program need to acquire exclusive access to a range of bytes. One application of this algorithm is for implementing MPI-IO’s atomic-access mode in the absence of atomicity guarantees from the underlying file system. Another application is for implementing data sieving, a technique for optimizing noncontiguous writes by doing an atomic read-modify-write of a large, contiguous block of data. This byte-range locking algorithm can be used instead of POSIX fcntl file locks on file systems that do not support fcntl locks, on file systems where fcntl locks are unreliable, and on file systems where fcntl locks perform poorly. Our performance results demonstrate that the algorithm has low overhead and significantly outperforms fcntl locks on NFS file systems on a Linux cluster and on a Sun SMP.


File System Atomic Mode Target Process Window Memory Exclusive Access 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Carns, P.H., Ligon III, W.B., Ross, R.B., Thakur, R.: PVFS: A parallel file system for Linux clusters. In: Proceedings of the 4th Annual Linux Showcase and Conference, Atlanta, October 2000, pp. 317–327 (2000)Google Scholar
  2. 2.
    Corbett, P.F., Feitelson, D.G.: The Vesta parallel file system. ACM Transactions on Computer Systems 14(3), 225–264 (1996)CrossRefGoogle Scholar
  3. 3.
    Gropp, W., Lusk, E., Thakur, R.: Using MPI-2: Advanced Features of the Message-Passing Interface. MIT Press, Cambridge (1999)Google Scholar
  4. 4.
    Huber, J., Elford, C.L., Reed, D.A., Chien, A.A., Blumenthal, D.S.: PPFS: A high performance portable parallel file system. In: Proceedings of the 9th ACM International Conference on Supercomputing, July 1995, pp. 385–394. ACM Press, New York (1995)CrossRefGoogle Scholar
  5. 5.
    IEEE/ANSI Std. 1003.1. Portable Operating System Interface (POSIX)–Part 1: System Application Program Interface (API) [C Language] (1996) (edition)Google Scholar
  6. 6.
    Mellor-Crummey, J.M., Scott, M.L.: Algorithms for scalable synchronization on shared-memory multiprocessors. ACM Transactions on Computer Systems (1991)Google Scholar
  7. 7.
    Message Passing Interface Forum. MPI-2: Extensions to the Message-Passing Interface (July 1997),
  8. 8.
    Nieuwejaar, N., Kotz, D.: The Galley parallel file system. Parallel Computing 23(4), 447–476 (1997)zbMATHCrossRefGoogle Scholar
  9. 9.
    PVFS2: Parallel virtual file system,
  10. 10.
    Red Hat Global File System,
  11. 11.
    ROMIO: A high-performance, portable MPI-IO implementation,
  12. 12.
    Ross, R., Latham, R., Gropp, W., Thakur, R., Toonen, B.: Implementing MPI-IO atomic mode without file system support. In: Proceedings of CCGrid 2005 (May 2005)Google Scholar
  13. 13.
    Thakur, R., Gropp, W., Lusk, E.: On implementing MPI-IO portably and with high performance. In: Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, May 1999, pp. 23–32. ACM Press, New York (1999)CrossRefGoogle Scholar
  14. 14.
    Thakur, R., Gropp, W., Lusk, E.: Optimizing noncontiguous accesses in MPI-IO. Parallel Computing 28(1), 83–105 (2002)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Rajeev Thakur
    • 1
  • Robert Ross
    • 1
  • Robert Latham
    • 1
  1. 1.Mathematics and Computer Science DivisionArgonne National LaboratoryArgonneUSA

Personalised recommendations