Implementing Byte-Range Locks Using MPI One-Sided Communication
We present an algorithm for implementing byte-range locks using MPI passive-target one-sided communication. This algorithm is useful in any scenario in which multiple processes of a parallel program need to acquire exclusive access to a range of bytes. One application of this algorithm is for implementing MPI-IO’s atomic-access mode in the absence of atomicity guarantees from the underlying file system. Another application is for implementing data sieving, a technique for optimizing noncontiguous writes by doing an atomic read-modify-write of a large, contiguous block of data. This byte-range locking algorithm can be used instead of POSIX fcntl file locks on file systems that do not support fcntl locks, on file systems where fcntl locks are unreliable, and on file systems where fcntl locks perform poorly. Our performance results demonstrate that the algorithm has low overhead and significantly outperforms fcntl locks on NFS file systems on a Linux cluster and on a Sun SMP.
KeywordsFile System Atomic Mode Target Process Window Memory Exclusive Access
Unable to display preview. Download preview PDF.
- 1.Carns, P.H., Ligon III, W.B., Ross, R.B., Thakur, R.: PVFS: A parallel file system for Linux clusters. In: Proceedings of the 4th Annual Linux Showcase and Conference, Atlanta, October 2000, pp. 317–327 (2000)Google Scholar
- 3.Gropp, W., Lusk, E., Thakur, R.: Using MPI-2: Advanced Features of the Message-Passing Interface. MIT Press, Cambridge (1999)Google Scholar
- 5.IEEE/ANSI Std. 1003.1. Portable Operating System Interface (POSIX)–Part 1: System Application Program Interface (API) [C Language] (1996) (edition)Google Scholar
- 6.Mellor-Crummey, J.M., Scott, M.L.: Algorithms for scalable synchronization on shared-memory multiprocessors. ACM Transactions on Computer Systems (1991)Google Scholar
- 7.Message Passing Interface Forum. MPI-2: Extensions to the Message-Passing Interface (July 1997), http://www.mpi-forum.org/docs/docs.html
- 9.PVFS2: Parallel virtual file system, http://www.pvfs.org/pvfs2/
- 10.Red Hat Global File System, http://www.redhat.com/software/rha/gfs
- 11.ROMIO: A high-performance, portable MPI-IO implementation, http://www.mcs.anl.gov/romio
- 12.Ross, R., Latham, R., Gropp, W., Thakur, R., Toonen, B.: Implementing MPI-IO atomic mode without file system support. In: Proceedings of CCGrid 2005 (May 2005)Google Scholar