Abstract
Providing data availability in a high performance computing environment is very important, especially in this data-intensive world. Most clusters either use RAID technology or redundant nodes to ensure data availability. People usually use parallel file systems to increase the throughput of a computing system. However, when a parallel file system is involved in a distributed environment, some mechanisms must be provided to overcome the side-effect of using striping. PVFS is a popular and open source parallel file system in the Linux environment, but it provides no fault tolerance. We propose an idea of using distributed RAID technology to ensure the data availability of using striping. By introducing a parity cache table (PCT), we can improve write performance when updating parity is needed. The evaluation of our MRPVFS (Modularized Redundant Parallel Virtual File System) shows that the read performance of MRPVFS is almost the same as that of the original PVFS. As to the write performance, there only exists a little performance degradation due to generating parities.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Intel Supercomputer System Division, Intel Paragon Users’ Guide, (June 1994)
Corbett, P.F., Feitelson, D.G., Prost, J.-P., Almasi, G.S., Baylor, S.J., Bolmaricich, A.S., Hsu, Y., Satran, J., Sinr, M., Colao, R., Herr, B., Kavaky, J., Morgan, T.R., Zlotel, A.: Parallel File Systems for IBM SP Computers. IBM Systems Journal 34(2), 222–248 (1995)
Corbett, P.F., Feitelson, D.G.: The Vesta Parallel File System. ACM Trans. on Computer Systems 14(3), 225–266 (1996)
The beowulf cluster, available at http://www.beowulf.org/
Kleiman, S., Walsh, D., Sandberg, R., Goldberg, D., Lyon, B.: Design and Implementation of the Sun Network File System. In: Proc. Summer USENIX Technical Conf., pp. 119–130 (1985)
Message Passing Interface Forum, MPI2: Extensions to the Message Passing Interface (1997)
Ross, R.B., Carns, P.H., Ligon III, W.B., Thakur, R.: PVFS: A Parallel File System for Linux Clusters. In: Proc. 4th Annual Linux Showcase and Conference, USENIX Association, Atlanta GA, pp. 317–327 (2000)
Gibson, G., Katz, R.H., Patterson, D.A.: A Case for Redundant Arrays of Inexpensive Disks (RAID). In: Int’l Conf. on Management of Data (SIGMOD), pp. 109–116 (1988)
Pâris, J.-F.: A Disk Architecture for Large Clusters of Workstations. In: Cluster Computing Conf., GA (March 1997)
Amdahl, G.M.: Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities. Proc. AFIPS Conf. 30, 483–485 (1967)
Brown, A., Patterson, D.A.: Embracing Failure: A Case for Recovery-Oriented Computing (ROC). In: High Performance Transaction Processing Symp., Asilomar, CA (October 2001)
Long, D.D.E., Montague, B.R., Cabrera, L.-F.: SWIFT/RAID: A Distributed RAID System. Computing Systems 7(2), 333–359 (1994)
Hartman, J.H., Ousterhout, J.K.: The Zebra Striped Network File System. In: Proc. of the Fourteenth ACM Symp. on Operating Systems Principles, pp. 29–43 (1993)
Anderson, C., Hu, W., Nishimoto, M., Sweeney, A., Doucette, D., Peck, G.: Scalability in the xFS File System. In: USENIX Annual Technical Conference (1996)
Schmuck, F., Haskin, R.: GPFS: A Shared-Disk File System for Large Computing Clusters. In: Proc. File and Storage Technologies Conf (FAST 2002), Janaury 2002, pp. 231–244 (2002)
Preslan, K., O’ Keefe, M., Soltis, S., Erickson, G., Ruwart, T.: The Global File System: A System for Shared Disk Storage. IEEE Trans. on Parallel and Distributed Systems (October 1997)
Zhu, Y., Jiang, H., Qin, X., Feng, D., Swanson, D.R.: Design, Implementation and Performance Evaluation of a Cost-Effective, Fault-Tolerance Parallel Virtual File System. In: Int’l Workshop on Storage Network Architecture and Parallel I/O (September 2003)
Hwang, K., Jin, H., Ho, R.S.C.: Orthogonal Striping and Mirroring in Distributed RAID for I/O-Centric Cluster Computing. IEEE Trans. on Parallel and Distributed Systems 13(1), 26–44 (2002)
Moyer, S.A., Sunderam, V.S.: PIOUS: A Scalable Parallel I/O System for Distributed Computing Environments. In: Proc. Scalable High-Performance Computing Conf., pp. 71–78 (1994)
Lee, E.K., Thekkath, C.A.: Petal: Distributed Virtual Disks. In: Proc. Seventh Int’l Conf. Architectural Support for Programming Languages and Operating Systems, pp. 84–92 (1996)
Iozone Filesystem Benchmark, available at http://www.iozone.org/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hung, SK., Hsu, Y. (2005). Modularized Redundant Parallel Virtual File System. In: Srikanthan, T., Xue, J., Chang, CH. (eds) Advances in Computer Systems Architecture. ACSAC 2005. Lecture Notes in Computer Science, vol 3740. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11572961_16
Download citation
DOI: https://doi.org/10.1007/11572961_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29643-0
Online ISBN: 978-3-540-32108-8
eBook Packages: Computer ScienceComputer Science (R0)