Abstract
In a large-scale cluster system with many applications running on it, cluster-wide I/O access workload disparity and disk saturation on only some storage servers have been the severe performance bottleneck that deteriorates the system I/O performance. As a result, the system response time will increase and the throughput of the system will decrease drastically. In this paper, we present a load-aware data placement policy that will distribute data across the storage servers based on the load of each server and automatically migrate data from heavily-loaded servers to lightly-loaded servers. This policy is adaptive and self-managing. It operates without any prior knowledge of application access workload characteristics or the capabilities of storage servers. It can make full use of the aggregate disk bandwidth of all storage servers efficiently. Performance evaluation shows that our policy will improve the aggregate I/O bandwidth by 10%-20% compared with random data placement policy especially under mixed workloads.
Chapter PDF
References
DOE National Nuclear Security Administration and the DOE National Security Agency. SGS file system (April 2001)
Kramer, W.T.C., Shoshani, A., Agarwal, D.A., Draney, B.R., Jin, G., Butler, G.F., Hules, J.A.: Deep scientific computing requires deep data. IBM J. Res. Dev. 48(2), 209–232 (2004)
Wang, F., Xin, Q., Hong, B., Brandt, S.A., Miller, E.L., Long, D.D.E., McLarty, T.T.: File system workload analysis for large scale scientific computing applications. In: Proceedings of the 21st IEEE / 12th NASA Goddard Conference on Mass Storage Systems and Technologies, College Park, MD (April 2004)
Leung, A.W., Pasupathy, S., Goodson, G., Miller, E.L.: Measurement and analysis of large scale network file system workloads. In: Proceedings of the 2008 USENIX Annual Technical Conference (June 2008)
Evans, K.M., Kuenning, G.H.: Irregularities in file-size distributions. In: Proceedings of the 2nd International Symposiusm on Performance Evaluation of Computer and Telecommunication Systems, SPECTS (July 2002)
McKusick, M., Joy, W., Leffler, S., Fabry, R.: A Fast File System for UNIX. ACM Trans. on Computer Systems 2(3), 181–197 (1984)
Lawrence Livermore National Laboratory. IOR software, http://www.llnl.gov/icc/lc/siop/downloads/download.html
Zhu, Y., Jiang, H., Qin, X., Swanson, D.: A Case Study of Parallel I/O for Biological Sequence Analysis on Linux Clusters. In: Proceedings of Cluster 2003, Hong Kong, December 1-4 (2003)
Karger, D., Lehman, E., Leighton, T., Levine, M., Levin, D., Panigraphy, R.: Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the World Wide Web. In: Proceedings of ACM Symposium on Theory of Computing (STOC 1997), pp. 654–663 (1997)
Stoica, I., Morris, R., Karger, D., Kaashoek, F., Balakrishnan, H.: Chord: A scalable peer-to-peer lookup service for internet applications. In: Proc. of the 2001 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM 2001), San Diego, CA (August 2001)
Choy, D.M., Fagin, R., Stockmeyer, L.: Efficiently extendible mappings for balanced data distribution. Algorithmica 16, 215–232 (1996)
Honicky, R.J., Miller, E.L.: Replication under scalable hashing: A family of algorithms for scalable decentralized data distribution. In: Proceedings of the 18th International Parallel & Distributed Processing Symposium (IPDPS 2004), Santa Fe, NM (April 2004)
Weil, S.A., Brandt, S.A., Miller, E.L., Maltzahn, C.: CRUSH: Controlled, scalable, decentralized placement of replicated data. In: Proc. of the 2006 ACM/IEEE Conference on Supercomputing, Tampa, FL (November 2006)
Brinkmann, A., Salzwedel, K., Scheideler, C.: Efficient, distributed data placement strategies for storage area networks. In: In Proceedings of the 12th ACM Symposium on Parallel Algorithms and Architectures SPAA (2000)
Wu, C., Lau, F.: Load Balancing in Parallel Computers: Theory and Practice. Kluwer Academic Publishers, Boston (1997)
Zhou, S., Wang, J., Zheng, X., Delisle, P.: Utopia: A load-sharing facility for large heterogeneous distributed computing systems. Software Practice and Experience 23(12) (1993)
Zhu, H., Yang, T., Zheng, Q., Watson, D., Ibarra, O.H.: Adaptive load sharing for clustered digital library servers. International Journal on Digital Libraries 2(4) (2000)
Lee, L., Scheauermann, P., Vingralek, R.: File Assignment in Parallel I/O Systems with Minimal Variance of Service time. IEEE Trans. on Computers 49
Xiao, Q., Jiang, H., Zhu, Y., Swanson, D.: Toward load balancing support for I/O intensive parallel jobs in a cluster of workstation. In: Proc. of the 5th IEEE International Conference Cluster Computing, Hong Kong (December 14, 2003)
DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: “Dynamo: Amazon’s Highly Available Key-Value Store”. In: ACM SOSP (2007)
Chang, F., Dean, J., Ghemawat, S., Hsieh, W., Wallach, D., Burrows, M., Chandra, T.: Bigtable: A Distributed Storage System for Structured Data. In: Proc. of OSDI 2006 (2006)
Tang, H., Gulbeden, A., Chu, L.: A Self-Organizing Storage Cluster for Parallel Data-Intensive Applications. In: Proc. of SC 2004, PA (November 2004)
Pedretti, K.T., Casavant, T.L., Roberts, C.A.: Three complementary approaches to parallelization of local BLAST service on workstation clusters. In: Malyshkin, V.E. (ed.) PaCT 1999. LNCS, vol. 1662, Springer, Heidelberg (1999)
Xing, J., Xiong, J., Ma, J., Sun, N.: Main Memory Metadata Server for Large Distributed File Systems. In: GCC 2008 (October 2008)
Gulati, A., Kumar, C., Ahmad, I., Kumar, K.: BASIL: automated IO load balancing across storage devices. In: Proceedings of the 8th USENIX Conference on File and Storage Technologies (FAST 2010), pp. 13–13. USENIX Association, Berkeley, CA, USA (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 IFIP International Federation for Information Processing
About this paper
Cite this paper
Wang, Y., Xing, J., Xiong, J., Meng, D. (2011). A Load-Aware Data Placement Policy on Cluster File System. In: Altman, E., Shi, W. (eds) Network and Parallel Computing. NPC 2011. Lecture Notes in Computer Science, vol 6985. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24403-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-24403-2_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24402-5
Online ISBN: 978-3-642-24403-2
eBook Packages: Computer ScienceComputer Science (R0)