Heuristic file sorted assignment algorithm of parallel I/O on cluster computing system
- 21 Downloads
A new file assignment strategy of parallel I/O, which is named heuristic file sorted assignment algorithm was proposed on cluster computing system. Based on the load balancing, it assigns the files to the same disk according to the similar service time. Firstly, the files were sorted and stored at the set I in descending order in terms of their service time, then one disk of cluster node was selected randomly when the files were to be assigned, and at last the continuous files were taken orderly from the set I to the disk until the disk reached its load maximum. The experimental results show that the new strategy improves the performance by 20.2% when the load of the system is light and by 31.6% when the load is heavy. And the higher the data access rate, the more evident the improvement of the performance obtained by the heuristic file sorted assignment algorithm.
Key wordscluster computing parallel I/O file sorted assignment variance of service time
Unable to display preview. Download preview PDF.
- LONG Xiang, LI Zhong-ze, GAO Xian-peng, et al. A new method to improve the I/O efficiency on network of workstations[J]. Journal of Computer Research and Development, 2000, 37(6): 650–656. (in Chinese)Google Scholar
- Rajkumar B. High Performance Cluster Computing: Architectures and Systems (Vol. 1)[M]. New Jersey: Prentice Hall PTR Inc, 1999.Google Scholar
- Bell K, Chien A, Lauria M. A high-performance cluster storage server[A]. Proceeding of the 11th IEEE International Symposium on High Performance Distributed Computing[C]. Edinburgh, Scotland, 2002.Google Scholar
- Ma X S, Jiao X M, Campbell M, et al. Flexible and efficient parallel I/O for large-scale multi-component simulations[A]. Proceedings of the International Parallel and Distributed Processing Symposium[C]. New Mexico, 2003.Google Scholar
- Venugopal C R, Rao S S S P. Impact of delays in parallel I/O system: an empirical study[A]. Proceedings of the High Performance Distributed Computing [C]. New York, 1996.Google Scholar
- Copeland G, Alexander W, Bougher E, et al. Data placement in bubba[A]. Proceeding ACM SIGMOD Int’l Conf Management of Data[C]. Los Angeles, 1988.Google Scholar
- Apon A W, Wolinski P D, Amerson G M. Sensitivity of cluster file system access to I/O server selection [A]. Proceeding of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid[C]. Berlin, 2002.Google Scholar
- SUN Jian-hua, JIN Hai, CHEN Hao, et al. Server scheduling scheme for asynchronous cluster video server [A]. Proceedings of the 17th International Conference on Advanced Information Networking and Applications[C]. Xi’an, 2003.Google Scholar
- Ching A, Choudhary A, Coloma K, et al. Noncontinuous I/O accesses through MPI-IO[A]. Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid [C]. Tokyo, 2003.Google Scholar
- ZHOU Xin-rong, WEI Tong. A greedy I/O scheduling method in the storage system of clusters[A]. Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid[C]. Tokyo, 2003.Google Scholar
- Perez J M, Garcia F, Carretero J, et al. Data allocation and load balancing for heterogeneous cluster storage systems[A]. Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid[C]. Tokyo, 2003.Google Scholar