Abstract
Traditional parallel schedulers running on cluster supercomputers support only static scheduling, where the number of processors allocated to an application remains fixed throughout the execution of the job. This results in under-utilization of idle system resources thereby decreasing overall system throughput. In our research, we have developed a prototype framework called ReSHAPE, which supports dynamic resizing of parallel MPI applications executing on distributed memory platforms. The resizing library in ReSHAPE includes support for releasing and acquiring processors and efficiently redistributing application state to a new set of processors. In this paper, we derive an algorithm for redistributing two-dimensional block-cyclic arrays from P to Q processors, organized as 2-D processor grids. The algorithm ensures a contention-free communication schedule for data redistribution if P r ≤ Q r and P c ≤ Q c . In other cases, the algorithm implements circular row and column shifts on the communication schedule to minimize node contention.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blackford, L.S., Choi, J., Cleary, A., D’Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammerling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C.: ScaLAPACK User’s Guide, SIAM, Philadelphia (1997)
Chung, Y.C., Hsu, C.H., Bai, S.W.: A Basic-Cycle Calculation Technique for Efficient Dynamic Data Redistribution. IEEE Trans. Parallel Distrib. Syst. 9, 359–377 (1998)
Desprez, F., Dongarra, J., Petitet, A., Randriamaro, C., Robert, Y.: Scheduling Block-Cyclic Array Redistribution. In: Proceedings of the Conference ParCo 1997, vol. 12, pp. 227–234 (1998)
Guo, M., Pan, Y.: Improving communication scheduling for array redistribution. J. Parallel Distrib. Comput. 65, 553–563 (2005)
Hsu, C.H., Chung, Y.C., Yang, D.L., Dow, C.R.: A Generalized Processor Mapping Technique for Array Redistribution. IEEE Trans. Parallel Distrib. Syst. 12, 743–757 (2001)
Kalns, E.T., Ni, L.M.: Processor Mapping Techniques Toward Efficient Data Redistribution. IEEE Trans. Parallel Distrib. Syst. 6, 1234–1247 (1995)
Kaushik, S.D., Huang, C.H., Johnson, R.W., Sadayappan, P.: An approach to communication-efficient data redistribution. In: ICS 1994: Proceedings of the 8th international conference on Supercomputing, pp. 364–373 (1994)
Lim, Y.W., Bhat, P.B., Prasanna, V.K.: Efficient Algorithms for Block-Cyclic Redistribution of Arrays. In: SPDP 1996. Proceedings of the 8th IEEE Symposium on Parallel and Distributed Processing, p. 74. IEEE Computer Society Press, Los Alamitos (1996)
Ramaswamy, S., Simons, B., Banerjee, P.: Optimizations for efficient array redistribution on distributed memory multicomputers. Journal of Parallel Distributed Computing 38, 217–228 (1996)
Thakur, R., Choudhary, A., Fox, G.: Runtime Array Redistribution in HPF Programs. In: Scalable High Performance Computing Conference, Knoxville, Tenn., pp. 309–316 (1994)
Thakur, R., Choudhary, A., Ramanujam, J.: Efficient Algorithms for Array Redistribution. IEEE Trans. Parallel Distrib. Syst. 7, 587–594 (1996)
Walker, D.W., Otto, S.W.: Redistribution of block-cyclic data distributions using MPI. Concurrency: Practice and Experience 8, 707–728 (1996)
Hsu, C.H., Bai, S.W., Chung, Y.C., Yang, C.S.: A Generalized Basic-Cycle Calculation Method for Efficient Array Redistribution. IEEE Trans. Parallel Distrib. Syst. 11, 1201–1216 (2000)
Prylli, L., Tourancheau, B.: Efficient Block-Cyclic Data Redistribution. In: Fraigniaud, P., Mignotte, A., Bougé, L., Robert, Y. (eds.) Euro-Par 1996. LNCS, vol. 1123, pp. 155–164. Springer, Heidelberg (1996)
Lim, Y.W., Park, N., Prasanna, V.K.: Efficient Algorithms for Multi-dimensional Block-Cyclic Redistribution of Arrays. In: ICPP 1997: Proceedings of the international Conference on Parallel Processing, pp. 234–241 (1997)
Park, N., Prasanna, V.K., Raghavendra, C.S.: Efficient Algorithms for Block-Cyclic Array Redistribution Between Processor Sets. IEEE Transactions on Parallel and Distributed Systems 10, 1217–1240 (1999)
Sudarsan, R., Ribbens, C.J.: ReSHAPE: A Framework for Dynamic Resizing and Scheduling of Homogeneous Applications in a Parallel Environment. Technical Report cs.DC/0703137, Computing Research Repository(CoRR) (2007)
Dongarra, J., Whaley, R.C.: A user’s guide to the BLACS v1.1. Technical Report CS-95-281, Computer Science Department, University of Tennesse, Knoxville, TN (also LAPACK Working Note #94) (1997)
MPICH2 v1.03 (2005), available from: http://www-unix.mcs.anl.gov/mpi/mpich
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sudarsan, R., Ribbens, C.J. (2007). Efficient Multidimensional Data Redistribution for Resizable Parallel Computations . In: Stojmenovic, I., Thulasiram, R.K., Yang, L.T., Jia, W., Guo, M., de Mello, R.F. (eds) Parallel and Distributed Processing and Applications. ISPA 2007. Lecture Notes in Computer Science, vol 4742. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74742-0_19
Download citation
DOI: https://doi.org/10.1007/978-3-540-74742-0_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74741-3
Online ISBN: 978-3-540-74742-0
eBook Packages: Computer ScienceComputer Science (R0)