Abstract
One of the major challenges in parallel computing over large scale clusters is fast and scalable process startup, which typically can be divided into two phases: process initiation and connection setup. In this paper, we characterize the startup of MPI programs in InfiniBand clusters and identify two startup scalability issues: serialized process initiation in the initiation phase and high communication overhead in the connection setup phase. To reduce the connection setup time, we have developed one approach with data reassembly to reduce data volume, and another with a bootstrap channel to parallelize the communication. Furthermore, a process management framework, Multi-Purpose Daemons (MPD) system is exploited to speed up process initiation. Our experimental results show that job startup time has been improved by more than 4 times for 128-process jobs, and the improvement can be more than two orders of magnitude for 2048-process jobs as suggested by our analytical models.
This research is supported in part by a DOE grant #DE-FC02-01ER25506, NSF Grants #CCR-0204429 and #CCR-0311542, and a grant from Los Alamos National Laboratory.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Baker, M., Fox, G., Yau, H.: Cluster Computing Review (November 1995)
Brightwell, R., Fisk, L.A.: Scalable parallel application launch on Cplant. In: Proceedings of Supercomputing, 2001, Denver, Colorado (November 2001)
Butler, R., Gropp, W., Lusk, E.: Components and interfaces of a process management system for parallel programs. Parallel Computing 27(11), 1417–1429 (2001)
Frachtenberg, E., Petrini, F., Fernandez, J., Pakin, S., Coll, S.: STORM: Lightning-Fast Resource Management. In: Proceedings of the Supercomputing 2002, Baltimore, MD (November 2002)
Gropp, W., Lusk, E., Doss, N., Skjellum, A.: A High-Performance, Portable Implementation of the MPI Message Passing Interface Standard. Parallel Computing 22(6), 789–828 (1996)
Hendriks, E.: Bproc: The beowulf distributed process space. In: Proceedings of the International Conference on Supercomputing, New York (June 2002)
IBM. Using the Parallel Operating Environment, Version 4, Release 1 (2004)
Infiniband Trade Association (2000), http://www.infinibandta.org
Jette, M., Grondona, M.: SLURM: Simple Linux Utility for Resource Management. In: Proceedings of the International Conference on Linux Clusters, San Jose, CA (June 2003)
Kavas, A., Er-El, D., Feitelson, D.G.: Using Multicast to Pre-Load Jobs on the ParPar Cluster. Parallel Computing 27(3), 315–327 (2001)
Lawrence Berkeley National Laboratory. MVICH: MPI for Virtual Interface Architecture (August 2001), http://www.nersc.gov/research/FTG/mvich/index.html
Message Passing Interface Forum: MPI: A message-passing interface standard. The International Journal of Supercomputer Applications 8(3–4), 159–416 (1994)
Network-Based Computing Laboratory. MVAPICH: MPI for InfiniBand on VAPI Layer, http://nowlab.cis.ohio-state.edu/projects/mpi-iba/index.html
OpenPBS Documentation (2004), http://www.openpbs.org/docs.html
TOP 500 Supercomputers (2003), http://www.top500.org/
Wu, J., Liu, J., Wyckoff, P., Panda, D.K.: Impact of On-Demand Connection Management in MPI over VIA. In: Proceedings of the International Conference on Cluster Computing (2002)
Yu, W., Wu, J., Panda, D.K.: Fast and Scalable Startup of MPI Programs in InfiniBand Clusters. Number OSU-CISRC-5/04-TR33, Columbus, OH 43210 (May 2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yu, W., Wu, J., Panda, D.K. (2004). Fast and Scalable Startup of MPI Programs in InfiniBand Clusters. In: Bougé, L., Prasanna, V.K. (eds) High Performance Computing - HiPC 2004. HiPC 2004. Lecture Notes in Computer Science, vol 3296. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30474-6_47
Download citation
DOI: https://doi.org/10.1007/978-3-540-30474-6_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24129-4
Online ISBN: 978-3-540-30474-6
eBook Packages: Computer ScienceComputer Science (R0)