Abstract
In systems consisting of multiple clusters of processors interconnected by relatively slow network connections such as our Distributed ASCI Supercomputer (DAS), applications may benefit from the availability of processors in multiple clusters. However, the performance of single-application multicluster execution may be degraded due to the slow wide-area links. In addition, scheduling policies for such systems have to deal with more restrictions than schedulers for single clusters in that every component of a job has to fit in separate clusters. In this paper we present a measurement study of the total runtime of two applications, and of the communication time of one of them, both on single clusters and on multicluster systems. In addition, we perform simulations of several multicluster scheduling policies based on our measurement results. Our results show that in many cases, restricted forms of co-allocation in multiclusters have better performance than not allowing co-allocation at all.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
The Distributed ASCI Supercomputer (DAS), http://www.cs.vu.nl/das2
The Global Grid Forum, http://www.gridforum.org
Foster, I., Kesselman, C. (eds.): The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, San Francisco (1999)
Bucur, A., Epema, D.: The Influence of the Structure and Sizes of Jobs on the Performance of Co-Allocation. In: Feitelson, D.G., Rudolph, L. (eds.) IPDPS-WS 2000 and JSSPP 2000. LNCS, vol. 1911, pp. 154–173. Springer, Heidelberg (2000)
Bucur, A., Epema, D.: The Influence of Communication on the Performance of Co-Allocation. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 66–86. Springer, Heidelberg (2001)
Bucur, A., Epema, D.: Local versus Global Queues with Processor Co-Allocation in Multicluster Systems. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2002. LNCS, vol. 2537, pp. 184–204. Springer, Heidelberg (2002)
Ernemann, C, Hamscher, V., Schwiegelshohn, U., Yahyapour, R., Streit, A.: On Advantages of Grid Computing for Parallel Job Scheduling. In: 2nd IEEE/ACM Int’l Symposium on Cluster Computing and the GRID (CCGrid2002), pp.39–46 (2002)
Bucur, A., Epema, D.: The Maximal Utilization of Processor Co-Allocation in Multicluster Systems. In: Proc. Int’l Parallel and Distributed Processing Symp. (IPDPS) (2003)
Bal, H.E., et al.: The Distributed ASCI Supercomputer Project. ACM Operating Systems Review 34, 76–96 (2000)
Bal, H., Plaat, A., Bakker, M., Dozy, P., Hofman, R.: Optimizing Parallel Applications for Wide-Area Clusters. In: Proc. of the 12th Int’l Parallel Processing Symp., pp. 784–790 (1998)
Kielmann, T., Hofman, R., Bal, H., Plaat, A., Bhoedjang, R.: MagPIe: MPI’s Collective Communication Operations for Clustered Wide Area Systems. In: ACM SIGPLAN Symp. on Principles and Practice of Parallel Programming, 131–140 (1999)
Plaat, A., Bal, H., Hofman, R., Kielmann, T.: Sensitivity of Parallel Applications to Large Differences in Bandwidth and Latency in Two-Layer Interconnects. Future Generation Computer Systems 17, 769–782 (2001)
The Portable Batch System, http://www.openpbs.org
Globus, http://www.globus.org
van Hees, F., van der Steen, A., van Leeuwen, P.: A parallel data assimilation model for oceanographic observations. Concurrency and Computation: Practice and Experience (2003) (to appear)
Kumar, V., Grama, A., Gupta, A., Karypis, G.: Introduction to Parallel Computing. Benjamin/Cummings (1994)
Mesquite Software, Inc.: (The CSIM18 Simulation Engine, User’s Guide)
Bucur, A., Epema, D.: An Evaluation of Processor Co-Allocation for Different System Configurations and Job Structures. In: Proc. of the 14th Symp. on Computer Architecture and High Performance Computing, pp. 195–203. IEEE Computer Society Press, Los Alamitos (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Banen, S., Bucur, A.I.D., Epema, D.H.J. (2003). A Measurement-Based Simulation Study of Processor Co-allocation in Multicluster Systems. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2003. Lecture Notes in Computer Science, vol 2862. Springer, Berlin, Heidelberg. https://doi.org/10.1007/10968987_6
Download citation
DOI: https://doi.org/10.1007/10968987_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20405-3
Online ISBN: 978-3-540-39727-4
eBook Packages: Springer Book Archive