Architecture-independent request-scheduling with tight waiting-time estimations

Abstract

In the course of the last few years, the user's interaction with parallel computer-systems has changed. A continuous growth in the number of interactive HPC-applications can be observed. When considering partitionable MPP-systems with exclusive usage of the physically separated regions, issues like the average waiting-time become more dominant for the users than the total system-throughput.

In this paper, we focus on the problem of scheduling an arbitrary mixture of resource-requests for batch and interactive applications in an architecture-independent manner. To help users plan their daily work tight waiting-time estimations are indispensable. However, the resulting scheduling problem interferes with the problem of mapping requests onto certain MPP-architectures to reduce their internal fragmentations.

We will show that this conflict can be alleviated by a distributed proververifier methodology. At first, we will introduce the distributed resource-management software CCS with its architecture-independent scheduling method. The message-based approach presented is used to verify the pre-calculated schedules with help of the system-dependent mapping instances. Simulations with the accounting data of our center have shown that tight waiting-time estimations can be made while the architecture-independent scheduling approach is still preserved. We will show that by using this methodology the mean error-value of the predicted waiting-time can be reduced by 76 %. Finally, we will discuss the impact of such a distributed resource-management system on the metacomputing challenge.

This work was partly supported by the German ”Ministerium für Wissenschaft und Forschung” and the research cooperation NRW-Metacomputing. Further grants were provided by the ”Stifterverband für die Deutsche Wissenschaft.”