Volunteer Computing on Clusters

Abstract

Clusters typically represent a homogeneous, well maintained pool of high-end computation resources. This makes them particularly attractive for volunteer computing, where unused compute cycles are utilized for scientific guest applications. Cluster nodes are not idle as often as public PCs, but they are frequently underutilized while actively executing parallel applications. Hence, fully exploiting clusters for volunteer computing requires the ability to efficiently and invisibly steal the unused cycles at a fine grain from the currently running host applications, without slowing them down. In this paper we present measurements on a production compute cluster that show long periods of CPU and memory underutilization patterns that could be used to execute guest applications. In our experiments with NAS benchmarks on a small Linux cluster, cycle stealing led to a 3.6% average slowdown of host applications in the best case. This was accompanied by an overall improvement in the system throughput of 38%, when progress of the guest applications was included. We introduce simple guidelines on using clusters for volunteer computing. We also argue for the support of “zero priority” processes in OS schedulers which could virtually eliminate the impact of volunteer computing on host applications.