On the Impact of Job Size Variability on Heterogeneity-Aware Load Balancing

Abstract

Load balancing is one of the key components in many distributed systems as it heavily impacts performance and resource utilization. We consider a heterogeneous system where each server belongs to one of K classes and the speed of the server depends on its class. Arriving jobs are immediately dispatched to a server class in a randomized manner, i.e., with probability $p_k$ a job is assigned to class k. Within each class a power of d choices rule is used to select the server that executes the job.

For large systems and exponential job size durations the optimal probabilities $p_k$ to minimize the mean response time can be determined easily via convex optimization. In this paper we develop a mean field model (validated by simulation) to investigate how these optimal probabilities $p_k$ are affected by the higher moments and in particular by the variability of the job size distribution when the service discipline at each server is first-come-first-served. The main insight provided is that optimizing the probabilities $p_k$ based on the higher moments is much more involved and provides only a non negligible gain for very specific system load regions.