CARP: Cost Effective Load-Balancing Approach for Range-Partitioned Data

Abstract

One of the important issues in range partitioning schemes is data skew. Tuples distribution across nodes may be skewed (some nodes have many tuples, while others may have fewer tuples). Processing skewed data not only slows down the response time, but also generates hot nodes. In such a situation, data may need to be moved from the most-loaded partitions to the least-loaded ones in order to achieve storage balancing requirements. Early works from the State-of-The-Art focused on achieving load balancing. However, today’s works focus on reducing the load balancing cost. This latter involves reducing the cost of maintaining partition statistics. In this context, we propose to improve one of the best load balancing work, that is the one of Ganesan et al., to reduce the cost of maintaining the statistics of load balancing. We introduce the concept of fuzzy system image. Both nodes and clients have approximate information about the load distribution. They can nevertheless locate any data with almost the same efficiency as using exact partition statistics. Furthermore, maintaining load distribution statistics do not require exchanging additional messages as opposed to the cost of efficient solutions from the State-of-The-Art (which requires at least $\mathbb {O}(\log {n})$ messages).