Optimized Adaptive Hybrid Indexing for In-memory Column Stores

Abstract

Modern applications and databases using dynamic storage environment are characterized by two challenging features: (a) little idle system time to devote in physical design; (b) little priori knowledge about the query and data workload. Traditional approaches to index building and maintenance do not work well in such dynamic environment; while adaptive indexing can be a remedy. An adaptive index is a partially created index. Refinement of the index is conducted during query execution. Database cracking and adaptive merging are two techniques for adaptive indexing. The former is advantageous at initialization, while the latter can converge to its optimal structure with a much faster speed. In this paper, we propose a hybrid approach by combining cracking and adaptive merging. We designed a cost model to measure the cost of data partition operations. Based on the model, we provide an algorithm to refine adaptive index. Experiments show that our hybrid approach can achieve appropriate performance tradeoff between database cracking and adaptive merging.