Keyword Oriented Bitmap Join Index for In-Memory Analytical Processing

Abstract

Nowadays computers are equipped with multicore processors and large RAM to support high performance processing. In-memory analytical processing and just-in-time data warehousing have become realistic for various enterprises. An analytical query normally requires a small proportion of ‘hot’ data, usually defined by a set of keywords, instead of the entire data set, which involves large volume table scan and costly star joins. Therefore, identifying frequent keywords to retrieve hot data can dramatically reduce the cost of full table scan or star-join. In this paper, we propose a keyword oriented bitmap join index to improve the space efficiency and performance of in-memory data warehouse. Keyword oriented bitmap join index is a global bitmap join index for the entire data warehouse, as opposed to conventional bitmap join indexes which are indicated for specified attributes. With our index, star-join is first converted into keyword search and bitmap combination. The resulting bitmap filters are then employed to filter records. Through the filtering by bitmaps, a star-join is converted into positional scan on the fact table and additional dimension filtering. Both memory bandwidth and analytical performance can then be improved.