Plover: Parallel In-Memory Database Logging on Scalable Storage Devices

Abstract

Despite the prevalence of multi-core processors and large main memories, most in-memory databases still universally adopt a centralized ARIES-logging with a single I/O channel, which can be a serious bottleneck. In this paper, we propose a parallel logging mechanism, named $\texttt {Plover}$ for in-memory databases, which utilizes the partial order property of transactions’ dependencies and allows for concurrent logging in scalable storage devices. To further alleviate the performance overheads caused by log partitioning, we present a workload-aware log partitioning scheme to minimize the number of cross-partition transactions, while maintaining load balance. As such, $\texttt {Plover}$ can scale well with the increasing number of storage devices and extensive experiments show that $\texttt {Plover}$ with workload-aware partitioning can achieve  $2\times$ speedup over a centralized logging scheme and more than 42% over $\texttt {Plover}$ with random partitioning.