Plover: Parallel In-Memory Database Logging on Scalable Storage Devices

  • Huan Zhou
  • Jinwei Guo
  • Ouya Pei
  • Weining QianEmail author
  • Xuan Zhou
  • Aoying Zhou
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10988)


Despite the prevalence of multi-core processors and large main memories, most in-memory databases still universally adopt a centralized ARIES-logging with a single I/O channel, which can be a serious bottleneck. In this paper, we propose a parallel logging mechanism, named \(\texttt {Plover}\) for in-memory databases, which utilizes the partial order property of transactions’ dependencies and allows for concurrent logging in scalable storage devices. To further alleviate the performance overheads caused by log partitioning, we present a workload-aware log partitioning scheme to minimize the number of cross-partition transactions, while maintaining load balance. As such, \(\texttt {Plover}\) can scale well with the increasing number of storage devices and extensive experiments show that \(\texttt {Plover}\) with workload-aware partitioning can achieve  \(2\times \) speedup over a centralized logging scheme and more than 42% over \(\texttt {Plover}\) with random partitioning.


In-memory database Parallel logging Scalability 



This work is partially supported by National High-tech R&D Program (863 Program) under grant number 2015AA015307, National Key Research & Development Program of China (No. 2018YFB1003400), National Science Foundation of China under grant numbers 61702189, 61432006 and 61672232.


  1. 1.
    Mohan, C., Haderle, D., Lindsay, B., Pirahesh, H., Schwarz, P.: ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging. TODS 17(1), 94–162 (1992)CrossRefGoogle Scholar
  2. 2.
    Johnson, R., Pandis, I., Stoica, R., Manos, A.: Aether: a scalable approach to logging. VLDB 3(1–2), 681–692 (2010)Google Scholar
  3. 3.
    Kim, K., Wang, T.Z., Johnson, R., et al.: Ermia: fast memory-optimized database system for heterogeneous workloads. In: SIGMOD, pp. 1675–1687 (2016)Google Scholar
  4. 4.
    Lamport, L.: Time, clocks, and the ordering of events in a distributed system. Commun. ACM 21(7), 558–565 (1978)CrossRefGoogle Scholar
  5. 5.
    Andreev, K., Racke, H.: Balanced graph partitioning. Theor. Comput. Syst. 39(6), 929–939 (2006)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Zheng, W.T., Tu, S.: Fast databases with fast durability and recovery through multicore parallelism. In: OSDI, pp. 465–477 (2014)Google Scholar
  7. 7.
    Huang, J., Schwan, K., Qureshi, M.K.: NVRAM-aware logging in transaction systems. PVLDB 8(4), 389–400 (2014)Google Scholar
  8. 8.
    Wang, T.Z., Johnson, R., Stoica, R., Manos, A.: Scalable logging through emerging non-volatile memory. PVLDB 7(10), 865–876 (2014)Google Scholar
  9. 9.
    Arulraj, J., Perron, M., Pavlo, A.: Write-behind logging. PVLDB 10(4), 337–348 (2016)Google Scholar
  10. 10.
    Jung, H., Han, H., Kang, S.: Scalable database logging for multicores. PVLDB 11(2), 135–148 (2017)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Huan Zhou
    • 1
  • Jinwei Guo
    • 1
  • Ouya Pei
    • 2
  • Weining Qian
    • 1
    Email author
  • Xuan Zhou
    • 1
  • Aoying Zhou
    • 1
  1. 1.School of Data Science and EngineeringEast China Normal UniversityShanghaiChina
  2. 2.Northwestern Polytechnical UniversityXianChina

Personalised recommendations