A Twin-Buffer Scheme for High-Throughput Logging

  • Qingzhong MengEmail author
  • Xuan Zhou
  • Shan Wang
  • Haiyan Huang
  • Xiaoli Liu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10828)


For a transactional database system, the efficiency of logging is usually crucial to its performance. The emergence of new hardware, such as NVM and SSD, eliminated the traditional I/O bottleneck of logging and released the potential of multi-core CPUs. As a result, the parallelism of logging becomes important. We propose a parallel logging subsystem called TwinBuf and implemented it in PostgreSQL. This solution can make better use of multi-core CPUs, and is generally applicable to all kinds of storage devices, such as hard disk, SSD and NVM. TwinBuf adopts per-thread logging slots to parallelize logging, and a twin-log-buffer mechanism to make sure that logging can be performed in a non-stop manner. It performs group commit to minimize the persistence overheads. Experimental evaluation was conducted to demonstrate its advantages.


  1. 1.
  2. 2.
  3. 3.
    Alomari, M., Cahill, M., Fekete, A., Rohm, U.: The cost of serializability on platforms that use snapshot isolation. In: IEEE 24th International Conference on Data Engineering, 2008. ICDE 2008, pp. 576–585. IEEE (2008)Google Scholar
  4. 4.
    Chen, S.: Flashlogging: exploiting flash devices for synchronous logging performance. In: Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, pp. 73–86. ACM (2009)Google Scholar
  5. 5.
    Fang, R., Hsiao, H.-I., He, B., Mohan, C., Wang, Y.: High performance database logging using storage class memory. In: 2011 IEEE 27th International Conference on Data Engineering (ICDE), pp. 1221–1231. IEEE (2011)Google Scholar
  6. 6.
    Gao, S., Xu, J., He, B., Choi, B., Hu, H.: PCMLogging: reducing transaction logging overhead with PCM. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, pp. 2401–2404. ACM (2011)Google Scholar
  7. 7.
    Helland, P., Sammer, H., Lyon, J., Carr, R., Garrett, P., Reuter, A.: Group commit timers and high volume transaction systems. In: Gawlick, D., Haynie, M., Reuter, A. (eds.) HPTS 1987. LNCS, vol. 359, pp. 301–329. Springer, Heidelberg (1989). Scholar
  8. 8.
    Huang, J., Schwan, K., Qureshi, M.K.: NVRAM-aware logging in transaction systems. Proc. VLDB Endow. 8(4), 389–400 (2014)CrossRefGoogle Scholar
  9. 9.
    Lee, S.-W., Moon, B., Park, C., Kim, J.-M., Kim, S.-W.: A case for flash memory SSD in enterprise database applications. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1075–1086. ACM (2008)Google Scholar
  10. 10.
    Mohan, C., Haderle, D., Lindsay, B., Pirahesh, H., Schwarz, P.: Aries: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging. ACM Trans. Database Syst. (TODS) 17(1), 94–162 (1992)CrossRefGoogle Scholar
  11. 11.
    Pelley, S., Wenisch, T.F., Gold, B.T., Bridge, B.: Storage management in the NVRAM era. Proc. VLDB Endow. 7(2), 121–132 (2013)CrossRefGoogle Scholar
  12. 12.
    Rafii, A., DuBois, D.: Performance tradeoffs of group commit logging. In: International CMG Conference, pp. 164–176 (1989)Google Scholar
  13. 13.
    Son, Y., Kang, H., Yeom, H.Y., Han, H.: A log-structured buffer for database systems using non-volatile memory. In: Proceedings of the Symposium on Applied Computing, pp. 880–886. ACM (2017)Google Scholar
  14. 14.
    Wang, T., Johnson, R.: Scalable logging through emerging non-volatile memory. Proc. VLDB Endow. 7(10), 865–876 (2014)CrossRefGoogle Scholar
  15. 15.
    Xiao-Feng, L.Z.-P.M., Da, Z.: HV-recovery: a high efficient recovery technique for flash-based database. Chin. J. Comput. 12, 007 (2010)Google Scholar
  16. 16.
    Xu, J., Swanson, S.: NOVA: a log-structured file system for hybrid volatile/non-volatile main memories. In: FAST, pp. 323–338 (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Qingzhong Meng
    • 1
    Email author
  • Xuan Zhou
    • 2
  • Shan Wang
    • 1
  • Haiyan Huang
    • 3
  • Xiaoli Liu
    • 3
  1. 1.MOE Key Laboratory of DEKERenmin University of ChinaBeijingChina
  2. 2.School of Data Science & EngineeringEast China Normal UniversityShanghaiChina
  3. 3.Huawei Technologies Co., Ltd.ShenzhenChina

Personalised recommendations