Optimizing Write Performance for Read Optimized Databases

  • Jens Krueger
  • Martin Grund
  • Christian Tinnefeld
  • Hasso Plattner
  • Alexander Zeier
  • Franz Faerber
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5982)


Compression in column-oriented databases has been proven to offer both performance enhancements and reductions in storage consumption. This is especially true for read access as compressed data can directly be processed for query execution.Nevertheless, compression happens to be disadvantageous when it comes to write access due to unavoidable re-compression: write-access requires significantly more data to be read than involved in the particular operation, more tuples may have to be modified depending on the compression algorithm, and table-level locks have to be acquired instead of row-level locks as long as no second version of the data is stored. As an effect the duration of a single modification — both insert and update — limits both throughput and response time significantly. In this paper, we propose to use an additional write-optimized buffer to maintain the delta that in conjunction with the compressed main store represents the current state of the data. This buffer facilitates an uncompressed, column-oriented data structure. To address the mentioned disadvantages of data compression, we trade write-performance for query-performance and memory consumption by using the buffer as an intermediate storage for several modifications which are then populated as a bulk in a merge operation. Hereby, the overhead created by one single re-compression is shared among all recent modifications. We evaluated our implementation inside SAP’s in memory column store. We then analyze the different parameters influencing the merge process, and make a complexity analysis. Finally, we show optimizations regarding resource consumption and merge duration.


Query Execution Enterprise Application Document Vector Sort Order OLAP Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abadi, D., Madden, S., Ferreira, M.: Integrating compression and execution in column-oriented database systems. In: SIGMOD 2006, pp. 671–682. ACM, New York (2006)CrossRefGoogle Scholar
  2. 2.
    Abadi, D.J.: Query Execution in Column-Oriented Database Systems. PhD thesisGoogle Scholar
  3. 3.
    Boncz, P.A., Manegold, S., Kersten, M.L.: Database architecture optimized for the new bottleneck: Memory access. In: VLDB, pp. 54–65 (1999)Google Scholar
  4. 4.
    Boncz, P.A., Zukowski, M., Nes, N.: Monetdb/x100: Hyper-pipelining query execution. In: CIDR, pp. 225–237 (2005)Google Scholar
  5. 5.
    Brown, K.P., Mehta, M., Carey, M.J., Livny, M.: Towards automated performance tuning for complex workloads. In: VLDB, pp. 72–84 (1994)Google Scholar
  6. 6.
    Chen, Z., Gehrke, J., Korn, F.: Query optimization in compressed database systems. In: SIGMOD 2001, pp. 271–282. ACM, New York (2001)CrossRefGoogle Scholar
  7. 7.
    Copeland, G.P., Khoshafian, S.: A decomposition storage model. In: SIGMOD Conference, pp. 268–279 (1985)Google Scholar
  8. 8.
    French, C.D.: “one size fits all” database architectures do not work for DDS. In: SIGMOD Conference, pp. 449–450 (1995)Google Scholar
  9. 9.
    French, C.D.: Teaching an OLTP database kernel advanced data warehousing techniques. In: ICDE, pp. 194–198 (1997)Google Scholar
  10. 10.
    Harizopoulos, S., Liang, V., Abadi, D.J., Madden, S.: Performance tradeoffs in read-optimized databases. In: VLDB, pp. 487–498 (2006)Google Scholar
  11. 11.
    Holloway, A.L., DeWitt, D.J.: Read-optimized databases, in depth. PVLDB 1(1), 502–513 (2008)Google Scholar
  12. 12.
    Legler, T., Lehner, W., Ross, A.: Data mining with the SAP NetWeaver BI accelerator. In: VLDB 2006, pp. 1059–1068. VLDB Endowment (2006)Google Scholar
  13. 13.
    Pang, H., Carey, M.J., Livny, M.: Multiclass query scheduling in real-time database systems. IEEE Trans. Knowl. Data Eng. 7(4), 533–551 (1995)CrossRefGoogle Scholar
  14. 14.
    Plattner, H.: A common database approach for OLTP and OLAP using an in-memory column database. In: SIGMOD Conference, pp. 1–2 (2009)Google Scholar
  15. 15.
    Ramamurthy, R., DeWitt, D.J., Su, Q.: A case for fractured mirrors. In: VLDB, pp. 430–441 (2002)Google Scholar
  16. 16.
    Rao, J., Ross, K.A.: Making b\(^{\mbox{+}}\)-trees cache conscious in main memory. In: SIGMOD Conference, pp. 475–486 (2000)Google Scholar
  17. 17.
    Rappaport, R.L.: File structure design to facilitate on-line instantaneous updating. In: SIGMOD 1975, pp. 1–14. ACM, New York (1975)CrossRefGoogle Scholar
  18. 18.
    Severance, D.G., Lohman, G.M.: Differential files: their application to the maintenance of large databases. ACM Trans. Database Syst. 1(3), 256–267 (1976)CrossRefGoogle Scholar
  19. 19.
    Stonebraker, M., Abadi, D.J., Batkin, A., Chen, X., Cherniack, M., Ferreira, M., Lau, E., Lin, A., Madden, S., O’Neil, E.J., O’Neil, P.E., Rasin, A., Tran, N., Zdonik, S.B.: C-store: A column-oriented dbms. In: VLDB, pp. 553–564 (2005)Google Scholar
  20. 20.
    Westmann, T., Kossmann, D., Helmer, S., Moerkotte, G.: The implementation and performance of compressed databases. SIGMOD Rec. 29(3), 55–67 (2000)CrossRefGoogle Scholar
  21. 21.
    Willhalm, T., Popovici, N., Boshmaf, Y., Plattner, H., Zeier, A., Schaffner, J.: Simd-scan: Ultra fast in-memory table scan using on-chip vector processing units. PVLDB 2(1), 385–394 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Jens Krueger
    • 1
  • Martin Grund
    • 1
  • Christian Tinnefeld
    • 1
  • Hasso Plattner
    • 1
  • Alexander Zeier
    • 1
  • Franz Faerber
    • 2
  1. 1.Hasso–Plattner–InstitutPotsdamGermany
  2. 2.SAP AGWalldorf

Personalised recommendations