Abstract
Emerging non-volatile memory technologies (NVM) offer fast and byte-addressable access, allowing to rethink the durability mechanisms of in-memory databases. In this paper, we present Hyrise-NV, a database storage engine that maintains table and index structures on NVM. Our architecture updates the database state and index structures transactionally consistent on NVM using multi-version data structures, allowing to instantly recover databases independent of their size. For index structures, we present nvBTree using multi-versioning to provide failure-atomic tree updates on NVM. We evaluate Hyrise-NV both on DRAM and with hardware-based emulation of NVM using the TPC-C benchmark. Hyrise-NV recovers databases independent of their size, allowing the recovery of a table with 10 million rows in less than 100 ms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Source available at https://github.com/hyrise/hyrise.
- 2.
STX B\(^+\)-Tree: http://idlebox.net/2007/stx-btree/.
- 3.
References
Chatzistergiou, A., et al.: REWIND: recovery write-ahead system for in-memory non-volatile data-structures. In: VLDB (2015)
Chen, S., Jin, Q.: Persistent B+-trees in non-volatile main memory. In: VLDB (2015)
Chen, S., et al.: Rethinking database algorithms for phase change memory. In: CIDR (2011)
Coburn, J., et al.: NV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories. SIGPLAN 46, 105–118 (2011)
Condit, J., et al.: Better I/O through byte-addressable, persistent memory. In: SOSP (2009)
DeBrabant, J., et al.: A prolegomenon on OLTP database systems for non-volatile memory. In: VLDB (2014)
Dulloor, S.R., et al.: System software for persistent memory. In: EuroSys (2014)
Fang, R., et al.: High performance database logging using storage class memory. In: ICDE (2011)
Faust, M., et al.: Fast lookups for in-memory column stores: group-key indices, lookup and maintenance. In: ADMS (2012)
Grund, M., et al.: HYRISE—a main memory hybrid storage engine. In: VLDB (2010)
Höppner, B., et al.: An approach for hybrid-memory scaling columnar in-memory databases. In: ADMS (2014)
Kallman, R., et al.: H-store: a high-performance, distributed main memory transaction processing system. In: VLDB (2008)
Kemper, A., Neumann, T.: HyPer: a hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. In: ICDE (2011)
Krüger, J., et al.: Fast updates on read-optimized databases using multi-core CPUs. VLDB 5, 61–72 (2011)
Malviya, N., et al.: Rethinking main memory OLTP recovery. In: ICDE (2014)
Mohan, C., et al.: ARIES: a transaction recovery method supporting fine- granularity locking and partial rollbacks using write-ahead logging. In: TODS (1998)
Narayanan, D., Hodson, O.: Whole-system persistence. In: ASPLOS (2012)
Ongaro, D., et al.: Fast crash recovery in RAMCloud. In: SOSP (2011)
Oukid, I., et al.: Instant recovery for main-memory databases. In: CIDR (2015)
Pelley, S., et al.: Storage management in the NVRAM era. In: VLDB (2013)
Plattner, H., et al.: The impact of columnar in-memory databases on enterprise systems. In: VLDB (2014)
Raman, V., et al.: DB2 with BLU acceleration: so much more than just a column store. In: VLDB (2013)
Schwalb, D., et al.: Efficient transaction processing for hyrise in mixed workload environments. In: IMDM (2014)
Schwalb, D., et al.: NVC-hashmap: a persistent and concurrent hashmap for non-volatile memories. In: IMDM (2015)
Schwalb, D., et al.: nvm malloc: memory allocation for NVRAM. In: ADMS (2015)
Stonebraker, M., et al.: C-Store: a column-oriented DBMS. In: VLDB (2005)
Thatte, S.M.: Persistent memory: a storage system for object-oriented databases. In: OODS (1991)
Tianzheng, W., Ryan, J.: Scalable logging through emerging non-volatile memory. In: VLDB (2014)
Venkataraman, S., et al.: Consistent and durable data structures for non-volatile byte-addressable memory. In: FAST (2011)
Volos, H., et al.: Mnemosyne: lightweight persistent memory. In: ASPLOS (2011)
Wang, C., et al.: NVMalloc: exposing an aggregate SSD store as a memory partition in extreme-scale machines. In: IPDPS (2012)
Wu, M., Zwaenepoel, W.: eNVy: a non-volatile, main memory storage system. In: ASPLOS (1994)
Wust, J., et al.: Efficient logging for enterprise workloads on column-oriented in-memory databases. In: CIKM (2012)
Yang, J., et al.: NV-Tree: reducing consistency cost for NVM-based single level systems. In: FAST (2015)
Zhang, Y., et al.: Mojim: a reliable and highly-available non-volatile memory system. In: ASPLOS (2015)
Acknowledgments
We thank Konrad Büker, Jürgen Schrage and Ahmad Waizy from Fujitsu and Rami Akkad, Bernhard Höppner and Jürgen Müller from the SAP Innovation Center Potsdam for their support and hardware access. We also thank Subramanya Dulloor from Intel Labs for access to the PMEP emulator.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Schwalb, D. et al. (2016). Hyrise-NV: Instant Recovery for In-Memory Databases Using Non-Volatile Memory. In: Navathe, S., Wu, W., Shekhar, S., Du, X., Wang, S., Xiong, H. (eds) Database Systems for Advanced Applications. DASFAA 2016. Lecture Notes in Computer Science(), vol 9643. Springer, Cham. https://doi.org/10.1007/978-3-319-32049-6_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-32049-6_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-32048-9
Online ISBN: 978-3-319-32049-6
eBook Packages: Computer ScienceComputer Science (R0)