Abstract
In this paper we present a novel disk-based distributed column-store, describe its architecture and discuss a number of technical solutions. Our system is essentially a query engine which was written completely from scratch. It is aimed for shared-nothing environments and supports different forms of parallel query processing.
Query processing in PosDB is organized according to the classic Volcano pull-based model which is adapted for the column-store case. Currently, we support late materialization only, and therefore employ a join index data structure to represent positional information. In our system query plan can consist of both positional and value operators. PosDB has about a dozen of core operators among which several variants of selections and joins, aggregation. We also have several operators that ensure intra-query parallelism and operators for network interoperability. In its current state the system is fully capable of processing the Star Schema Benchmark in a local and distributed environment.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
O’Neil, P.E., O’Neil, E.J., Chen, X.: The Star Schema Benchmark (SSB) (2009). http://www.cs.umb.edu/~poneil/StarSchemaB.PDF. Accessed 20 July 2012
Google Supersonic Library (2017). https://code.google.com/archive/p/supersonic/. Accessed 12 February 2017
Abadi, D., Boncz, P., Harizopoulos, S.: The Design and Implementation of Modern Column-Oriented Database Systems. Now Publishers Inc., Hanover, massachusetts (2013)
Abadi, D.J., Boncz, P.A., Harizopoulos, S.: Column-oriented database systems. Proc. VLDB Endow. 2(2), 1664–1665 (2009)
Arulraj, J., Pavlo, A., Menon, P.: Bridging the Archipelago between row-stores and column-stores for hybrid workloads. In: Proceedings of the 2016 International Conference on Management of Data, SIGMOD 2016, pp. 583–598 (2016)
Chernishev, G.: Physical design approaches for column-stores. SPIIRAS Proceedings 30, 204–222 (2013)
Chernishev, G.: Towards self-management in a distributed column-store system. In: Morzy, T., Valduriez, P., Bellatreche, L. (eds.) ADBIS 2015. CCIS, vol. 539, pp. 97–107. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23201-0_12
Chernishev, G.: The design of an adaptive column-store system. J. Big Data 4(1), 21 (2017)
Chernishev, G., Galaktionov, V., Grigorev, V., Klyuchikov, E., Smirnov, K.: A study of PosDB performance in a distributed environment. In: Proceedings of the 2017 Software Engineering and Information Management, SEIM 2017 (2017)
Graefe, G.: Query evaluation techniques for large databases. ACM Comput. Surv. 25(2), 73–169 (1993)
Idreos, S., Groffen, F., Nes, N., Manegold, S., Mullender, K.S., Kersten, M.L.: MonetDB: Two decades of research in column-oriented database architectures. IEEE Data Eng. Bull. 35(1), 40–45 (2012)
Liu, Y., et al.: DCODE: A distributed column-oriented database engine for big data analytics. In: Khalil, I., Neuhold, E., Tjoa, A.M., Da Xu, L., You, I. (eds.) CONFENIS/ICT-EurAsia -2015. LNCS, vol. 9357, pp. 289–299. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24315-3_30
Stonebraker, M., et al.: C-Store: A column-oriented DBMS. In: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB 2005, pp. 553–564. VLDB Endowment (2005)
Valduriez, P.: Join indices. ACM Trans. Database Syst. 12(2), 218–246 (1987)
Zhang, Y., Xiao, Y., Wang, Z., Ji, X., Huang, Y., Wang, S.: ScaMMDB: Facing challenge of mass data processing with MMDB. In: Chen, L., et al. (eds.) APWeb/WAIM -2009. LNCS, vol. 5731, pp. 1–12. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03996-6_1
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Chernishev, G., Galaktionov, V., Grigorev, V., Klyuchikov, E., Smirnov, K. (2018). PosDB: A Distributed Column-Store Engine. In: Petrenko, A., Voronkov, A. (eds) Perspectives of System Informatics. PSI 2017. Lecture Notes in Computer Science(), vol 10742. Springer, Cham. https://doi.org/10.1007/978-3-319-74313-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-74313-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-74312-7
Online ISBN: 978-3-319-74313-4
eBook Packages: Computer ScienceComputer Science (R0)