PolyBench: The First Benchmark for Polystores

  • Jeyhun KarimovEmail author
  • Tilmann Rabl
  • Volker Markl
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11135)


Modern business intelligence requires data processing not only across a huge variety of domains but also across different paradigms, such as relational, stream, and graph models. This variety is a challenge for existing systems that typically only support a single or few different data models. Polystores were proposed as a solution for this challenge and received wide attention both in academia and in industry. These are systems that integrate different specialized data processing engines to enable fast processing of a large variety of data models. Yet, there is no standard to assess the performance of polystores. The goal of this work is to develop the first benchmark for polystores. To capture the flexibility of polystores, we focus on high level features in order to enable an execution of our benchmark suite on a large set of polystore solutions.



This work has been supported by the European Commission through Proteus (ref. 687691) and Streamline (ref. 688191) and by the German Ministry for Education and Research as Berlin Big Data Center BBDC (funding mark 01IS14013A).


  1. 1.
    Apache Arrow: a cross-language development platform for in-memory data. Accessed 24 Feb 2018
  2. 2.
    Query modeling and optimization in the BigDAWG polystore system. Accessed 10 Mar 2018
  3. 3.
    Avery, C.: Giraph: large-scale graph processing infrastructure on Hadoop. In: Proceedings of the Hadoop Summit, Santa Clara, vol. 11, pp. 5–9 (2011)Google Scholar
  4. 4.
    Bondiombouy, C., Kolev, B., Levchenko, O., Valduriez, P.: Integrating big data and relational data with a functional SQL-like query language. In: Chen, Q., Hameurlain, A., Toumani, F., Wagner, R., Decker, H. (eds.) DEXA 2015. LNCS, vol. 9261, pp. 170–185. Springer, Cham (2015). Scholar
  5. 5.
    Chaudhuri, S., Narasayya, V.: Self-tuning database systems: a decade of progress. In: Proceedings of the 33rd International Conference on Very Large Data Bases, pp. 3–14. VLDB Endowment (2007)Google Scholar
  6. 6.
    Chen, Y., Xu, C., Rao, W., Min, H., Su, G.: Octopus: hybrid big data integration engine. In: 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom), pp. 462–466. IEEE (2015)Google Scholar
  7. 7.
    Duggan, J., et al.: The BigDAWG polystore system. ACM SIGMOD Rec. 44(2), 11–16 (2015)CrossRefGoogle Scholar
  8. 8.
    Dziedzic, A., Elmore, A.J., Stonebraker, M.: Data transformation and migration in polystores. In: 2016 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1–6. IEEE (2016)Google Scholar
  9. 9.
    Gadepally, V., et al.: The BigDAWG polystore system and architecture. In: 2016 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1–6. IEEE (2016)Google Scholar
  10. 10.
    Ghazal, A., et al.: BigBench: towards an industry standard benchmark for big data analytics. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pp. 1197–1208. ACM (2013)Google Scholar
  11. 11.
    Haynes, B., Cheung, A., Balazinska, M.: PipeGen: data pipe generator for hybrid analytics. In: Proceedings of the Seventh ACM Symposium on Cloud Computing, pp. 470–483. ACM (2016)Google Scholar
  12. 12.
    Jovanovic, P., Simitsis, A., Wilkinson, K.: Engine independence for logical analytic flows. In: 2014 IEEE 30th International Conference on Data Engineering (ICDE), pp. 1060–1071. IEEE (2014)Google Scholar
  13. 13.
    Kolev, B., Pau, R., Levchenko, O., Valduriez, P., Jiménez-Peris, R., Pereira, J.: Benchmarking polystores: the cloudMdsQL experience. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 2574–2579. IEEE (2016)Google Scholar
  14. 14.
    Kolev, B., Valduriez, P., Bondiombouy, C., Jiménez-Peris, R., Pau, R., Pereira, J.: CloudMdsQL: querying heterogeneous cloud data stores with a common language. Distrib. Parallel Databases 34(4), 463–503 (2016)CrossRefGoogle Scholar
  15. 15.
    LeFevre, J., Sankaranarayanan, J., Hacigumus, H., Tatemura, J., Polyzotis, N., Carey, M.J.: MISO: souping up big data query processing with a multistore system. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, pp. 1591–1602. ACM (2014)Google Scholar
  16. 16.
    Leskovec, J., Sosič, R.: SNAP: a general-purpose network analysis and graph-mining library. ACM Trans. Intell. Syst. Technol. (TIST) 8(1), 1 (2016)CrossRefGoogle Scholar
  17. 17.
    Lu, J.: Towards benchmarking multi-model databases. In: CIDR (2017)Google Scholar
  18. 18.
    Lu, J., Holubová, I.: Multi-model data management: what’s new and what’s next? In: EDBT, pp. 602–605 (2017)Google Scholar
  19. 19.
    Lu, J., Liu, Z.H., Xu, P., Zhang, C.: UDBMS: road to unification for multi-model data management. arXiv preprint arXiv:1612.08050 (2016)
  20. 20.
    Palkar, S., et al.: Weld: a common runtime for high performance data analytics. In: Conference on Innovative Data Systems Research (CIDR) (2017)Google Scholar
  21. 21.
    Simitsis, A., Wilkinson, K., Castellanos, M., Dayal, U.: Optimizing analytic data flows for multiple execution engines. In: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp. 829–840. ACM (2012)Google Scholar
  22. 22.
    Stonebraker, M., Cetintemel, U.: “One size fits all”: an idea whose time has come and gone. In: Proceedings of 21st International Conference on Data Engineering, ICDE 2005, pp. 2–11. IEEE (2005)Google Scholar
  23. 23.
    Sun, N., Morris, J., Xu, J., Zhu, X., Xie, M.: ICARE: a framework for big data-based banking customer analytics. IBM J. Res. Dev. 58(5/6), 4:1–4:9 (2014)CrossRefGoogle Scholar
  24. 24.
    Valduriez, P.: Parallel database systems: open problems and new issues. Distrib. Parallel Databases 1(2), 137–165 (1993)CrossRefGoogle Scholar
  25. 25.
    Xu, C., Chen, Y., Liu, Q., Rao, W., Min, H., Su, G.: A unified computation engine for big data analytics. In: 2015 IEEE/ACM 2nd International Symposium on Big Data Computing (BDC), pp. 73–77. IEEE (2015)Google Scholar
  26. 26.
    Yu, K., Gadepally, V., Stonebraker, M.: Database engine integration and performance analysis of the BigDAWG polystore system. In: 2017 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1–7. IEEE (2017)Google Scholar
  27. 27.
    Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: HotCloud, vol. 10, no. 10–10, p. 95 (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.DFKIKaiserslauternGermany
  2. 2.TU BerlinBerlinGermany

Personalised recommendations