Abstract
In-memory database management systems have the potential to reduce the execution time of complex operational analytical queries to the order of seconds while executing business transactions in parallel. The main reasons for this increase of performance are massive intra-query parallelism on many-core CPUs and primary data storage in main memory instead of disks or SSDs. However, database management systems in enterprise scenarios typically run a mix of different applications and users, of varying importance, concurrently. As an example, interactive applications have a much higher response-time objective compared to periodic jobs producing daily reports and should be run with priority. In addition to strict prioritization, enforcing a fair share of database resources is desirable, if several users work on applications that share a database. Solutions for resource management based on priorities have been proposed for disk-based database management systems. They typically rely on multiplexing threads on a number of processing units, which is unfavorable for in-memory databases on multi-cores, as single queries are executed in parallel and numerous context switches disrupt cache-conscious algorithms. Consequently, we propose an approach towards resource management based on a task-based query execution that avoids thread multiplexing. The basic idea is to calculate the allowed share of execution time for each user based on the priorities of all users and adjust priorities of tasks of incoming queries to converge to this share.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brown, K., Carey, M., DeWitt, D., Mehta, M., Naughton, F.: Resource allocation and scheduling for mixed database workloads, Jan 1992. http://www.cs.wisc.edu
Brown, R.G.: Smoothing, Forecasting and Prediction of Discrete Time Series (Dover Phoenix editions). Dover Publications, New York (2004)
Biersack, E.W., Schroeder, B., Urvoy-Keller, G.: Scheduling in practice. SIGMETRICS Perform. Eval. Rev. 34(4), 21–28 (2007)
Balkesen, C., Teubner, J., Alonso, G., Tamer szu, M.: Main-memory hash joins on multi-core CPUs: tuning to the underlying hardware. In: IEEE Computer Society ICDE ’13 (2013)
Carey, M.J., Jauhari, R., Livny, M.: Priority in DBMS resource scheduling. In: VLDB, pp. 397–410 (1989)
Grund, M., Krüger, J., Plattner, H., Zeier, A., Cudre-Mauroux, P., Madden, S.: HYRISE: a main memory hybrid storage engine. Proc. VLDB Endow. 4(2), 105–116 (2010)
Gupta, C., Mehta, A., Wang, S., Dayal, U.: Fair, effective, efficient and differentiated scheduling in an enterprise data warehouse. In: EDBT ’09, pp. 696–707. ACM, New York, NY, USA (2009)
Kuno, H., Dayal, U., Wiener, J.L., Wilkinson, K., Ganapathi, A., Krompass, S.: Managing dynamic mixed workloads for operational business intelligence. In: Kikuchi, S., Sachdeva, S., Bhalla, S. (eds.) DNIS 2010. LNCS, vol. 5999, pp. 11–26. Springer, Heidelberg (2010)
Krueger, J., Kim, C., Grund, M., Satish, N., Schwalb, D., Chhugani, J., Dubey, P., Plattner, H., Zeier, A.: Fast updates on read-optimized databases using multi-core CPUs. PVLDB 5(1), 61–72 (2011)
Krompass, S., Kuno, H., Wilkinson, K., Dayal, U., Kemper, A.: Adaptive query scheduling for mixed database workloads with multiple objectives. In: DBTest ’10, pp. 1:1–1:6. ACM, New York, NY, USA (2010)
Leung, J., Kelly, L., Anderson, J.H.: Handbook of Scheduling: Algorithms, Models, and Performance Analysis. CRC Press Inc, Boca Raton, FL, USA (2004)
Mcwherter, D.T., Schroeder, B., Ailamaki, A., Harchol-balter, M.: Priority mechanisms for OLTP and transactional web applications, pp. 535–546 (2003)
Niu, B., Martin, P., Powley, W.: Towards autonomic workload management in DBMSs. J. Database Manage. 20(3), 1–17 (2009)
Narayanan. S., Waas, F.: Dynamic prioritization of database queries. In: IEEE Computer Society ICDE ’11, Washington, DC, USA (2011)
Plattner, H.: A common database approach for OLTP and OLAP using an in-memory column database. In: SIGMOD, pp. 1–2 (2009)
Plattner, H.: SanssouciDB: an in-memory database for processing enterprise workloads. In: BTW, pp. 2–21 (2011)
Schroeder, B., Harchol-balter, M.:. Achieving class-based QoS for transactional workloads. In: Proceedings of the IEEE ICDE, p. 153 (2006)
Schroeder, B., Harchol-Balter, M., Arun, I., Nahum, E., Wierman, A.: How to determine a good multi-programming level for external scheduling. In: International Conference on Data Engineering vol. 60 (2006)
Wust, J, Grund, M., Plattner, H.: TAMEX: a task-based query execution framework for mixed enterprise workloads on in-memory databases. INFORMATIK, Koblenz (accepted for publication). In: Workshop on In-Memory Data Management (2013)
Xu, D., Wu, C., Yew, P.C., Li, J., Wang, Z.: Providing fairness on shared-memory multiprocessors via process scheduling. In:ACM SIGMETRICS/PERFORMANCE, SIGMETRICS ’12, pp. 295–306. ACM, New York, NY, USA (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Wust, J., Grund, M., Plattner, H. (2015). Dynamic Query Prioritization for In-Memory Databases. In: Jagatheesan, A., Levandoski, J., Neumann, T., Pavlo, A. (eds) In Memory Data Management and Analysis. IMDM IMDM 2013 2014. Lecture Notes in Computer Science(), vol 8921. Springer, Cham. https://doi.org/10.1007/978-3-319-13960-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-13960-9_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-13959-3
Online ISBN: 978-3-319-13960-9
eBook Packages: Computer ScienceComputer Science (R0)