Abstract
Dynamic load balancing is a prerequisite for effectively utilizing large parallel database systems. Load balancing at different levels is required in particular for assigning transactions and queries as well as subqueries to nodes. Special problems are posed by the need to support both inter-transaction/query as well as intra-transaction/query parallelism due to conflicting performance requirements. We compare the major architectures for parallel database systems, Shared Nothing and Shared Disk, with respect to their load balancing potential. For this purpose, we focus on parallel scan and join processing in multi-user mode. It turns out that both the degree of query parallelism as well as the processor allocation should be determined in a coordinated way and based on the current utilization of critical resource types, in particular CPU and memory.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Bober, P.M., Carey, M.J.: On Mixing Queries and Transactions via Multiversion Locking. Proc. 8th IEEE Data Engineering Conf., 535–545, 1992
Brown, K.P.; Mehta, M.; Carey, M.J.; Livny, M.: Towards Automated Performance Tuning for Complex Workloads. Proc. 20th VLDB Conf., 72–84, 1994
Copeland, G., Alexander, W., Boughter, E., Keller, T.: Data Placement in Bubba. Proc. ACM SIGMOD Conf., 99–108, 1988
Chen, P.M., Lee, E.K., Gibson, G.: RAID: High-Performance, Reliable Secondary Storage. ACM Computing Surveys 26 (2), 145–185, 1994
Carey, M.J., Muhanna, W.A.: The Performance of Multiversion Concurrency Control Algorithms. ACM Trans. on Computer Systems 4 (4), 338–378, 1986
DeWitt, D.J., Gray, J.: Parallel Database Systems: The Future of High Performance Database Systems. Comm. ACM 35 (6), 85–98, 1992
Davison, D.L.; Graefe, G.: Memory-Contention Responsive Hash Joins. Proc. 20th VLDB Conf., 379–390, 1994.
Davison, D.L.; Graefe, G.: Dynamic Resource Brokering for Multi-User Query Execution. Proc. ACM SIGMOD Conf., 281–292, 1995
DeWitt, D.J., Naughton, J.F., Schneider, D.A., Seshadri, S.: Practical Skew Handling in Parallel Joins. Proc. 18th VLDB Conf., 27–40, 1992
Ghandeharizadeh, S.: Physical Database Design in Multiprocessor Systems. Ph.D. Thesis, Univ. of Wisconsin-Madison, 1990
Graefe, G.: Query Evaluation Techniques for Large Databases. ACM Comput. Surveys 25 (2), 73–170, 1993
Gray, J.: Super-Servers: Commodity Computer Clusters Pose a Software Challenge. Proc. German Database Conf. BTW, March 1995
Gray, J., Reuter, A.: Transaction Processing. Morgan Kaufmann, 1993
Hong, W.: Exploiting Inter-Operation Parallelism in XPRS. Proc. ACM SIGMOD Conf., 19–28, 1992
Hua, K.A., Lee, C.; Hua, C.M.: Dynamic Load Balancing in Multicomputer Database Systems Using Partition Tuning. IEEE Trans. on Knowledge and Data Engineering 7(6), 968–983, 1995
Hua, K.A., Lo, Y., Young, H.C.: Considering Data Skew Factor in Multi-Way Join Query Optimization for Parallel Execution. VLDB Journal 2(3), 303–330, 1993
Hirano, Y., Satoh, T., Inoue, U., Teranaka, K.: Load Balancing Algorithms for Parallel Database Processing on Shared Memory Multiprocessors. Proc. 1st Int. Conf. on Parallel and Distributed Information Systems, 210–217 1991
Kim, W.: Highly Available Systems for Database Applications. ACM Computing Surveys 16 (1), 71–98, 1984
Lu, H., Tan, K.: Dynamic and Load-Balanced Task-Oriented Database Query Processing in Parallel Systems. Proc. EDBT, LNCS 580 357–372 1992
Marek, R.: A Cost Model for Parallel Query Processing in Shared Nothing DBS (in German). Proc. German Database Conf. BTW, March 1995
Mehta, M., DeWitt, D.J.: Dynamic Memory Allocation for Multiple-Query Workloads. Proc 19th VLDB Conf., 354–367, 1993
Mehta, M., DeWitt, D.J.: Managing Intra-Operator Parallelism in Parallel Database Systems. Proc 21th VLDB Conf., 382–394, 1995
Mohan, C., Narang, I.: Recovery and Coherency-control Protocols for Fast Intersystem Page Transfer and Fine-Granularity Locking in a Shared Disks Transaction Environment. Proc. 17th VLDB Conf., 193–207, 1991
Mohan, C., Pirahesh, H., Tang, W.G., Wang, Y.: Parallelism in Relational Database Management Systems. IBM Systems Journal 33 (2), 1994
Marek, R., Rahm, E.: Performance Evaluation of Parallel Transaction Processing in Shared Nothing Database Systems, Proc. 4th Int. PARLE Conf., LNCS 605, 295–310, 1992
Omiecinski, E.: Performance Analysis of a Load-Balancing Hash-Join Algorithm for a Shared-Memory Multiprocessor. Proc 17th VLDB Conf. 375–385,1991
Özsu, M.T., Valduriez, P.: Principles of Distributed Database Systems. Prentice Hall, 1991
Pang, H., Carey, M.J., Livny, M.: Partially Preemptible Hash Joins. Proc. ACM SIGMOD Conf., 59–68, 1993
Patterson, D.A., Gibson, G., Katz, R.H.: A Case for Redundant Arrays of Inexpensive Disks (RAID). Proc. ACM SIGMOD Conf., 109–116, 1988
Rahm, E.: A Framework for Workload Allocation in Distributed Transaction Processing Systems. Journal of Systems and Software 18, 171–190, 1992
Rahm, E.: Empirical Performance Evaluation of Concurrency and Coherency Control for Database Sharing Systems. ACM Trans. on Database Systems 18 (2), 333–377, 1993
Rahm, E., Marek, R.: Analysis of Dynamic Load Balancing Strategies for Parallel Shared Nothing Database Systems. Proc 19th VLDB Conf., 182–193, 1993
Rahm, E., Marek, R.: Dynamic Multi-Resource Load Balancing in Parallel Database Systems. Proc 21th VLDB Conf., 395–406, 1995
Rahm, E., Stöhr, T.: Analysis of Parallel Scan Processing in Shared Disk Database Systems. Proc. Euro-PAR95, LNCS 966, 485–500, 1995
Selinger, P.: Predictions and Challenges for Database Systems in the Year 2000. Proc 19th VLDB Conf., 667–675, 1993
Valduriez, P.: Parallel Database Systems: Open Problems and New Issues. Distr. and Parallel Databases 1 (2), 137–165, 1993
Walton, C.B; Dale A.G.; Jenevein, R.M.: A Taxonomy and Performance Model of Data Skew Effects in Parallel Joins. Proc. 17th VLDB Conf., 537–548, 1991
Wolf, J.L., Dias, D.M., Yu, P.S., Turek, J.: New Algorithms for Parallelizing Relational Database Joins in the Presence of Data Skew. IEEE Trans. on Knowledge and Data Engineering 6(6), 990–997, 1994
Wilschut, A.; Flokstra, J.; Apers, P.: Parallelism in a Main-Memory DBMS: The performance of PRISMA/DB. Proc. 18th VLDB Conf., 521–532, 1992
Yu, P.S., Cornell, D.W., Dias, D.M., Iyer, B.R.: Analysis of Affinity-based Routing in Multi-system Data Sharing. Performance Evaluation 7 (2), 87–109, 1987
Zeller H., Gray, J.: An Adaptive Hash Join Algorithm for Multiuser Environments. Proc. 16th VLDB Conf., 186–197, 1990
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rahm, E. (1996). Dynamic load balancing in parallel database systems. In: Bougé, L., Fraigniaud, P., Mignotte, A., Robert, Y. (eds) Euro-Par'96 Parallel Processing. Euro-Par 1996. Lecture Notes in Computer Science, vol 1123. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61626-8_4
Download citation
DOI: https://doi.org/10.1007/3-540-61626-8_4
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61626-9
Online ISBN: 978-3-540-70633-5
eBook Packages: Springer Book Archive