Abstract
Achieving good load balance in cluster environment is difficult due to its dynamic nature. Besides, the data to be processed may be transfered from geographically distant sites at a fluctuating transfer rate. This paper proposes a load balancing algorithm to deal with such fluctuating transfer rate. The novelty of the algorithm is to use buddy concept in processing node grouping and the data distribution statistics obtained at the source sites. We use single join processing in database query processing as its application.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
L. Amsaleg, M. J. Franklin, A. Tomasic, and T. Urhan. Scrambling Query Plans to Cope with Unexpected Delays. In Fourth International Conference on Parallel and Distributed Information Systems (PDIS’ 96), pages 208–219, Miami Beach, Florida, USA, Dec. 1996. IEEE Computer Society Press.
R. Avnur and J. M. Hellerstein. Eddies: Continuously Adaptive Query Processing. In W. Chen, J. Naughton, and P. A. Bernstein, editors, The 2000 ACM SIGMOD International Conference on Management of Data, volume 29 (2) of SIGMOD Record (ACM Special Interest Group on Management of Data), pages 261–272, Dallas, Texas, May 2000. ACM Press.
L. Bouganim, F. Fabert, C. Mohan, and P. Valduriez. A Dynamic Query Processing Architecture for Data Integration Systems. In 16th International Conference on Data Engineering, pages 425–434. IEEE Computer Society Press, Mar. 2000.
L. Bouganim, F. Fabert, C. Mohan, and P. Valduriez. Dynamic Query Scheduling in Data Integration Systems. IEEE Data Engineering Bulletin: Special Issue on Adaptive Query Processing, 23(2):42–48, June 2000.
J. M. Hellerstein, M. J. Franklin, S. Chandrasekaran, A. Deshpande, K. Hildrum, S. Madden, V. Raman, and M. Shah. Adaptive Query Processing: Technology in Evolution. IEEE Data Engineering Bulletin: Special Issue on Adaptive Query Processing, 23(2):7–18, June 2000.
K. A. Hua and W. Tavanapong. Performance of Load Balancing Techniques for Join Operations in Shared-nothing Database Management Systems. Journal of Parallel and Distributed Computing, 56(1):17–46, Jan. 1999.
K. Imasaki and S. Dandamudi. Performance Evaluation of Nested-loop Join Processing on Networks of Workstations. In Proceedings of the Seventh International Conference on Parallel and Distrubuted Systems, pages 537–544, Iwate, Japan, July 2000.
K. Imasaki and S. Dandamudi. An Adapive Hash Join Algorithm on a Network of Workstations. In International Parallel and Distributed Processing Symposium (IPDPS), Fort Lauderdale, Florida, Apr. 2002.
K. Imasaki, H. Nguyen, and S. Dandamudi. Performance Comparison of Pipelined Hash Joins on Workstation Clusters. In 9th International Conference on High Performance Computing (HiPC2002), pages 264–275, Bangalore, India, Dec. 2002.
K. Imasaki, H. Nguyen, and S. Dandamudi. Performance of Pipelined Nested Loop and Hash Joins on a Workstation Cluster. In The 2002 International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA’02), June 2002.
D. A. Schneider and D. J. DeWitt. A Performance Evaluation of Four Parallel Join Algorithms in a Shared-nothing Multiprocessor environment. SIGMOD Record, 18(2):110–121, June 1989.
T. Urhan and M. J. Franklin. XJoin: Getting Fast Answers From Slow and Bursty Networks. Technical Report CS-TR-3994, University of Maryland, College Park, Feb. 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Imasaki, K., Abawajy, J.H. (2003). An Adaptive Load Balancing Algorithm for Large Data Parallel Processing with Communication Delay. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Gorbachev, Y.E., Dongarra, J.J., Zomaya, A.Y. (eds) Computational Science — ICCS 2003. ICCS 2003. Lecture Notes in Computer Science, vol 2660. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44864-0_25
Download citation
DOI: https://doi.org/10.1007/3-540-44864-0_25
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40197-1
Online ISBN: 978-3-540-44864-8
eBook Packages: Springer Book Archive