Some Strategies for Load Balancing

  • Claude G. Diderich
  • Marc Gengler


In this paper we discuss the needs for load balancing, also called scheduling. We exhibit different reasons that render static (compile-time) scheduling impossible and that determine the dynamic (run-time) load balancing schemes needed in order to get efficient parallel algorithms One distinguishes between local load balancing policies where processors base their decisions on information about the load in some neighborhood and global load balancing policies where processors base their decisions on the load of the entire machine. Depending on the static information available and on the dependencies between the different tasks, some parallel algorithms accommodate with simple load balancing or load sharing mechanisms while others need more sophisticated solutions. The former are typically local while the later are global load balancing schemes. In particular, we analyze the branch and bound algorithm and show that it needs smart load balancing mechanisms ideally founded on global knowledge. We argue that for this algorithm a global load balancing policy may be interesting. Indeed, the best-first branch and bound algorithm can be defined as a sequence of independent computations allowing the design of a parallel algorithm that alternates between coarse grained parallel computation phases and so-called synchronization phases which provide perfect global load balancing.


Execution Time Load Balance Parallel Algorithm Travel Salesman Problem Static Allocation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    T. Casavant, J. G. Kuhl: A Taxonomy of Scheduling in General-purpose Distributed Computing Systems. IEEE Trans. on Software Engineering, 14, pp. 141–154, 1988.CrossRefGoogle Scholar
  2. [2]
    A. Corradi, L. Leonardi, F. Zambonelli: Load balancing strategies for massively parallel architectures. Parallel Processing Letters, 2(2 and 3), pp. 139148, 1992.Google Scholar
  3. [3]
    G. Cybenko: Dynamic Load Balancing for Distributed Memory Multiprocessors. J. of Parallel and Distributed Computing, 7, pp. 279–301, 1989.CrossRefGoogle Scholar
  4. [4]
    C. G. Diderich, M. Gengler: Experiments with a Parallel Synchronized Branch and Bound Algorithm. In this volume.Google Scholar
  5. [5]
    C. G. Diderich, M. Gengler, St. Ubéda: An Efficient Algorithm for Solving the Token Distribution Problem on k-ary d-cube Networks. (Extended abstract) Proceedings ISPAN 94, to appear, 1994.Google Scholar
  6. [6]
    M. Dion, M. Gengler, St. Ubéda: Comparing two Probabilistic Models of the Computational Complexity of the Branch and Bound Algorithm. Proceedings CONPAR 94 — VAPP VI, LNCS 854, pp. 359–370, 1994.Google Scholar
  7. [7]
    D. L. Eager, E. D. Lazowska, J. Zahorjan: Adaptive load sharing in homo-geneous distributed systems. IEEE Trans. on Software Engineering, 12 (5), pp. 662–675, 1986.CrossRefGoogle Scholar
  8. [8]
    M. J. Flynn: Very High Speed Computing Systems. Proceedings IEEE 54 (12), pp. 1901–1909, 1966.CrossRefGoogle Scholar
  9. [9]
    M. R. Garey, D. S. Johnson: Computers and Intractability - A Guide to the Theory of NP-Completeness. W.H. Freeman, 1979.Google Scholar
  10. [10]
    M. Gengler, G. Coray: A Parallel Best-first B and B with Synchronization Phases. Proceedings CONPAR 92 — VAPP V, LNCS 634, pp. 515–526, 1992.Google Scholar
  11. [11]
    M. Gengler, G. Coray: A Parallel Best-first B and B Algorithm and its Axiomatization. IEEE Proceedings HICSS-26, Vol. 2, pp. 263–272, 1993. Also in J. of Parallel Algorithms and Applications, Vol. 2, pp. 61–80, 1994.zbMATHGoogle Scholar
  12. [12]
    M. Held, R. M. Karp: The Traveling Salesman Problem and Minimum Spannig Trees. Operations Research 18, pp. 1138–1162, 1970.MathSciNetzbMATHCrossRefGoogle Scholar
  13. [13]
    M. Held, R. M. Karp: The Traveling Salesman Problem and Minimum Spanning Trees: part II. Mathematical Programming 1, pp. 6–25, 1971.MathSciNetzbMATHCrossRefGoogle Scholar
  14. [14]
    R. Jonker, T. Volgenant: Non-optimal Edges for the Symmetric Traveling Salesman Problem. Operations Research 32, pp. 837–846, 1984.MathSciNetzbMATHCrossRefGoogle Scholar
  15. [15]
    S. Kirkpatrick, C. D. Gelatt, M. P. Vecchi: Optimization by Simulated Annealing. Science, 220 (4598), pp. 671–680, 1983.MathSciNetzbMATHCrossRefGoogle Scholar
  16. [16]
    T. H. Lai, S. Sahni: Anomalies in Parallel Branch and Bound Algorithms Comm. ACM, 27 (6), pp. 594–602, 1984.MathSciNetzbMATHCrossRefGoogle Scholar
  17. [17]
    E. L. Lawler, J. K. Lenstra, A. H. G. Rinnooy Kan, D. B. Shmoys (eds.): The Traveling Salesman Problem: a Guided Tour of Combinatorial Optimization. Wiley and sons, Chichester (GB), 1985.zbMATHGoogle Scholar
  18. [18]
    M. R. Samatham, D. K. Pradhan: The de Bruijn Multiprocessor Network: a Versatile Parallel Processing and Sorting Network for VLSI. IEEE Trans. on Comp., 38 (4), pp. 567–581, 1989.MathSciNetzbMATHCrossRefGoogle Scholar
  19. [19]
    Th. Le Sergent, B. Berthomieu: Balancing Load under Large and Fast Load Changes in Distributed Computing Systems–A Case Study. Proceedings CONPAR 94 — VAPP VI, LNCS 854, pp. 854–866, 1994.Google Scholar
  20. [20]
    R. Liiling, B. Monien, M. Räcke, S. Tschöke: Efficient Parallelization of a Branch and Bound Algorithm for the Symmetric Traveling Salesman Problem. Tech. Report, University of Padreborn, 1992.Google Scholar
  21. [21]
    R. Lüling, B. Monien: Load Balancing for Distributed Branch and Bound Algorithms. Proceedings Int. Parallel Processing Symp. (IPPS), pp. 543549, 1992.Google Scholar
  22. [22]
    J. W. Meyer: Self-organizing Processes. Proceedings CONPAR 94–VAPP VI, LNCS 854, pp. 842–853, 1994.Google Scholar
  23. [23]
    F. Meyer auf der Heide, B. Oesterdiekhoff, R. Wanka: Strongly Adaptive Token Distribution. Proceedings 20th ICALP 93, LNCS 700, pp. 398–409, 1993.Google Scholar
  24. [24]
    L. G. Mitten: Branch and Bound Methods: General Formulation and Properties. Operations Research 18, pp. 24–34, 1970.MathSciNetzbMATHCrossRefGoogle Scholar
  25. [25]
    J. von Neumann: John von Neumann - Collected Works, Volume 5. A. H. Toub (ed. ), Pergamon Press, 1961.Google Scholar
  26. [26]
    C. H. Papadimitriou, K. Steiglitz: Combinatorial Optimization, Algorithms and Complexity. Prentice Hall, 1982.Google Scholar
  27. [27]
    J. Pearl: Heuristics, Addison-Wesley, 1985.Google Scholar
  28. [28]
    D. Peleg, E. Upfal: The Token Distribution Problem. SIAM J. Comput. 18 (2), pp. 229–243, 1989.MathSciNetzbMATHGoogle Scholar
  29. [29]
    W. Pijls, A. de Bruin: Another View on the SSS* Algorithm. Proceedings SIGAL’90, 1990.Google Scholar
  30. [30]
    G. Reinelt: Tsplib v1.2 - Traveling Salesman Problems Library. University of Augsburg, Germany, 1990. Anonymous ftp from Scholar
  31. [31]
    A. H. G. Rinnooy Kan: On Mitten’s Axioms for Branch and Bound. Graduate School of Management, Delft, Tech. Rep. W/74/45/03, 1974.Google Scholar
  32. [32]
    C. Roucairol: Parallel Branch and Bound Algorithms: an Overview. In Parallel and Distributed Algorithms, Cosnard, Robert, Quinton, Raynal (eds.), pp. 153–163, Elsevier Science Publishers, 1988.Google Scholar
  33. [33]
    M. Schwehm, Th. Walter: Mapping and Scheduling by Genetic Algorithms. Proceedings CONPAR 94–VAPP VI, LNCS 854, pp. 832–841, 1994.Google Scholar
  34. [34]
    G. C. Stockman: A Minimax Algorithm better than Alpha-beta? Artificial Intelligence, 12, pp. 179–196, 1979.MathSciNetzbMATHCrossRefGoogle Scholar
  35. [35]
    J. D. Ullman: NP-complete scheduling problems. J. Comput. System Sci., 10, pp. 384–393, 1975.MathSciNetzbMATHCrossRefGoogle Scholar
  36. [36]
    T. Volgenant, R. Jonker: The Symmetric Traveling Salesman Problem and Edge Exchanges in Minima 1-trees. European Journal of Operational Research 12, pp. 394–403, 1983.MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1995

Authors and Affiliations

  • Claude G. Diderich
    • 1
  • Marc Gengler
    • 1
  1. 1.Computer Science DepartmentSwiss Federal Institute of Technology — LausanneLausanneSwitzerland

Personalised recommendations