Advertisement

Selected Algorithmic Techniques for Parallel Optimization

  • Ricardo C. Corrêa
  • Afonso Ferreira
  • Stella C. S. Porto
Chapter

Abstract

The use of parallel algorithms for solving computationally hard problems becomes attractive as parallel systems, consisting of a collection of powerful processors, offer large computing power and memory storage capacity. Even though parallelism will not be able to overdue the assumed worst case exponential time or memory complexity of those problems (unless an exponential number of processors is used) [11], the average execution time of heuristic search algorithms which find good suboptimal solutions for many hard problems is polynomial. Consequently, parallel systems, possibly with hundreds or thousands of processors, give us the perspective of efficiently solving relatively large instances of hard problems.

Keywords

Genetic Algorithm Quadratic Assignment Problem Discrete Optimization Problem Algorithmic Technique Good Speedup 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    E. Aarts and J. Korst, Simulated Annealing and Boltzmann Machines, ( Wiley, Chichester, 1989 ).zbMATHGoogle Scholar
  2. [2]
    D. Abramson, Constructing school timetables using simulated annealing: Sequential and parallel algorithms, Management Science, Vol. 37 (1991) pp. 98–113.CrossRefGoogle Scholar
  3. [3]
    G. Ananth, V. Kumar, and P. Pardalos, Parallel processing of discrete optimization problems, Encyclopedia of Microcomputers, Vol. 13 (1993) pp. 129–147.Google Scholar
  4. [4]
    S. Arvindam, V. Kumar, and V. N. Rao, Efficient parallel algorithms for search problems: Applications in VLSI CAD, in Proceedings of the Frontiers 90 Conference on Massively Parallel Computation (1990).Google Scholar
  5. [5]
    V. Barbosa, Introduction to Distributed Algorithms, (The MIT Press, 1996 ).Google Scholar
  6. [6]
    R. Battiti and G. Tecchioli, Parallel biased search for combinatorial optimization: Genetic algorithms and tabu, Microprocessors and Microsystems, Vol. 16 (1992) pp. 351–367.CrossRefGoogle Scholar
  7. [7]
    V. Cerny, A thermodynamical approach to the travelling salesman problem: An efficient simulation algorithm, Journal of Optimization Theory and Applications, Vol. 45 (1985) pp. 41–55.CrossRefzbMATHMathSciNetGoogle Scholar
  8. [8]
    A. Chipperfield and P. Fleming, Parallel genetic algorithms, in A. Zomaya (ed.) Parallel and Distributed Computing Handbook, (McGraw-Hill, 1996) Chapter 39, pp. 1118–1143.Google Scholar
  9. [9]
    J. Clausen, Do inherently sequential branch-and-bound algorithms exist? Parallel Processing Letters (1994).Google Scholar
  10. [10]
    J. Cohoon, S. Hedge, W. Martin, and D. Richards, Punctuated equilibra: A parallel genetic algorithm, in Proceedings of the Second Int. Conf. on Genetic Algorithms, ( MIT, Cambridge, 1987 ) pp. 148–154.Google Scholar
  11. [11]
    T. Cormen, C. Leiserson, and R. Rivest, Introduction to Algorithms, ( The MIT Press, McGraw-Hill, 1990 ).zbMATHGoogle Scholar
  12. [12]
    R. Corrêa, Recherche Arborescente Parallèle: de la Formulation Algorithmique aux Applications, PhD thesis ( Institut National Polytechnique de Grenoble, France, 1997 ).Google Scholar
  13. [13]
    R. Corrêa and A. Ferreira, A distributed implementation of asynchronous parallel branch-and-bound, in A. Ferreira and J. Rolim (eds.) Parallel Algorithms for Irregular Problems: State of the Art, (Boston, Kluwer Academic Publisher, 1995), Chapter 8, pp. 157–176.Google Scholar
  14. [14]
    R. Corrêa and A. Ferreira, On the effectivenes of parallel branch and bound, Parallel Processing Letters Vol. 5 No. 3 (1995) pp. 375–386.CrossRefGoogle Scholar
  15. [15]
    R. Corrêa and A. Ferreira, Parallel best-first branch-and-bound in discrete optimization: A framework, in A. Ferreira and P. Pardalos (eds.) Solving Combinatorial Optimization Problems in Parallel, volume 1054 of LNCS State-of-the-Art Surveys, (Springer-Verlag, 1996 ) pp. 171–200.CrossRefGoogle Scholar
  16. [16]
    T. Crainic, M. Toulouse, and M. Gendreau, Towards a taxonomy of parallel tabu search algorithms, Technical Report CRT-933, ( Centre de Recherche sur les Transports, Université de Montréal, 1993 ).Google Scholar
  17. [17]
    M. Creutz, Microcanonical monte carlo simulation, Physics Review Letters Vol. 50 (1983) p. 1411.CrossRefMathSciNetGoogle Scholar
  18. [18]
    L. Davis (ed.), Handbook of genetic algorithms, ( New York, Van Nostrand Reinhold, 1991 ).Google Scholar
  19. [19]
    K. Dowsland, Simulated annealing, in C. R. Reeves (ed.) Modern Heuristics Techniques for Combinatorial Problems, Advanced Topics in Computer Science, ( Blackwell Scientific Publications, 1993 ), pp. 20–69.Google Scholar
  20. [20]
    J. Eckstein, Control strategies for parallel mixed integer branch and bound, in Proceedings of Supercomputing (1994).Google Scholar
  21. [21]
    J. Eckstein, Parallel branch-and-bound algorithms for general mixed integer programming on the CM-5, SIAM Journal on Optimization Vol. 4 No. 4 (1994) pp. 794–814.CrossRefzbMATHMathSciNetGoogle Scholar
  22. [22]
    A. Ferreira and P. Pardalos (eds.) Solving Combinatorial Optimization Problems in Parallel: Methods and Techniques, volume 1054 of LNCS State-of-the-Art Surveys, (Springer-Verlag, 1996 ).zbMATHGoogle Scholar
  23. [23]
    A. Ferreira and J. Rolim, (eds.) Solving Irregular Problems in Parallel: State of the Art, ( Boston, Kluwer Academic Publisher, 1995 ).Google Scholar
  24. [24]
    C.-N. Fiechter, A parallel tabu search algorithm for large traveling salesman problems, Discrete Applied Mathematics Vol. 51 (1994) pp. 243–267.CrossRefzbMATHMathSciNetGoogle Scholar
  25. [25]
    B. Garcia and M. Toulouse, A parallel tabu search for the vehicle routing problem with time windows, Computers and Operations Research Vol. 21 (1994) pp. 1025–1033.CrossRefzbMATHGoogle Scholar
  26. [26]
    F. Glover, Tabu search-part I, ORSA Journal on Computing Vol. 1 (1989) pp. 190–206.zbMATHMathSciNetGoogle Scholar
  27. [27]
    F. Glover, Tabu search-part II, ORSA Journal on Computing Vol. 2 (1990) pp. 4–32.zbMATHGoogle Scholar
  28. [28]
    F. Glover and M. Laguna, Tabu search, in C. R. Reeves (ed.) Modern Heuristics Techniques for Combinatorial Problems, Advanced Topics in Computer Science, ( Blackwell Scientific Publications, 1993 ) pp. 70–150.Google Scholar
  29. [29]
    F. Glover, E. Taillard, and D. de Werra, A user’s guide to tabu search, Annals of Operations Research Vol. 41 (1993) pp. 3–28.CrossRefzbMATHGoogle Scholar
  30. [30]
    D. Goldberg, Genetic algorithms in search, optimization, and machine learning, (Addison-Wesley, 1989 ).zbMATHGoogle Scholar
  31. [31]
    M. Gorges-Schleuter, Explicit parallelism of genetic algorithms through population structures, in Proceedings of the First Conference on Parallel Problem Solving from Nature-PPSN I, volume 496 of Lecture Notes in Computer Science, (Springer-Verlag, 1990 ) pp. 150–159.Google Scholar
  32. [32]
    R. Hauser and R. Manner, Implementation of standard genetic algorithm on mimd machines, in Proceedings of the Third Conference on Parallel Problem Solving from Nature-PPSN III, volume 866 of Lecture Notes in Computer Science, (Springer-Verlag, 1994 ) pp. 504–513.Google Scholar
  33. [33]
    J. Holland, Adaptation in natural and artificial systems, ( Ann Arbor, University of Michigan Press, 1975 ).Google Scholar
  34. [34]
    K. Homqvist, A. Migdalas, and P. Pardalos, Parallelized heuristics for combinatorial search, in A. Migdalas, P. Pardalos, and S. Storoy (eds.) Parallel computing in optimization, (Kluwer Academic Publishers, 1997 ) pp. 269–294.CrossRefGoogle Scholar
  35. [35]
    T. Ibaraki, The power of dominance relations in branch-and-bound algorithms, Journal of the ACM Vol. 24 No. 2 (1977) pp. 264–279.CrossRefzbMATHMathSciNetGoogle Scholar
  36. [36]
    T. Ibaraki, Enumerative approaches to combinatorial optimisation, Annals of Operations Research Vol. 11 No. 1–4 (1988).Google Scholar
  37. [37]
    P. Jog, J. Suh, and D. van Gucht, Parallel genetic algorithms applied to the traveling salesman problem, SIAM Journal of Optimization Vol. 1 No. 4 (1991) pp. 515–529.CrossRefzbMATHGoogle Scholar
  38. [38]
    R. Karp and Y. Zhang, A randomized parallel branch-and-bound procedure, in Symposium on Theory of Computing (1998) pp. 290–300.Google Scholar
  39. [39]
    R. Karp and Y. Zhang, Randomized parallel algorithms for backtrack search and branch-and-bound computations, Journal of the ACM Vol. 40 No. 3 (1993) 765–789.CrossRefzbMATHMathSciNetGoogle Scholar
  40. [40]
    S. Kirkpatrick, C. Gellat, and M. Vecchi, Optimization by simulated annealing, Science Vol. 220 (1983) pp. 671–680.CrossRefzbMATHMathSciNetGoogle Scholar
  41. [41]
    J. Knopman and J. Aude, Parallel simulated annealing: An adaptive approach, in International Parallel Processing Symposium, (Geneva, 1997 ) pp. 522–526.Google Scholar
  42. [42]
    V. Kumar, A. Grama, A. Gupta, and G. Karypis, Introduction to Parallel Computing: Design and Analysis of Algorithms, (The Benjamin/Cummings Publishing Company, Inc., 1994 ).zbMATHGoogle Scholar
  43. [43]
    V. Kumar and L. Kanal, The CDP: a unifying formulation for heuristic search, dynamic programming, and branch-and-bound, in National Conf. on A.I. (1983).Google Scholar
  44. [44]
    V. Kumar, K. Ramesh, and V. N. Rao, Parallel best-first search of state-space graphs: A summary of results, in Proceedings of the 1988 National Conf. on Artificial Intelligence (1988) pp. 122–127.Google Scholar
  45. [45]
    T. Lai and S. Sahni, Anomalies in parallel branch-and-bound algorithms, Communications of the ACM Vol. 27 (1984) pp. 594–602.CrossRefzbMATHMathSciNetGoogle Scholar
  46. [46]
    P. Laursen, Parallel heuristic search-introduction and a new approach, in A. Ferreira and P. Pardalos (eds.) Solving Combinatorial Optimization Problems in Parallel, volume 1054 of Lecture Notes in Computer Science, (Springer-Verlag, 1996 ) pp. 248–274.CrossRefGoogle Scholar
  47. [47]
    P. S. Laursen, Simple approaches to parallel Branch and Bound, Parallel Computing Vol. 19 (1993) pp. 143–152.CrossRefzbMATHGoogle Scholar
  48. [48]
    G. Li and B. Wah, Coping with anomalies in parallel branch-and-bound algorithms, IEEE Transactions on Computers Vol.C-35 No. 6 (1986) pp. 568–573.CrossRefMathSciNetGoogle Scholar
  49. [49]
    A. Linhares, Microcanonical optimization applied to the travelling salesperson problem (in portuguese), M.sc. dissertation (Applied Computing & Automation, Universidade Federal Fluminense, 1996 ).Google Scholar
  50. [50]
    R. Lüling and B. Monien, Load balancing for distributed branch & bound algorithms, in International Parallel Processing Symposium (Beverly Hills, 1992 ) pp. 543–549.Google Scholar
  51. [51]
    R. Lüling and B. Monien, Load balancing for distributed branch & bound algorithms, Technical Report Nr. 114, (Universität Gesamthochschule Paderborn, 1993 ).Google Scholar
  52. [52]
    R. Ma, F. Tsung, and M. Ma, A dynamic load balancer for a parallel branch and bound algorithm, in 1988 ACM Conf. on Lisp and Funct. Prog. (1988) pp. 1505–1513.Google Scholar
  53. [53]
    B. Manderick and P. Spiessens, Fine-grained parallel genetic algorithms, in Proceedings of ICGA 3 (1989) pp. 428–433.Google Scholar
  54. [54]
    B. Mans, Contribution à l’Algorithmique Non Numérique Parallèle: Parallélisation de Méthodes de Recherche Arborescente. PhD thesis (Université Paris V I, 1992 ).Google Scholar
  55. [55]
    T. Maruyama, T. Hirose, and A. Konagaya, A fine-grained parallel genetic algorithm for distributed parallel systems, in Proceedings of ICGA 5 (1993) pp. 184–190.Google Scholar
  56. [56]
    W. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller, Equation of state calculations by fast computing machines, Journal Chem. Phys. Vol. 21 (1953) pp. 1087–1092.CrossRefGoogle Scholar
  57. [57]
    Z. Michalewicz, Genetic algorithms + Data structures = Evolution programs, (Springer-Verlag, 1994 ).zbMATHGoogle Scholar
  58. [58]
    D. Miller and J. Pekny, Results from a parallel branch and bound algorithm for the asymmetric traveling salesman problem, Operations Research Letters Vol. 8 (1989) pp. 129–135.CrossRefzbMATHMathSciNetGoogle Scholar
  59. [59]
    L. Mitten, Branch-and-bound methods: General formulation and properties, Operations Research Vol.18 (1970) pp. 24–34. Errata in Operations Research Vol. 19 (1971) p. 550.Google Scholar
  60. [60]
    H. Muhlenbein, Parallel genetic algorithms, population genetics and combinatorial optimization, in J. Becker, I. Eisele, and F. Mundemann (eds.) Parallelism, Learning, Evolution, number 565 of Lecture Notes in Artificial Intelligence, (Springer-Verlag, 1989 ) pp. 398–406.Google Scholar
  61. [61]
    H. Muhlenbein and J. Kindermann, The dynamics of evolution and learning: Towards genetic neural networks, in R. P. et al. (eds.) Connectionism in Perspective (North-Holland, 1989 ) pp. 1753–197.Google Scholar
  62. [62]
    D. Nau, V. Kumar, and L. Kanal, General branch and bound, and its relation to A* and AO*, Artificial Intelligence Vol. 23 (1984) pp. 29–58.CrossRefzbMATHMathSciNetGoogle Scholar
  63. [63]
    G. Nemhauser and L. Wolsey, Integer and Combinatorial Optimization, (John Wiley and Sons Interscience, 1988 ).zbMATHGoogle Scholar
  64. [64]
    C. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity, ( Englewood Cliffs, Prentice Hall Inc., 1982 ).zbMATHGoogle Scholar
  65. [65]
    P. Pardalos and J. Crouse, A parallel algorithm for the quadratic assignment problem, in Proceedings Supercomputing 1989 (1989) pp. 351–360.Google Scholar
  66. [66]
    P. Pardalos, L. Pitsoulis, T. Mavridou, and M. Resende, Parallel search for combinatorial optimization: Genetic algorithms, simulated annealing, tabu search and grasp, in A. Ferreira and J. Rolim (eds.) Parallel algorithms for irregularly structured problems, volume 980 of Lecture Notes in Computer Science, (Springer-Verlag, 1996 ) pp. 317–331.Google Scholar
  67. [67]
    J. Pearl, Heuristics-Intelligent Search Strategies for Computer Problem Solving, ( Reading, Addison-Wesley, 1984 ).Google Scholar
  68. [68]
    J. Pekny and D. Miller, A parallel branch and bound algorithm for solving large asymmetric traveling salesman problems, Mathematical Programming Vol. 55 (1992) pp. 17–33.CrossRefzbMATHMathSciNetGoogle Scholar
  69. [69]
    C. Petty, M. Leuze, and J. Grefenstette, A parallel genetic algorithm, in Proceedings of the Second Int. Conf. on Genetic Algorithms, ( Cambridge, MIT, 1987 ) pp. 155–161.Google Scholar
  70. [70]
    S. Porto and C. Ribeiro, Parallel tabu search message-passing synchronous strategies for task scheduling under precedence constraints, Journal of Heuristics Vo1. 1 (1996) pp. 207–233.Google Scholar
  71. [71]
    S. Porto, J. Torreâo, and A. Barroso, A parallel microcanonical optimization algorithm for the task scheduling problem, in Metaheuristic International Conference (1997).Google Scholar
  72. [72]
    G. Robertson, Parallel implementation of genetic algorithms in a classifier system, in L. Davis (ed.) Genetic Algorithms and Simulated Annealing, ( London, Pitman, 1987 ) pp. 129–140.Google Scholar
  73. [73]
    P. Spiessens and B. Manderick, A massively parallel genetic algorithm: implementation and first analysis, in Proceedings of ICGA 4 (1989) pp. 279–286.Google Scholar
  74. [74]
    E. Taillard, Robust tabu search for the quadratic assignment problem, Parallel Computing Vol. 7 (1991) pp. 443–455.CrossRefMathSciNetGoogle Scholar
  75. [75]
    E. Taillard, Parallel taboo search techniques for the job shop scheduling problem, ORSA Journal on Computing Vol. 6 (1994) pp. 108–117.zbMATHGoogle Scholar
  76. [76]
    R. Tanse, Parallel genetic algorithms for a hypercube, in Proceedings of the Second Int. Conf. on Genetic Algorithms ( Cambridge, MIT, 1987 ) pp. 177–183.Google Scholar
  77. [77]
    J. Torreâo and E. Roe, Microcanonical optimization applied to visual processing, Physics Letters A Vol. 205 (1995) pp. 377–382.CrossRefGoogle Scholar
  78. [78]
    H. Trienekens, Parallel Branch and Bound Algorithms, PhD thesis (Erasmus University, 1990 ).Google Scholar
  79. [79]
    D. Whitley, T. Starkweather, and K. Mathias, Optimization using distributed genetic algorithms, in Proceedings of the First Conference on Parallel Problem Solving from Nature-PPSN I, volume 496 of Lecture Notes in Computer Science, (Springer-Verlag, 1990 ) pp. 134–144.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Ricardo C. Corrêa
    • 1
  • Afonso Ferreira
    • 2
  • Stella C. S. Porto
    • 3
  1. 1.Núcleo de Computação EletrônicaUniversidade Federal do Rio de JaneiroRio de JaneiroBrazil
  2. 2.SLOOP ProjectCNRS-I3S-INRIASophia-AntipolisFrance
  3. 3.Computação Aplicada e AutomaçãoUniversidade Federal FluminenseNiteróiBrazil

Personalised recommendations