Stochastic Search in Metaheuristics

  • Walter J. Gutjahr
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 146)


Stochastic search is a key mechanism underlying many metaheuristics. The chapter starts with the presentation of a general framework algorithm in the form of a stochastic search process that contains a large variety of familiar metaheuristic techniques as special cases. Based on this unified view, questions concerning convergence and runtime are discussed on the level of a theoretical analysis. Concrete examples from diverse metaheuristic fields are given. In connection with runtime results, important topics as instance difficulty, phase transitions, parameter choice, No-Free-Lunch theorems, or fitness landscape analysis are addressed. Furthermore, a short sketch of the theory of black-box optimization is given, and generalizations of results to stochastic search under noise are outlined.


Particle Swarm Optimization Simulated Annealing Variable Neighborhood Search Fitness Landscape Metaheuristic Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Achlioptas, D., Naor, A., Peres, Y.: Rigorous location of phase transitions in hard optimization problems. Nature 435, 759–764 (2005)CrossRefGoogle Scholar
  2. 2.
    A simulated annealing algorithm with constant temperature for discrete stochastic optimization. Manage. Sci. 45, 748–764 (1999)Google Scholar
  3. 3.
    Barbosa, V.C., Ferreira, R.G.: On the phase transitions of graph coloring and independent sets. Physika A 343, 401–423 (2004)Google Scholar
  4. 4.
    Bianchi, L., Dorigo, M., Gambardella, L.M., Gutjahr, W.J.: A survey on metaheuristics for stochastic combinatorial optimization. To appear In: Natural Computing 8, 239–287 (2009)Google Scholar
  5. 5.
    Birattari, M., Balaprakash, P., Dorigo, M.: The ACO/FRACE algorithm for combinatorial optimization under uncertainty. In: Doerner K. et al. (ed.) Metaheuristics – Progress in Complex Systems Optimization, Springer: Berlin, Germany (2006)Google Scholar
  6. 6.
    Borenstein, Y., Poli, R.: Information Perspective of Optimization. Proceedings of 9th Conference on Parallel Problem Solving from Nature, Springer LNCS, vol. 4193, pp. 102–111. Berlin Heidelberg (2006)Google Scholar
  7. 7.
    Borenstein, Y. Poli, R.: Structure and Metaheuristics. Proceedings of Genetic and Evolutionary Computation Conference ’06, pp. 1087–1093 (2006)Google Scholar
  8. 8.
    Borisovsky, P.A., Eremeev, A.V.: A study on performance of the (1+1)-evolutionary algorithm. In: Proceedings of Foundations of Genetic Algorithms, vol. 7, pp. 271–287. Morgan Kaufmann, San Francisco (2003)Google Scholar
  9. 9.
    Cheesman, P., Kenafsky, B., Taylor, W.M.: Where the really hard problems are. In: Morgan Kaufmann (ed.) Proceedings of IJCAI ’91, pp. 331–337 (1991)Google Scholar
  10. 10.
    Droste, S., Jansen, T., Wegener, I.: Perhaps not a free lunch but at least a free appetizer. Proceedings of Genetic and Evolutionary Computation Conference ’99, Orlando, FL, USA pp. 833–839 (1999)Google Scholar
  11. 11.
    Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) evolutionary algorithm. Theor. Comput. Sci. 276, 51–81 (2002)CrossRefGoogle Scholar
  12. 12.
    Droste, S., Jansen, T., Wegener, I.: Upper and lower bounds for randomized search heuristics in black-box optimization. Theory Comput. Syst. 39 525–544 (2006)CrossRefGoogle Scholar
  13. 13.
    Dorigo, M., Maniezzo, V., Colorni, A.: Ant System: optimization by a colony of cooperating agents. IEEE Trans. Syst, Man Cybern 26, 1–13 (1996)Google Scholar
  14. 14.
    English, T.: Optimization is easy and learning is hard in the typical function. Proceedings of Congress in Evolutionary Computation ’00, La Jolla, USA pp. 924–931 (2000)Google Scholar
  15. 15.
    English, T.: On the structure of sequential search: beyond “no free lunch”. Proc. EvoCOP ’04, Springer LNCS, Coimbra, Portugal 3004, 95–103 (2004)Google Scholar
  16. 16.
    Eremeev, A.V., Reeves, C.R.: On confidence intervals for the number of local optima. Applications of Evolutionary Computing, Springer LNCS, Berlin Heidelberg vol. 2611, pp. 224–235 (2003)Google Scholar
  17. 17.
    Ferreira, F.F., Fontanari, J.F.: Probabilistic analysis of the number partitioning problem. J. Phys. A: Math. Gen. 31, 3417–3428 (1998)CrossRefGoogle Scholar
  18. 18.
    Fischetti, M., Lodi, A.: Local branching. Mathematical Programming Ser. B 98, 23–47 (2003)CrossRefGoogle Scholar
  19. 19.
    Friedrich, T., He, J., Hebbinghaus, N., Neumann, F., Witt, C.: Approximating covering problems by randomized search heuristics using multi-objective models. Proceedings of 9th Annual Conference on Genetic and Evolutionary Computation, pp. 797–804 (2007)Google Scholar
  20. 20.
    Garnier, J., Kalel, L., Schoenauer, M.: Rigorous hitting times for binary mutations. Evol. Comput. 7, 45–68 (1999)CrossRefGoogle Scholar
  21. 21.
    Garnier, J., Kallel, L.: Efficiency of local search with multiple local optima. SIAM J. Discrete Math. 15, 122–141 (2002)CrossRefGoogle Scholar
  22. 22.
    Gelfand, S.B., Mitter, S.K.: Simulated annealing with noisy or imprecise measurements. J. Optim. Theor. Appl. 69, 49–62 (1989)CrossRefGoogle Scholar
  23. 23.
    Gendreau, M., Laporte, G., Seguin, R.: An exact algorithm for the vehicle routing problem with stochastic demands and customers. Transport. Sci. 29, 143–155 (1995)CrossRefGoogle Scholar
  24. 24.
    Gent, I.P., Walsh, T.: Analysis of heuristics for number partitioning. Comput. Intell. 14, 430–450 (1998)CrossRefGoogle Scholar
  25. 25.
    Giel, O., Wegener, I.: Evolutionary algorithms and the maximum matching problem. Proceedings of 20th Annual Symposium on Theoretical Aspects of Computer Science, Seattle, Washington, USA pp. 415–426 (2003)Google Scholar
  26. 26.
    González, C., Lozano, J.A., Larrañaga, P.: Analyzing the PBIL algorithm by means of discrete dynamical systems. Complex Syst. 11, 1–15 (1997)Google Scholar
  27. 27.
    González, C., Lozano, J.A., Larrañaga, P.: Mathematical modelling of discrete estimation of distribution algorithms. In: Larrañaga et al. (eds.) Estimation of Distribution Algorithms, A New Tool for Evolutionary Computation, Kluwer, Dordrecht 147–163 (2002)Google Scholar
  28. 28.
    Gutjahr, W.J.: A graph–based ant system and its convergence. Future Gen. Comput. Syst. 16, 873–888 (2000)CrossRefGoogle Scholar
  29. 29.
    Gutjahr, W.J.: ACO algorithms with guaranteed convergence to the optimal solution. In. Process. Lett. 82, 145–153 (2002)CrossRefGoogle Scholar
  30. 30.
    Gutjahr, W.J.: A converging ACO algorithm for stochastic combinatorial optimization. Proceedings of 2nd Symposium on Stochastic Algorithms, Foundations and Applications, Springer LNCS, Berlin Heidelberg vol. 2827, pp. 10–25 (2003)Google Scholar
  31. 31.
    Gutjahr, W.J.: S-ACO: An ant-based approach to combinatorial optimization under uncertainty. Proceedings of 4nd Int. Workshop on Ant Colony Optimization and Swarm Intelligence, Springer LNCS, Berlin Heidelberg New York vol. 3172, pp. 238–249 (2004)Google Scholar
  32. 32.
    Gutjahr, W.J.: On the finite-time dynamics of ant colony optimization. Methodol. Comput. Appl. Probability 8, 105–133 (2006)CrossRefGoogle Scholar
  33. 33.
    Gutjahr, W.J.: Mathematical runtime analysis of ACO algorithms: survey on an emerging issue. Swarm Intell. 1, 59–79 (2007)CrossRefGoogle Scholar
  34. 34.
    Gutjahr, W.J.: First steps to the runtime complexity analysis of ant colony optimization. Comput. Oper. Res. 35, 2711–2727 (2008)CrossRefGoogle Scholar
  35. 35.
    Gutjahr, W.J., Katzensteiner, S., Reiter, P.: A VNS algorithm for noisy problems and its application to project portfolio analysis. Proceedings of SAGA 2007 (Stochastic Algorithms: Foundations and Applications), Springer LNCS, Berlin Heidelberg vol. 4665, pp. 93–104 (2007)Google Scholar
  36. 36.
    Gutjahr, W.J., Pflug, G.: Simulated annealing for noisy cost functions. J. Global Optim. 8, 1–13 (1996)CrossRefGoogle Scholar
  37. 37.
    Gutjahr, W.J., Sebastiani, G.: Runtime analysis of ant colony optimization with best-so-far reinforcement. Methodology and Computing in Applied Probability 10, 409–433 (2008)CrossRefGoogle Scholar
  38. 38.
    Hajek, B.: Cooling schedules for optimal annealing. Math. of Operat. Res. 13, 311–329 (1988)CrossRefGoogle Scholar
  39. 39.
    Hartl, R.F.: A global convergence proof for a class of genetic algorithms. Technical Report, University of Vienna (1990)Google Scholar
  40. 40.
    Hartmann, A.K., Barthel, W., Weigt, M.: Phase transition and finite-size scaling in the vertex-cover problem. Comput. Phys. Commn. 169, 234–237 (2005)CrossRefGoogle Scholar
  41. 41.
    He, J., Yao, X.: Drift analysis and average time complexity of evolutionary algorithms. Artifi. Intell. 127, 57–85 (2003)CrossRefGoogle Scholar
  42. 42.
    He, J., Yao, X.: Towards an analytic framework for analysing the computation time of evolutionary algorithms. Artif. Intell. 145, 59–97 (2003)CrossRefGoogle Scholar
  43. 43.
    He, J., Yao, X.: A study of drift analysis for estimating computation time of evolutionary algorithms. Nat. Comput. 3, 21–35 (2004)CrossRefGoogle Scholar
  44. 44.
    He, J., Yu, X.: Conditions for the convergence of evolutionary algorithms. J. Syst. Arch. 47, 601–612 (2001)CrossRefGoogle Scholar
  45. 45.
    Herroelen, W., De Reyck, B.: Phase transitions in project scheduling. J. Oper. Res. Soc. 50, 148–156 (1999)Google Scholar
  46. 46.
    Igel, C., Toussaint, M.: On classes of functions for which no free lunch results hold. Inf. Process. Lett. 86, 317–321 (2003)CrossRefGoogle Scholar
  47. 47.
    Igel, C., Toussaint, M.: A no-free-lunch theorem for non-uniform distributions of target functions. J. Math. Model. Algorithms 3, 313–322 (2004)CrossRefGoogle Scholar
  48. 48.
    Jacobson, S.H., Yücesan, E.: Analyzing the performance of generalized hill climbing algorithms. J. Heuristics 10, 387–405 (2004)CrossRefGoogle Scholar
  49. 49.
    Jansen, T., Wegener, I.: On the analysis of a dynamic evolutionary algorithm. J. Discrete Algorithms 4, 181–199 (2006)CrossRefGoogle Scholar
  50. 50.
    Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments – a survey. IEEE Trans. Evol. Comput. 9, 303–317 (2005).CrossRefGoogle Scholar
  51. 51.
    Kennedy, J., Eberhart, R.C.: A discrete binary version of the particle swarm algorithm. Proceedings of the World Multiconference on Systemics, Cybernetics and Informatics, Orlando, FL, USA pp. 4104–4109 (1997)Google Scholar
  52. 52.
    Koehler, G.J.: Conditions that obviate the no-free-lunch theorems for optimization. Informs J. Comput. 19, 273–279 (2007)CrossRefGoogle Scholar
  53. 53.
    Ladret, V.: Asymptotic hitting time for a simple evolutionary model of protein folding. J. Appl. Probability 42, 39–51 (2005)CrossRefGoogle Scholar
  54. 54.
    Margolin, L.: On the convergence of the cross-entropy method. Ann. Oper. Res. 134, 201–214 (2005)CrossRefGoogle Scholar
  55. 55.
    Martin, O.C., Monasson, R., Zecchina, R.: Statistical mechanics methods and phase transitions in optimization problems. Theor. Compu. Sci. 265, 3–67 (2001)CrossRefGoogle Scholar
  56. 56.
    Merelo, J.-J., Cotta, C.: Building bridges: the role of subfields in metaheuristics. SIGEVOlution 1(4), 9–15 (2006)CrossRefGoogle Scholar
  57. 57.
    Mertens, S.: A physicist’s approach to number partitioning. Theor. Comput. Sci. 265, 79–108 (2001)CrossRefGoogle Scholar
  58. 58.
    Merz, P., Freisleben, B.: Fitness landscape analysis and memetic algorithms for the quadratic assignment problem. IEEE Trans. Evol Comput 4, 337–352 (2000)CrossRefGoogle Scholar
  59. 59.
    Monasson, R.: Introduction to phase transitions in random optimization problems. Technical Report, Laboratoire de Physique Theorique de l’ENS, Paris (2007)Google Scholar
  60. 60.
    Neumann, F., Wegener, I.: Randomized local search, evolutionary algorithms, and the minimum spanning tree problem. Theor Comp. Sci. 378, 32–40 (2007).CrossRefGoogle Scholar
  61. 61.
    Neumann, F., Witt, C.: Runtime analysis of a simple ant colony optimization algorithm. Proceedings of ISAAC ’06, Springer LNCS, Berlin Heidelberg vol. 4288, pp. 618–627 (2006)Google Scholar
  62. 62.
    Neumann, F., Witt, C.: Ant colony optimization and the minimum spanning tree problem. Proceedings of LION ’08, Learning and Intelligent Optimization, Theoretical Computer Science 411, 2406–2413 (2010)Google Scholar
  63. 63.
    Norman, F.: Markov Processes and Learning Models. Academic Press, New York (1972)Google Scholar
  64. 64.
    Oliveto, P.S., He, J., Yao, X.: Time complexity of evolutionary algorithms for combinatorial optimization: a decade of results. Int. J. Automat. and Comput. 4, 281–293 (2007)CrossRefGoogle Scholar
  65. 65.
    Oliveto, P.S., He, J., Yao, X.: Evolutionary algorithms and the vertex cover problem. Proceedings of the Congress on Evolutionary Computation CEC ’07, Singapore pp. 1870–1877 (2007)Google Scholar
  66. 66.
    Purkayastha, P., Baras, J.S.: Convergence results for ant routing algorithms via stochastic approximation and optimization. Proceedings of 46th IEEE Conference on Decision and Control, pp. 340–345 (2007)Google Scholar
  67. 67.
    Reidys C.M., Stadler, P.F.: Combinatorial landscapes. SIAM Rev. 44, 3–54 (2002)CrossRefGoogle Scholar
  68. 68.
    Rudolph, G.: Convergence Analysis of canonical genetic algorithms. IEEE Trans. Neural. Netw. 5, 96–101 (1994)CrossRefGoogle Scholar
  69. 69.
    Sasaki, G.H., Hajek, B.: The time complexity of maximum matching by simulated annealing. J. ACM 35, 67–89 (1988)CrossRefGoogle Scholar
  70. 70.
    Scharnow, J., Tinnefeld, K., Wegener, I.: Fitness landscapes based on sorting and shortest path problems. Proceedings of 7th Conference on Parallel Problem Solving from Nature, 54–63 (2002)Google Scholar
  71. 71.
    Schiavinotto, T., Stûtzle, T.: A review of metrics on permutations for search landscape analysis. Comput. Oper. Res. 34, 3143–3153 (2007)CrossRefGoogle Scholar
  72. 72.
    Sebastiani, G., Torrisi, G.L.: An extended ant colony algorithm and its convergence analysis. Methodol. Comput. Appl. Probability 7, 249–263 (2005)CrossRefGoogle Scholar
  73. 73.
    Storch, T.: How randomized search heuristics find maximum cliques in planar graphs. Proceedings of 8th Annual Conference on Genetic and Evolutionary Computation, Seattle, Washington, USA pp. 567–574 (2006)Google Scholar
  74. 74.
    Stützle, T., Hoos, H.H.: MAX-MIN Ant System. Future Gen. Comput. Sys. 16, 889–914 (2000)CrossRefGoogle Scholar
  75. 75.
    Stützle, T., Dorigo, M.: A short convergence proof for a class of ACO algorithms. IEEE Trans. Evol. Comput. 6, 358–365 (2002)CrossRefGoogle Scholar
  76. 76.
    Sudholt, D., Witt, C.: Runtime analysis of binary PSO. Proceedings of 10th Annual Conf. on Genetic and Evolutionary Computation, New York, USA pp. 135–142 (2008)Google Scholar
  77. 77.
    Teytaud, O., Gelly, S.: General lower bounds for evolutionary algorithms. Proceedings of 9th Conference on Parallel Problem Solving from Nature, pp. 21–31 (2006)Google Scholar
  78. 78.
    Trelea, I.C.: The particle swarm optimization algorithm: convergence analysis and parameter selection. Inf. Process. Lett. 85, 317–325 (2003)CrossRefGoogle Scholar
  79. 79.
    Vaughan, D.E., Jacobson, S.H., Kaul, H.: Analyzing the performance of simultaneous generalized hill climbing algorithms. Comput. Optim. Appl. 37, 103–119 (2007)CrossRefGoogle Scholar
  80. 80.
    Wegener, I.: Simulated annealing beats metropolis in combinatorial optimization. Proceedings of ICALP ’05, Springer LNCS, Berlin Heidelberg vol. 3580, pp. 589–601 (2005)Google Scholar
  81. 81.
    Wegener, I., Witt, C.: On the analysis of a simple evolutionary algorithm on quadratic pseudo-boolean functions. J. Discrete Algorithms 3, 61–78 (2005)CrossRefGoogle Scholar
  82. 82.
    Whitley, D., Watson, J.P.: Complexity theory and the no free lunch theorem. In: Burke, E.K., Kendall, G. (eds.) Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, Kluwer, Boston, 317–399 (2005)Google Scholar
  83. 83.
    Witt, C.: Worst-case and average-case approximations by simple randomized search heuristics. Proceedings of 22nd Annual Symposium on Theoretical Aspects of Computer Science, Springer LNCS, Berlin Heidelberg vol. 3404, pp. 44–56 (2005)Google Scholar
  84. 84.
    Witt, C.: Runtime analysis of the \((\mu+1)\) EA on simple pseudo-bolan functions. Proceedings of 8th Annual Conference on Genetic and Evolutionary Computation, Seattle, Washington, pp. 651–658 (2006)Google Scholar
  85. 85.
    Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1, 67–82 (1997).CrossRefGoogle Scholar
  86. 86.
    Yao, A.C.: Probabilistic computations: towards a unified measure of complexity. Proceedings of 17th IEEE Symposium on the Foundations of Computer Science, 222–227 (1977)Google Scholar
  87. 87.
    Yu, Y., Zhou, Z.-H.: A new approach to estimating the expected first hitting time of evolutionary algorithms. Proceedings of 21th National Conference on Artificial Intelligence, Boston, MA, 555–560 (2006)Google Scholar
  88. 88.
    Zhang, W.: Phase transitions and backbones of the asymmetric travelling salesman problem. J. Artif. Intell. Res. 21, 471–497 (2004)Google Scholar
  89. 89.
    Zlochin, M., Birattari, M., Meuleau, N., Dorigo, M.: Model-based search for combinatorial optimization: a critical survey. Ann. Oper. Res. 131, 373–379 (2004)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.University of ViennaViennaAustria

Personalised recommendations