Simulated Annealing Algorithms for Continuous Global Optimization

  • Marco Locatelli
Part of the Nonconvex Optimization and Its Applications book series (NOIA, volume 62)

Abstract

In this chapter we will consider Simulated Annealing algorithms for continuous global optimization. After a description of the generic Simulated Annealing algorithm, its four main components (the distribution of the next candidate point, the acceptance function, the cooling schedule and the stopping criterion) will be analyzed in greater detail and some key ideas will be presented. In view of the strict connection with Simulated Annealing algorithms, we will also discuss the Langevin equation and its discretized version. The theoretical issue of convergence in probability to the set of global optima of the sequences of iterates and candidates will be explored. We will also give some insight into the properties of the objective functions for which a successful application of Simulated Annealing algorithms can be expected. Finally, some other issues such as applications, computational tests and parallelization of these algorithms will be discussed.

Keywords

Local Search Simulated Annealing Global Optimization Feasible Region Langevin Equation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ali, M.M. and Storey, C. (1997). Aspiration based simulated annealing algorithm. Journal of Global Optimization, 11: 181–191.MathSciNetMATHCrossRefGoogle Scholar
  2. Aluffi-Pentini, F., Parisi, V., and Zirilli, F. (1985). Global optimization and stochastic differential equations. Journal of Optimization Theory and Applications, 47: 1–16.MathSciNetMATHCrossRefGoogle Scholar
  3. Basu, A. arid Frazer, L.N. (1990). Rapid determination of the criticaltemperature in simulated annealing inversion. Science, 249: 1409–1412.MATHGoogle Scholar
  4. Belisle, C.J.P. (1992). Convergence theorems for a class of simulated annealing algorithms on Rd. Journal of Applied Probability, 29: 885–892.MathSciNetMATHCrossRefGoogle Scholar
  5. Bilbro, G.L. and Snyder, W.E. (1991). Optimization of functions with many minima. IEEE Transactions on Systems, Man, and Cybernetics, 21: 840–849.MathSciNetCrossRefGoogle Scholar
  6. Biswas, R. and Hamann, D.R. (1986). Simulated annealing of silicon atom clusters in Langevin molecular dynamics. Physical Review B, 34: 895–901.CrossRefGoogle Scholar
  7. Bohachevsky, I.O., Johnson, M.E., and Stein, M.L. (1986). Generalized simulated annealing for function optimization. Technometries, 28: 209–217.MATHCrossRefGoogle Scholar
  8. Bollweg, W., Kroll, H., and Maurer, H. (1997). Numerical prediction of crystal structures by simulated annealing. In Bomze, I.M., Csendes, T., Horst, R., and Pardalos, P.M., editors, Developments in Global Optimization. Kluwer Academic Publishers, Dordrecht.Google Scholar
  9. Brooks, D.G. and Verdini, W.A. (1988). Computational experience with generalized simulated annealing over continuous variables. American Journal of Mathematical and Management Sciences, 8: 425–449.MathSciNetMATHGoogle Scholar
  10. Cerny, V. (1985). Thermodynamical approach to the travelling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Applications, 45: 41–51.MathSciNetMATHCrossRefGoogle Scholar
  11. Chan, T. (1989). On multi-dimensional annealing problems. Math. Proc. Camb. Phil. Soc., 105: 177–184.MATHCrossRefGoogle Scholar
  12. Chiang, T.-S., Hwang, C.-R., and Sheu, S.-J. (1987). Diffusion for global optimization in Rd. SIAM Journal on Control and Optimization, 25: 737–753.MathSciNetMATHCrossRefGoogle Scholar
  13. Coleman, T., Shalloway, D., and Wu, Z. (1993). Isotropic effective energy simulated annealing searches for low energy molecular cluster states. Computational Optimization and Applications, 2: 145–170.MathSciNetMATHCrossRefGoogle Scholar
  14. Coleman, T., Shalloway, D., and Wu, Z. (1994). A parallel build-up algorithm for global energy minimizations of molecular clusters using effective energy simulated annealing. Journal of Global Optimization, 4: 171–185.MathSciNetMATHCrossRefGoogle Scholar
  15. Corana, A., Marchesi, M., Martini, C., and Ridella, S. (1987). Minimizing multimodal functions of continuous variables with the ‘simulated annealing’ algorithm. ACM Trans. Math. Software, 13: 262–280.MathSciNetMATHCrossRefGoogle Scholar
  16. Dekkers, A. and Aarts, E. (1991). Global optimization and simulated annealing. Mathematical Programming, 50: 367–393.MathSciNetMATHCrossRefGoogle Scholar
  17. Desai, R. and Patil, R. (1996). SALO: combining simulated annealing and local optimization for efficient global optimization. In Proceedings of the 9th Florida AI Research Symposium (FLAIRS-’96), pages 233–237.Google Scholar
  18. Dixon, L.C.W. and Szegö, G.P. (1978). Towards Global Optimization 2. North-Holland, Amsterdam, The Netherlands.Google Scholar
  19. Dorea, C.C.Y. (1997). On the efficiency of a continuous version of the simulated annealing algorithm. Statistics ê4 Probability Letters, 31: 247–253.MathSciNetMATHCrossRefGoogle Scholar
  20. Doye, J.P.K., Miller, M.A., and Wales, D.J. (1999a). The double-funnel energy landscape of the 38-atom Lennard-Jones cluster. Journal of Chemical Physics, 110: 6896–6906.CrossRefGoogle Scholar
  21. Doye, J.P.K., Miller, M.A., and Wales, D.J. (1999b). Evolution of the potential energy surface with size for Lennard-Jones clusters. Journal of Chemical Physics, 111: 8417–8428.CrossRefGoogle Scholar
  22. Fang, H., Qian, M., and Gong, G. (1997). An improved annealing method and its large-time behavior. Stochastic Processes and their Applications, 71: 55–74.MathSciNetMATHCrossRefGoogle Scholar
  23. Fox, B.L. (1995). Simulated annealing: folklore, facts, and directions. In Monte Carlo and quasi-Monte Carlo methods in scientific computing (Las Vegas, NV, 1994), pages 17–48. Springer, New York.CrossRefGoogle Scholar
  24. Gelfand, S.B. and Mitter, S.K. (1991a). Recursive stochastic algorithms for global optimization in R d . SIAM Journal on Control and Optimization, 29: 999–1018.MathSciNetMATHCrossRefGoogle Scholar
  25. Gelfand, S.B. and Mitter, S.K. (1991b). Simulated annealing type algorithms for multivariate optimization. Algorithmica, 6: 419–436.MathSciNetMATHCrossRefGoogle Scholar
  26. Gelfand, S.B. and Mitter, S.K. (1991c). Weak convergence of Markov Chain sampling methods and annealing algorithms to diffusions. Journal of Optimization Theory and Applications, 68: 483–498.MathSciNetMATHCrossRefGoogle Scholar
  27. Gelfand, S.B. and Mitter, S.K. (1993). Metropolis-type annealing algorithms for global optimization in R d . SIAM Journal on Control and Optimization, 31: 111–131.MathSciNetMATHCrossRefGoogle Scholar
  28. Geman, S. and Hwang, C.-R. (1986). Diffusion for global optimization. SIAM Journal on Control and Optimization, 24: 1031–1043.MathSciNetMATHCrossRefGoogle Scholar
  29. Gidas, B. (1985). Global minimization via the Langevin equation. In Proceedings of 24th Conf. on Decision and Control, pages 774–778, Ft. Lauderdale, FL.Google Scholar
  30. Goffe, W.L., Ferrier, G.D., and Rogers, J. (1994). Global optimization of statistical functions with simulated annealing. Journal of Econometrics, 60: 65–99.MATHCrossRefGoogle Scholar
  31. Goldstein, L. (1988). Mean square rates of convergence in the continuous time simulated annealing on R d . Advances in Applied Mathematics, 9: 35–39.MATHCrossRefGoogle Scholar
  32. Gutmann, H.-M. (2001). A radial basis function method for global optimization. Journal of Global Optimization, 19: 201–227.MathSciNetMATHCrossRefGoogle Scholar
  33. Haario, H. and Saksman, E. (1991). Simulated annealing process in general state space. Adv. Appl. Prob., 23: 866–893.MathSciNetMATHCrossRefGoogle Scholar
  34. Hajek, B. (1988). Cooling schedules for optimal annealing. Mathematics of Operations Research, 13: 311–329.MathSciNetMATHCrossRefGoogle Scholar
  35. Hamma, B.S., Viitanen, S., and Torn, A. (1993). Parallel continuous simulated annealing for global optimization. Presented at the NATO Advanced Study Institute - Algorithms for Continuous Optimization: The State of the Art, Il Ciocco-Castelvecchio Pascoli, Italy.Google Scholar
  36. Horst, R. and Tuy, H. (1996). Global Optimization: Deterministic Approaches Springer, Heidelberg, 3rd enlarged edition.Google Scholar
  37. Huyer, W. and Neumaier, A. (1999). Global optimization by multilevel coordinate search. Journal of Global Optimization, 14: 331–355.MathSciNetMATHCrossRefGoogle Scholar
  38. Hwang, C.-R. and Sheu, S.-J. (1990). Large-time behavior of perturbed diffusion Markov processes with applications to the second eigenvalue problem for Fokker-Planck operators and simulated annealing. Acta Applicandae Mathematicae, 19: 253–295.MathSciNetMATHCrossRefGoogle Scholar
  39. Ingber, L. (1989). Very fast simulated re-annealing. Mathematical and Computer Modelling, 12: 967–973.MathSciNetMATHCrossRefGoogle Scholar
  40. Ingber, L. (1992). Genetic algorithms and very fast simulated re-annealing: A comparison. Mathematical and Computer Modelling, 16: 87–100.MathSciNetMATHCrossRefGoogle Scholar
  41. Ingber, L. (1993). Simulated annealing: Practice versus theory. Mathematical and Computer Modelling, 18: 29–57.MathSciNetMATHCrossRefGoogle Scholar
  42. Ingber, L. (1996). Adaptive simulated annealing (ASA): lessons learned. Control and Cybernetics, 25: 33–54.MATHGoogle Scholar
  43. Jones, A.E.W. and Forbes, G.W. (1995). An adaptive simulated annealing algorithm for global optimization over continuous variables. Journal of Global Optimization, 6: 1–37.MathSciNetMATHCrossRefGoogle Scholar
  44. Jones, D.R., Perttunen, C., and Stuckman, B.E. (1993). Lipschitz optimization without the Lipschitz constant. Journal of Optimization Theory and Applications, 78: 157–181.MathSciNetCrossRefGoogle Scholar
  45. Jones, D.R., Schonlau, M., and Welch, W.J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13: 455–492.MathSciNetMATHCrossRefGoogle Scholar
  46. Kaufman, D.E. and Smith, R.L. (1998). Direction choice for accelerated convergence in Hit-and-Run sampling. Operations Research, 46: 84–95.MathSciNetMATHCrossRefGoogle Scholar
  47. Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P. (1983). Optimization by simulated annealing. Science, 220: 671–680.MathSciNetMATHCrossRefGoogle Scholar
  48. Kushner, H.J. (1987). Asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects: global minimization via Monte Carlo. SIAM J. Appl. Math., 47: 169–185.MathSciNetMATHCrossRefGoogle Scholar
  49. Locatelli, M. (1996). Convergence properties of simulated annealing for continuous global optimization. Journal of Applied Probability, 33: 1127–1140.MathSciNetMATHCrossRefGoogle Scholar
  50. Locatelli, M. (2000a). Convergence of a simulated annealing algorithm for continuous global optimization. Journal of Global Optimization, 18: 219–233.MathSciNetMATHCrossRefGoogle Scholar
  51. Locatelli, M. (2000b). Simulated annealing algorithms for continuous global optimization: Convergence conditions. Journal of Optimization Theory and Applications, 104: 121–133.MathSciNetMATHCrossRefGoogle Scholar
  52. Locatelli, M. (2001). Convergence and first hitting time of simulated annealing algorithms for continuous global optimization. Mathematical Methods of Operations Research, to appear.Google Scholar
  53. Lucidi, S. and Piccioni, M. (1989). Random tunneling by means of acceptance-rejection sampling for global optimization. Journal of Optimization Theory and Applications, 62: 255–277.MathSciNetMATHCrossRefGoogle Scholar
  54. Marquez, D. (1997). Convergence rates for annealing diffusion processes. Annals of Applied Probability, 7: 1118–1139.MathSciNetMATHCrossRefGoogle Scholar
  55. Metropolis, N., Rosenbluth, A.W., and Teller, A.H. (1953). Equation of state calculations by fast computer machines. J. Chem. Phys., 21: 1087–1092.CrossRefGoogle Scholar
  56. Miclo, L. (1992). Recuit simulé sur I[. étude de l’évolution de l’énergie libre. Ann. Inst. Henry Poincaré, 28: 235–266.MathSciNetMATHGoogle Scholar
  57. Mühlenbein, H., Schomisch, M., and Born, J. (1991). The parallel genetic algorithm as function optimizer. Parallel Computing, 17: 619–632.MATHCrossRefGoogle Scholar
  58. Pannetier, J., Bassas-Alsina, J., and Rodriguez-Carvajal, J. (1990). Prediction of crystal structures from crystal chemistry rules by simulated annealing. Nature, 346: 343–345.CrossRefGoogle Scholar
  59. Pelletier, M. (1998). Weak convergence rates for stochastic approximation with application to multiple targets and simulated annealing. Annals of Applied Probability, 8: 10–44.MathSciNetMATHCrossRefGoogle Scholar
  60. Piccioni, M. (1987). A combined multistart-annealing algorithm for continuous global optimization. Technical Research Report 87–45, Systems and Research Center, The University of Maryland, College Park, MD.Google Scholar
  61. Romeijn, H.E. and Smith, R.L. (1994a). Simulated annealing and adaptive search in global optimization. Probability in the Engineering and Informational Sciences, 8: 571–590.CrossRefGoogle Scholar
  62. Romeijn, H.E. and Smith, R.L. (1994b). Simulated annealing for constrained global optimization. Journal of Global Optimization, 5: 101–126.MathSciNetMATHCrossRefGoogle Scholar
  63. Romeijn, H.E., Zabinski, Z.B., Graesser, D.L., and Neogi, S. (1999). New reflection generator for simulated annealing in mixed-integer/continuous global optimization. Journal of Optimization Theory and Applications, 101: 403–427.MathSciNetMATHCrossRefGoogle Scholar
  64. Royer, G. (1989). A remark on simulated annealing of diffusion processes. SIAM Journal on Control and Optimization, 27: 1403–1408.MathSciNetMATHCrossRefGoogle Scholar
  65. Schön, J.C. (1997). Preferential trapping on energy landscapes in regions containing deep-lying minima: the reason for the success of simulated annealing? J. Phys. A: Math. Gen., 30: 2367–2389.CrossRefGoogle Scholar
  66. Schuur, P.C. (1997). Classification of acceptance criteria for the simulated annealing algorithm. Mathematics of Operations Research, 22: 266–275.MathSciNetMATHCrossRefGoogle Scholar
  67. Semenovskaya, S., Khachaturyan, K.A., and Khachaturyan, A.G. (1985). Statistical mechanics approach to the structure determination of a crystal. Acta Cryst, A41: 268–273.CrossRefGoogle Scholar
  68. Siarry, P., Berthiau, G., Durbin, F., and Haussy, J. (1997). Enhanced simulated annealing for globally minimizing functions of many continuous variables. ACM Transactions on Mathematical Software, 23: 209–228.MathSciNetMATHCrossRefGoogle Scholar
  69. Sorkin, G.B. (1991). Efficient simulated annealing on fractal energy landscapes. Algorithmica, 6: 367–418.MathSciNetMATHCrossRefGoogle Scholar
  70. Szu, H. and Hartley, R. (1987). Fast simulated annealing. Phys. Lett. A, 122: 157–162.CrossRefGoogle Scholar
  71. Theodorocatos, V.E. and Grimsley, J.L. (1995). The optimal packing of arbitrarily-shaped polygons using simulated annealing and polynomial-time cooling schedules. Comput. Methods Appl. Mech. Engrg., 125: 53–70.MathSciNetCrossRefGoogle Scholar
  72. van Laarhoven, P.J.M. and Aarts, E.H.L. (1987). Simulated annealing: theory and applications. D.Reidel Publishing Co., Dordrecht-Boston, Mass.MATHCrossRefGoogle Scholar
  73. Vanderbilt, D. and Louie, S.G. (1984). A monte carlo simulated annealing approach to optimization over continuous variables. Journal of Computational Physics, 56: 259–271.MathSciNetMATHCrossRefGoogle Scholar
  74. Wales, D.J. and Doye, J.P.K. (1997). Global optimization by basin-hopping and the lowest energy structures of Lennard-Jones clusters containing up to 110 atoms. Chem. Phys. Lett., 269: 408–412.CrossRefGoogle Scholar
  75. Wang, P.P. and Chen, D.-S. (1996). Continuous optimization by a variant of simulated annealing. Computational Optimization and Applications, 6: 59–71.MathSciNetMATHCrossRefGoogle Scholar
  76. Wille, L.T. (1987a). Close packing in curved space by simulated annealing. J. Phys. A: Math. Gen., 20: L1211–L1218.MathSciNetCrossRefGoogle Scholar
  77. Wille, L.T. (1987b). Minimum-energy configurations of atomic clusters: new results obtained by simulated annealing. Chemical Physics Letters, 133: 405–410.CrossRefGoogle Scholar
  78. Wilson, S. and Cui, W. (1990). Applications of simulated annealing to peptides. Biopolymers, 29: 225–235.CrossRefGoogle Scholar
  79. Xue, G.L. (1994). Molecular conformation on the cm-5 by parallel two-level simulated annealing. Journal of Global Optimization, 4: 187–208.MATHCrossRefGoogle Scholar
  80. Yang, R.L. (2000). Convergence of the simulated annealing algorithm for continuous global optimization. Journal of Optimization Theory and Applications, 104: 691–716.MathSciNetMATHCrossRefGoogle Scholar
  81. Yong, L., Lishan, K., and Evans, D.J. (1995). The annealing evolution algorithm as function optimizer. Parallel Computing, 21: 389–400.MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2002

Authors and Affiliations

  • Marco Locatelli
    • 1
  1. 1.Dipartimento di InformaticaUniversità di TorinoTorinoItaly

Personalised recommendations