Abstract
In this chapter we will consider Simulated Annealing algorithms for continuous global optimization. After a description of the generic Simulated Annealing algorithm, its four main components (the distribution of the next candidate point, the acceptance function, the cooling schedule and the stopping criterion) will be analyzed in greater detail and some key ideas will be presented. In view of the strict connection with Simulated Annealing algorithms, we will also discuss the Langevin equation and its discretized version. The theoretical issue of convergence in probability to the set of global optima of the sequences of iterates and candidates will be explored. We will also give some insight into the properties of the objective functions for which a successful application of Simulated Annealing algorithms can be expected. Finally, some other issues such as applications, computational tests and parallelization of these algorithms will be discussed.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ali, M.M. and Storey, C. (1997). Aspiration based simulated annealing algorithm. Journal of Global Optimization, 11: 181–191.
Aluffi-Pentini, F., Parisi, V., and Zirilli, F. (1985). Global optimization and stochastic differential equations. Journal of Optimization Theory and Applications, 47: 1–16.
Basu, A. arid Frazer, L.N. (1990). Rapid determination of the criticaltemperature in simulated annealing inversion. Science, 249: 1409–1412.
Belisle, C.J.P. (1992). Convergence theorems for a class of simulated annealing algorithms on Rd. Journal of Applied Probability, 29: 885–892.
Bilbro, G.L. and Snyder, W.E. (1991). Optimization of functions with many minima. IEEE Transactions on Systems, Man, and Cybernetics, 21: 840–849.
Biswas, R. and Hamann, D.R. (1986). Simulated annealing of silicon atom clusters in Langevin molecular dynamics. Physical Review B, 34: 895–901.
Bohachevsky, I.O., Johnson, M.E., and Stein, M.L. (1986). Generalized simulated annealing for function optimization. Technometries, 28: 209–217.
Bollweg, W., Kroll, H., and Maurer, H. (1997). Numerical prediction of crystal structures by simulated annealing. In Bomze, I.M., Csendes, T., Horst, R., and Pardalos, P.M., editors, Developments in Global Optimization. Kluwer Academic Publishers, Dordrecht.
Brooks, D.G. and Verdini, W.A. (1988). Computational experience with generalized simulated annealing over continuous variables. American Journal of Mathematical and Management Sciences, 8: 425–449.
Cerny, V. (1985). Thermodynamical approach to the travelling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Applications, 45: 41–51.
Chan, T. (1989). On multi-dimensional annealing problems. Math. Proc. Camb. Phil. Soc., 105: 177–184.
Chiang, T.-S., Hwang, C.-R., and Sheu, S.-J. (1987). Diffusion for global optimization in Rd. SIAM Journal on Control and Optimization, 25: 737–753.
Coleman, T., Shalloway, D., and Wu, Z. (1993). Isotropic effective energy simulated annealing searches for low energy molecular cluster states. Computational Optimization and Applications, 2: 145–170.
Coleman, T., Shalloway, D., and Wu, Z. (1994). A parallel build-up algorithm for global energy minimizations of molecular clusters using effective energy simulated annealing. Journal of Global Optimization, 4: 171–185.
Corana, A., Marchesi, M., Martini, C., and Ridella, S. (1987). Minimizing multimodal functions of continuous variables with the ‘simulated annealing’ algorithm. ACM Trans. Math. Software, 13: 262–280.
Dekkers, A. and Aarts, E. (1991). Global optimization and simulated annealing. Mathematical Programming, 50: 367–393.
Desai, R. and Patil, R. (1996). SALO: combining simulated annealing and local optimization for efficient global optimization. In Proceedings of the 9th Florida AI Research Symposium (FLAIRS-’96), pages 233–237.
Dixon, L.C.W. and Szegö, G.P. (1978). Towards Global Optimization 2. North-Holland, Amsterdam, The Netherlands.
Dorea, C.C.Y. (1997). On the efficiency of a continuous version of the simulated annealing algorithm. Statistics ê4 Probability Letters, 31: 247–253.
Doye, J.P.K., Miller, M.A., and Wales, D.J. (1999a). The double-funnel energy landscape of the 38-atom Lennard-Jones cluster. Journal of Chemical Physics, 110: 6896–6906.
Doye, J.P.K., Miller, M.A., and Wales, D.J. (1999b). Evolution of the potential energy surface with size for Lennard-Jones clusters. Journal of Chemical Physics, 111: 8417–8428.
Fang, H., Qian, M., and Gong, G. (1997). An improved annealing method and its large-time behavior. Stochastic Processes and their Applications, 71: 55–74.
Fox, B.L. (1995). Simulated annealing: folklore, facts, and directions. In Monte Carlo and quasi-Monte Carlo methods in scientific computing (Las Vegas, NV, 1994), pages 17–48. Springer, New York.
Gelfand, S.B. and Mitter, S.K. (1991a). Recursive stochastic algorithms for global optimization in R d . SIAM Journal on Control and Optimization, 29: 999–1018.
Gelfand, S.B. and Mitter, S.K. (1991b). Simulated annealing type algorithms for multivariate optimization. Algorithmica, 6: 419–436.
Gelfand, S.B. and Mitter, S.K. (1991c). Weak convergence of Markov Chain sampling methods and annealing algorithms to diffusions. Journal of Optimization Theory and Applications, 68: 483–498.
Gelfand, S.B. and Mitter, S.K. (1993). Metropolis-type annealing algorithms for global optimization in R d . SIAM Journal on Control and Optimization, 31: 111–131.
Geman, S. and Hwang, C.-R. (1986). Diffusion for global optimization. SIAM Journal on Control and Optimization, 24: 1031–1043.
Gidas, B. (1985). Global minimization via the Langevin equation. In Proceedings of 24th Conf. on Decision and Control, pages 774–778, Ft. Lauderdale, FL.
Goffe, W.L., Ferrier, G.D., and Rogers, J. (1994). Global optimization of statistical functions with simulated annealing. Journal of Econometrics, 60: 65–99.
Goldstein, L. (1988). Mean square rates of convergence in the continuous time simulated annealing on R d . Advances in Applied Mathematics, 9: 35–39.
Gutmann, H.-M. (2001). A radial basis function method for global optimization. Journal of Global Optimization, 19: 201–227.
Haario, H. and Saksman, E. (1991). Simulated annealing process in general state space. Adv. Appl. Prob., 23: 866–893.
Hajek, B. (1988). Cooling schedules for optimal annealing. Mathematics of Operations Research, 13: 311–329.
Hamma, B.S., Viitanen, S., and Torn, A. (1993). Parallel continuous simulated annealing for global optimization. Presented at the NATO Advanced Study Institute - Algorithms for Continuous Optimization: The State of the Art, Il Ciocco-Castelvecchio Pascoli, Italy.
Horst, R. and Tuy, H. (1996). Global Optimization: Deterministic Approaches Springer, Heidelberg, 3rd enlarged edition.
Huyer, W. and Neumaier, A. (1999). Global optimization by multilevel coordinate search. Journal of Global Optimization, 14: 331–355.
Hwang, C.-R. and Sheu, S.-J. (1990). Large-time behavior of perturbed diffusion Markov processes with applications to the second eigenvalue problem for Fokker-Planck operators and simulated annealing. Acta Applicandae Mathematicae, 19: 253–295.
Ingber, L. (1989). Very fast simulated re-annealing. Mathematical and Computer Modelling, 12: 967–973.
Ingber, L. (1992). Genetic algorithms and very fast simulated re-annealing: A comparison. Mathematical and Computer Modelling, 16: 87–100.
Ingber, L. (1993). Simulated annealing: Practice versus theory. Mathematical and Computer Modelling, 18: 29–57.
Ingber, L. (1996). Adaptive simulated annealing (ASA): lessons learned. Control and Cybernetics, 25: 33–54.
Jones, A.E.W. and Forbes, G.W. (1995). An adaptive simulated annealing algorithm for global optimization over continuous variables. Journal of Global Optimization, 6: 1–37.
Jones, D.R., Perttunen, C., and Stuckman, B.E. (1993). Lipschitz optimization without the Lipschitz constant. Journal of Optimization Theory and Applications, 78: 157–181.
Jones, D.R., Schonlau, M., and Welch, W.J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13: 455–492.
Kaufman, D.E. and Smith, R.L. (1998). Direction choice for accelerated convergence in Hit-and-Run sampling. Operations Research, 46: 84–95.
Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P. (1983). Optimization by simulated annealing. Science, 220: 671–680.
Kushner, H.J. (1987). Asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects: global minimization via Monte Carlo. SIAM J. Appl. Math., 47: 169–185.
Locatelli, M. (1996). Convergence properties of simulated annealing for continuous global optimization. Journal of Applied Probability, 33: 1127–1140.
Locatelli, M. (2000a). Convergence of a simulated annealing algorithm for continuous global optimization. Journal of Global Optimization, 18: 219–233.
Locatelli, M. (2000b). Simulated annealing algorithms for continuous global optimization: Convergence conditions. Journal of Optimization Theory and Applications, 104: 121–133.
Locatelli, M. (2001). Convergence and first hitting time of simulated annealing algorithms for continuous global optimization. Mathematical Methods of Operations Research, to appear.
Lucidi, S. and Piccioni, M. (1989). Random tunneling by means of acceptance-rejection sampling for global optimization. Journal of Optimization Theory and Applications, 62: 255–277.
Marquez, D. (1997). Convergence rates for annealing diffusion processes. Annals of Applied Probability, 7: 1118–1139.
Metropolis, N., Rosenbluth, A.W., and Teller, A.H. (1953). Equation of state calculations by fast computer machines. J. Chem. Phys., 21: 1087–1092.
Miclo, L. (1992). Recuit simulé sur I[. étude de l’évolution de l’énergie libre. Ann. Inst. Henry Poincaré, 28: 235–266.
Mühlenbein, H., Schomisch, M., and Born, J. (1991). The parallel genetic algorithm as function optimizer. Parallel Computing, 17: 619–632.
Pannetier, J., Bassas-Alsina, J., and Rodriguez-Carvajal, J. (1990). Prediction of crystal structures from crystal chemistry rules by simulated annealing. Nature, 346: 343–345.
Pelletier, M. (1998). Weak convergence rates for stochastic approximation with application to multiple targets and simulated annealing. Annals of Applied Probability, 8: 10–44.
Piccioni, M. (1987). A combined multistart-annealing algorithm for continuous global optimization. Technical Research Report 87–45, Systems and Research Center, The University of Maryland, College Park, MD.
Romeijn, H.E. and Smith, R.L. (1994a). Simulated annealing and adaptive search in global optimization. Probability in the Engineering and Informational Sciences, 8: 571–590.
Romeijn, H.E. and Smith, R.L. (1994b). Simulated annealing for constrained global optimization. Journal of Global Optimization, 5: 101–126.
Romeijn, H.E., Zabinski, Z.B., Graesser, D.L., and Neogi, S. (1999). New reflection generator for simulated annealing in mixed-integer/continuous global optimization. Journal of Optimization Theory and Applications, 101: 403–427.
Royer, G. (1989). A remark on simulated annealing of diffusion processes. SIAM Journal on Control and Optimization, 27: 1403–1408.
Schön, J.C. (1997). Preferential trapping on energy landscapes in regions containing deep-lying minima: the reason for the success of simulated annealing? J. Phys. A: Math. Gen., 30: 2367–2389.
Schuur, P.C. (1997). Classification of acceptance criteria for the simulated annealing algorithm. Mathematics of Operations Research, 22: 266–275.
Semenovskaya, S., Khachaturyan, K.A., and Khachaturyan, A.G. (1985). Statistical mechanics approach to the structure determination of a crystal. Acta Cryst, A41: 268–273.
Siarry, P., Berthiau, G., Durbin, F., and Haussy, J. (1997). Enhanced simulated annealing for globally minimizing functions of many continuous variables. ACM Transactions on Mathematical Software, 23: 209–228.
Sorkin, G.B. (1991). Efficient simulated annealing on fractal energy landscapes. Algorithmica, 6: 367–418.
Szu, H. and Hartley, R. (1987). Fast simulated annealing. Phys. Lett. A, 122: 157–162.
Theodorocatos, V.E. and Grimsley, J.L. (1995). The optimal packing of arbitrarily-shaped polygons using simulated annealing and polynomial-time cooling schedules. Comput. Methods Appl. Mech. Engrg., 125: 53–70.
van Laarhoven, P.J.M. and Aarts, E.H.L. (1987). Simulated annealing: theory and applications. D.Reidel Publishing Co., Dordrecht-Boston, Mass.
Vanderbilt, D. and Louie, S.G. (1984). A monte carlo simulated annealing approach to optimization over continuous variables. Journal of Computational Physics, 56: 259–271.
Wales, D.J. and Doye, J.P.K. (1997). Global optimization by basin-hopping and the lowest energy structures of Lennard-Jones clusters containing up to 110 atoms. Chem. Phys. Lett., 269: 408–412.
Wang, P.P. and Chen, D.-S. (1996). Continuous optimization by a variant of simulated annealing. Computational Optimization and Applications, 6: 59–71.
Wille, L.T. (1987a). Close packing in curved space by simulated annealing. J. Phys. A: Math. Gen., 20: L1211–L1218.
Wille, L.T. (1987b). Minimum-energy configurations of atomic clusters: new results obtained by simulated annealing. Chemical Physics Letters, 133: 405–410.
Wilson, S. and Cui, W. (1990). Applications of simulated annealing to peptides. Biopolymers, 29: 225–235.
Xue, G.L. (1994). Molecular conformation on the cm-5 by parallel two-level simulated annealing. Journal of Global Optimization, 4: 187–208.
Yang, R.L. (2000). Convergence of the simulated annealing algorithm for continuous global optimization. Journal of Optimization Theory and Applications, 104: 691–716.
Yong, L., Lishan, K., and Evans, D.J. (1995). The annealing evolution algorithm as function optimizer. Parallel Computing, 21: 389–400.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Locatelli, M. (2002). Simulated Annealing Algorithms for Continuous Global Optimization. In: Pardalos, P.M., Romeijn, H.E. (eds) Handbook of Global Optimization. Nonconvex Optimization and Its Applications, vol 62. Springer, Boston, MA. https://doi.org/10.1007/978-1-4757-5362-2_6
Download citation
DOI: https://doi.org/10.1007/978-1-4757-5362-2_6
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4419-5221-9
Online ISBN: 978-1-4757-5362-2
eBook Packages: Springer Book Archive