Abstract
Simulated annealing (SA) is a generic optimization method whose popularity stems from its simplicity and its global convergence properties; it emulates the physical process of annealing whereby a solid is heated and then cooled down to eventually reach a minimum energy configuration. Although successfully applied to many difficult problems, SA is widely reported to converge very slowly, and it is common practice to relax some of its convergence conditions as well as to allow extra freedom in its design. However, variations on the theme of annealing usually come without optimal convergence guarantees.
In this paper, we review the fundamentals of SA and we focus on acceleration techniques that come with a rigorous mathematical justification. We discuss the design of the candidate-solution generation mechanism, the issue of finite-time cooling, and the technique of acceleration by concave distortion of the objective function. We also investigate a recently introduced generalization of SA—stochastic continuation—which significantly increases the design flexibility by allowing the candidate-solution generation mechanism and the objective function to vary with temperature.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Discussion on the meeting on the Gibbs sampler and other Markov chain Monte Carlo methods. J. Roy. Statist. Soc. Ser. B 55(1), 53–102 (1993)
Azencott, R.: A common large deviations mathematical framework for sequential annealing and parallel annealing. In: Azencott, R. (ed.) Simulated Annealing: Parallelization Techniques, pp. 11–23. Wiley, New York (1992)
Azencott, R.: Sequential simulated annealing: speed of convergence and acceleration techniques. In: Azencott, R. (ed.) Simulated Annealing: Parallelization Techniques, pp. 1–10. Wiley, New York (1992)
Blake, A., Zisserman, A.: Visual reconstruction. The MIT Press (1987)
Catoni, O.: Large deviations and cooling schedules for the simulated annealing algorithm. C. R. Acad. Sci. Paris Sér. I Math. 307, 535–538 (1988) (in French)
Catoni, O.: Rough large deviation estimates for simulated annealing: application to exponential schedules. Ann. Probab. 20(3), 1109–1146 (1992)
Catoni, O.: Metropolis, simulated annealing, and iterated energy transformation algorithms: theory and experiments. J. Complexity 12(4), 595–623 (1996)
Catoni, O.: Simulated annealing algorithms and Markov chains with rare transitions. In: Séminaire de Probabilités XXXIII. Lecture Notes in Math., vol. 1709, pp. 69–119. Springer, New York (1999)
Chiang, T.S., Chow, Y.: On the convergence rate of annealing processes. SIAM J. Control Optim. 26(6), 1455–1470 (1988)
Cohn, H., Fielding, M.: Simulated annealing: searching for an optimal temperature schedule. SIAM J. Optim. 9(3), 779–802 (1999)
Del Moral, P., Miclo, L.: On the convergence and applications of generalized simulated annealing. SIAM J. Control Optim. 37(4), 1222–1250 (1999)
Desai, M.: Some results characterizing the finite time behaviour of the simulated annealing algorithm. Sādhanā 24(4-5), 317–337 (1999)
Fielding, M.: Simulated annealing with an optimal fixed temperature. SIAM J. Optim. 11(2), 289–307 (2000)
Frigerio, A., Grillo, G.: Simulated annealing with time-dependent energy function. Math. Z. 213, 97–116 (1993)
Gidas, B.: Nonstationary Markov chains and convergence of the annealing algorithm. J. Statist. Phys. 39(1/2), 73–131 (1985)
Hajek, B.: Cooling schedules for optimal annealing. Math. Oper. Res. 13(2), 311–329 (1988)
Johnson, A., Jacobson, S.: On the convergence of generalized hill climbing algorithms. Discrete Appl. Math. 119(1-2), 37–57 (2002)
van Laarhoven, P.J.M., Aarts, E.H.L.: Simulated annealing: theory and practice. D. Reidel Publishing Company (1987)
Löwe, M.: Simulated annealing with time-dependent energy function via Sobolev inequalities. Stochastic Process. Appl. 63(2), 221–233 (1996)
Nielsen, M.: Graduated nonconvexity by functional focusing. IEEE Trans. Pattern Anal. Machine Intell. 19(5), 521–525 (1997)
Nikolova, M.: Markovian reconstruction using a GNC approach. IEEE Trans. Image Process. 8(9), 1204–1220 (1999)
Orosz, J., Jacobson, S.: Analysis of static simulated annealing algorithms. J. Optim. Theory Appl. 115(1), 165–182 (2002)
Orosz, J., Jacobson, S.: Finite-time performance analysis of static simulated annealing algorithms. Comput. Optim. Appl. 21(1), 21–53 (2002)
Robini, M.C., Lachal, A., Magnin, I.E.: A stochastic continuation approach to piecewise constant reconstruction. IEEE Trans. Image Process. 16(10), 2576–2589 (2007)
Robini, M.C., Magnin, I.E.: 3-D reconstruction from a few radiographs using the Metropolis dynamics with annealing. In: Proc. IEEE Int. Conf. Image Processing, Kobe, Japan, vol. 3, pp. 876–880 (1999)
Robini, M.C., Magnin, I.E.: Stochastic nonlinear image restoration using the wavelet transform. IEEE Trans. Image Process. 12(8), 890–905 (2003)
Robini, M.C., Magnin, I.E.: Optimization by stochastic continuation. SIAM J. Imaging Sci. 3(4), 1096–1121 (2010)
Robini, M.C., Rastello, T., Magnin, I.E.: Simulated annealing, acceleration techniques and image restoration. IEEE Trans. Image Process. 8(10), 1374–1387 (1999)
Robini, M.C., Reissman, P.J.: On simulated annealing with temperature-dependent energy and temperature-dependent communication. Statist. Probab. Lett. 81(8), 915–920 (2011)
Robini, M.C., Reissman, P.J.: From simulated annealing to stochastic continuation: a new trend in combinatorial optimization. J. Global Optim. (to appear, 2012)
Robini, M.C., Smekens, F., Sixou, B.: Optimal inverse treatment planning by stochastic continuation. In: Proc. 8th IEEE Int. Symp. Biomedical Imaging, Chicago, IL, pp. 1792–1796 (2011)
Schuur, P.: Classification of acceptance criteria for the simulated annealing algorithm. Math. Oper. Res. 22(2), 266–275 (1997)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Robini, M.C. (2013). Theoretically Grounded Acceleration Techniques for Simulated Annealing. In: Zelinka, I., Snášel, V., Abraham, A. (eds) Handbook of Optimization. Intelligent Systems Reference Library, vol 38. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30504-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-30504-7_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30503-0
Online ISBN: 978-3-642-30504-7
eBook Packages: EngineeringEngineering (R0)