Abstract
TheSimulated Annealing algorithm is a probabilistic search technique for finding the minimum cost state in a set Ω. The algorithm has been successfully used to obtain near-optimal solutions for problems for which no other effective algorithms exist. For example, problems in integrated circuit layout and in finite impulse response (FIR) filter design have been solved using annealing. In most applications, Ω is finite set, and the annealing algorithm may be modelled as a time-inhomogeneous Markov chain on Ω with transition probabilities that are powers of a time varying parameter ε. It has been shown by several researchers that if ε is driven to 0 sufficiently slowly, then the algorithm will eventually find a minimum cost state in Ω with probability 1. In this paper, we will focus on the finite-time behaviour of the annealing algorithm. In particular, we will summarize some results relating the number of steps taken by the algorithm to the quality of the solutions obtained. These results provide qualitative as well as quantitative information about the status of the annealing algorithm after a finite number of steps. This will be illustrated using some examples.
Similar content being viewed by others
References
Apostol T M 1976Introduction to analytic number theory (New York: Springer-Verlag)
Catthoor F, DeMan H, Vanderwalle J 1986 Investigation of finite word length effects in arbitrary digital filters using simulated annealing.Proceedings of the IEEE International. Symposium on Circuits and Systems (Piscataway, NJ: IEEE Press) pp 1296–1297
Catoni O 1991a Sharp large deviation estimates for simulated annealing algorithms.Ann. Inst. Henri Poincare 27: 291–383
Catoni O 1991b Application of sharp large deviation estimates to optimal cooling schedules.Ann. Inst. Henri Poincare 27: 463–518
Catoni O 1992 Rough large deviation estimates for simulated annealing algorithms: application to exponential schedules.Ann. Probab. 20: 1109–1140
Cerny V 1985 Thermodynamical approach to the travelling salesman problem: An efficient simulation algorithm.J. Optim. Theory Appl. 45: 41–51
Cheeger J 1970 A lower bound for the smallest eigenvalue of the Laplacian. InProblems in analysis (ed.) R C Gunning (Princeton, NJ: University Press) pp 195–199
Chiang T-S, Chow Y 1984 On eigenvalues and annealing rates.Math. Oper. Res. 9: 508–511
Chiang T-S, Chow Y 1989 A limit theorem for a class of inhomogeneous Markov processes.Ann. Probab. 17: 1483–1502
Connors D P, Kumar P R 1988 Balance of recurrence order in time-inhomogeneous Markov chains with application to simulated annealing.Probab. Eng. Inf. Sci. 2: 157–184
Desai M P 1991 An eigenvalue-based approach to the finite time behavior of simulated annealing. Ph D thesis, Department of Electrical and Computer Engineering, University of Illinois, Urbana Champaign, IL
Desai M P, Rao V B 1993 On the convergence of reversible Markov chains.SIAM J. Matrix Anal. 14: 950–966
Desai M P, Rao V B 1997 Finite time behaviour of slowly cooled annealing chains.Probab. Eng. Inf. Sci. 11: 137–176
Desai M P, Sunil Kumar, Kumar P R 1994 Quasi-statically cooled Markov chains.Probab. Eng. Inf. Sci. 8: 1–19
Diaconis P, Stroock D 1991 Geometric bounds for eigenvalues of Markov chains.Ann. Appl. Probab. 1: 36–61
Dyer M, Frieze A, Kannan R 1989 A random polynomial algorithm for approximating the volume of a convex body.Proceedings of the 21st Symposium on the Theory of Computation, pp 375–381
Fleisher H, Giraldi J, Martin D B, Phoenix R L, Tavel M A 1985 Simulated annealing as a tool for logic optimization in a CAD environment.Proceedings of the IEEE International Conference on Computer-Aided Design, Santa Clara, pp 203–205
Garey M R, Johnson D S 1979Computers and intractability: A guide to the theory of NP-completeness (New York: W H Freeman)
Geman S, Geman D 1984 Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images.IEEE Trans. Pattern Anal. Machine Intell. PAMI-6:
Gidas B 1985 Non-stationary Markov chains and convergence of the annealing algorithm.J. Stat. Phys. 39: 73–131
Hajek B 1988 Cooling schedules for optimal annealing.Math. Oper. Res. 13: 311–329
Kirkpatrick S, Gelatt C D Jr, Vecchi M P 1983 Optimization by simulated annealing.Science 220: 671–680
van Laarhoven P J M, Aarts E H 1987Simulated annealing: Theory and applications (Dordrecht: D Reidel)
Lawler G, Sokal A 1988 Bounds on theL 2 spectrum for Markov chains and Markov processes: A generalization of Cheeger’s inequality.Trans. Am. Math. Soc. 309: 557–580
Mazza C 1991 Asymptotic first hitting time distribution of annealing processes. Preprint
Metropolis N, Rosenbluth A, Rosenbluth M, Teller A, Teller E 1953 Equation of state calculations by fast computing machines.J. Chem. Phy. 21: 87–92
Mitra D, Romeo F, Sangiovanni-Vincentelli A L 1986 Convergence and finite-time behavior of simulated annealing.Adv. Appl. Probab. 18: 747–771
Mohar B 1988 The Laplacian spectrum of graphs.International Graph Theory Conference, Western Michigan University
Otten R H J M, Van Ginneken L P P P 1984 Floorplan design using simulated annealing.Proceedings of the International Conference on Computer-Aided Design (Los Alamitos, CA: IEEE Comput. Press) pp 96–98
Papadimitrou C H, Stieglitz K 1981Combinatorial optimization: Algorithms and complexity (Englewood Cliffs, NJ: Prentice-Hall)
Sasaki G, Hajek B 1988 The time-complexity of maximum matching by simulated annealing.J. Assoc. Comput. Mach. 35: 387–403
Sechen C T, Sangovianni-Vincentelli A L 1984 The Timber-Wolf placement and routing package.Proceedings of the 1984 Custom Integrated Circuit Conference, Rochester
Seneta E 1980Nonnegative matrices and Markov chains. (New York: Springer Verlag)
Sinclair A, Jerrum M 1989 Approximate counting, uniform generation and rapidly mixing Markov chains.Inf. Comput. 82: 93–133
Tsitsiklis J N 1989 Markov chains with rare transitions and simulated annealing.Math. Oper. Res. 14: 70–90
Vecchi M P, Kirkpatrick S 1983 Global wiring by simulated annealing.IEEE Trans. Comput.-Aided Design CAD-2: 215–222
Ventcel A D 1972 On the asymptotics of eigenvalues of matrices with elements of order exp(−V ij /2ε 2).Dokl. Akad. Nauk SSSR 202: 65–68
White S 1984 Concepts of scale in simulated annealing.Proceedings of the IEEE International Conference on Computer Design (Los Alamitos, CA: IEEE Comput. Press) pp 646–651
Wolberg G, Pavlidis T 1985 Restoration of binary images using stochastic relaxation with annealing.Pattern Recogn. Lett. 3: 375–388
Wong D F, Liu C L 1986 A new algorithm for floorplan design.Proceedings of the 23rd Design Automation Conference (New York: ACM Press) pp 1–7
Yao X, Liu C L 1990 PLA logic minimization by simulated annealing.Integration, VLSI J. 9: 243–257
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Desai, M.P. Some results characterizing the finite time behaviour of the simulated annealing algorithm. Sadhana 24, 317–337 (1999). https://doi.org/10.1007/BF02823146
Issue Date:
DOI: https://doi.org/10.1007/BF02823146