# Some results characterizing the finite time behaviour of the simulated annealing algorithm

- 69 Downloads
- 4 Citations

## Abstract

The*Simulated Annealing* algorithm is a probabilistic search technique for finding the minimum cost state in a set Ω. The algorithm has been successfully used to obtain near-optimal solutions for problems for which no other effective algorithms exist. For example, problems in integrated circuit layout and in finite impulse response (FIR) filter design have been solved using annealing. In most applications, Ω is finite set, and the annealing algorithm may be modelled as a time-inhomogeneous Markov chain on Ω with transition probabilities that are powers of a time varying parameter ε. It has been shown by several researchers that if ε is driven to 0 sufficiently slowly, then the algorithm will eventually find a minimum cost state in Ω with probability 1. In this paper, we will focus on the finite-time behaviour of the annealing algorithm. In particular, we will summarize some results relating the number of steps taken by the algorithm to the quality of the solutions obtained. These results provide qualitative as well as quantitative information about the status of the annealing algorithm after a finite number of steps. This will be illustrated using some examples.

## Keywords

Simulated annealing algorithm finite time behaviour minimum cost state probabilistic search technique## Preview

Unable to display preview. Download preview PDF.

## References

- Apostol T M 1976
*Introduction to analytic number theory*(New York: Springer-Verlag)MATHGoogle Scholar - Catthoor F, DeMan H, Vanderwalle J 1986 Investigation of finite word length effects in arbitrary digital filters using simulated annealing.
*Proceedings of the IEEE International. Symposium on Circuits and Systems*(Piscataway, NJ: IEEE Press) pp 1296–1297Google Scholar - Catoni O 1991a Sharp large deviation estimates for simulated annealing algorithms.
*Ann. Inst. Henri Poincare*27: 291–383MATHMathSciNetGoogle Scholar - Catoni O 1991b Application of sharp large deviation estimates to optimal cooling schedules.
*Ann. Inst. Henri Poincare*27: 463–518MATHMathSciNetGoogle Scholar - Catoni O 1992 Rough large deviation estimates for simulated annealing algorithms: application to exponential schedules.
*Ann. Probab.*20: 1109–1140MATHCrossRefMathSciNetGoogle Scholar - Cerny V 1985 Thermodynamical approach to the travelling salesman problem: An efficient simulation algorithm.
*J. Optim. Theory Appl.*45: 41–51MATHCrossRefMathSciNetGoogle Scholar - Cheeger J 1970 A lower bound for the smallest eigenvalue of the Laplacian. In
*Problems in analysis*(ed.) R C Gunning (Princeton, NJ: University Press) pp 195–199Google Scholar - Chiang T-S, Chow Y 1984 On eigenvalues and annealing rates.
*Math. Oper. Res.*9: 508–511MathSciNetGoogle Scholar - Chiang T-S, Chow Y 1989 A limit theorem for a class of inhomogeneous Markov processes.
*Ann. Probab.*17: 1483–1502MATHCrossRefMathSciNetGoogle Scholar - Connors D P, Kumar P R 1988 Balance of recurrence order in time-inhomogeneous Markov chains with application to simulated annealing.
*Probab. Eng. Inf. Sci.*2: 157–184MATHCrossRefGoogle Scholar - Desai M P 1991 An eigenvalue-based approach to the finite time behavior of simulated annealing. Ph D thesis, Department of Electrical and Computer Engineering, University of Illinois, Urbana Champaign, ILGoogle Scholar
- Desai M P, Rao V B 1993 On the convergence of reversible Markov chains.
*SIAM J. Matrix Anal.*14: 950–966MATHCrossRefMathSciNetGoogle Scholar - Desai M P, Rao V B 1997 Finite time behaviour of slowly cooled annealing chains.
*Probab. Eng. Inf. Sci.*11: 137–176MATHMathSciNetGoogle Scholar - Desai M P, Sunil Kumar, Kumar P R 1994 Quasi-statically cooled Markov chains.
*Probab. Eng. Inf. Sci.*8: 1–19Google Scholar - Diaconis P, Stroock D 1991 Geometric bounds for eigenvalues of Markov chains.
*Ann. Appl. Probab.*1: 36–61MATHCrossRefMathSciNetGoogle Scholar - Dyer M, Frieze A, Kannan R 1989 A random polynomial algorithm for approximating the volume of a convex body.
*Proceedings of the 21st Symposium on the Theory of Computation*, pp 375–381Google Scholar - Fleisher H, Giraldi J, Martin D B, Phoenix R L, Tavel M A 1985 Simulated annealing as a tool for logic optimization in a CAD environment.
*Proceedings of the IEEE International Conference on Computer-Aided Design*, Santa Clara, pp 203–205Google Scholar - Garey M R, Johnson D S 1979
*Computers and intractability: A guide to the theory of NP-completeness*(New York: W H Freeman)MATHGoogle Scholar - Geman S, Geman D 1984 Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images.
*IEEE Trans. Pattern Anal. Machine Intell.*PAMI-6:Google Scholar - Gidas B 1985 Non-stationary Markov chains and convergence of the annealing algorithm.
*J. Stat. Phys.*39: 73–131MATHCrossRefMathSciNetGoogle Scholar - Hajek B 1988 Cooling schedules for optimal annealing.
*Math. Oper. Res.*13: 311–329MATHMathSciNetCrossRefGoogle Scholar - Kirkpatrick S, Gelatt C D Jr, Vecchi M P 1983 Optimization by simulated annealing.
*Science*220: 671–680CrossRefMathSciNetGoogle Scholar - van Laarhoven P J M, Aarts E H 1987
*Simulated annealing: Theory and applications*(Dordrecht: D Reidel)MATHGoogle Scholar - Lawler G, Sokal A 1988 Bounds on the
*L*^{2}spectrum for Markov chains and Markov processes: A generalization of Cheeger’s inequality.*Trans. Am. Math. Soc.*309: 557–580MATHCrossRefMathSciNetGoogle Scholar - Mazza C 1991 Asymptotic first hitting time distribution of annealing processes. PreprintGoogle Scholar
- Metropolis N, Rosenbluth A, Rosenbluth M, Teller A, Teller E 1953 Equation of state calculations by fast computing machines.
*J. Chem. Phy.*21: 87–92CrossRefGoogle Scholar - Mitra D, Romeo F, Sangiovanni-Vincentelli A L 1986 Convergence and finite-time behavior of simulated annealing.
*Adv. Appl. Probab.*18: 747–771MATHCrossRefMathSciNetGoogle Scholar - Mohar B 1988 The Laplacian spectrum of graphs.
*International Graph Theory Conference*, Western Michigan UniversityGoogle Scholar - Otten R H J M, Van Ginneken L P P P 1984 Floorplan design using simulated annealing.
*Proceedings of the International Conference on Computer-Aided Design*(Los Alamitos, CA: IEEE Comput. Press) pp 96–98Google Scholar - Papadimitrou C H, Stieglitz K 1981
*Combinatorial optimization: Algorithms and complexity*(Englewood Cliffs, NJ: Prentice-Hall)Google Scholar - Sasaki G, Hajek B 1988 The time-complexity of maximum matching by simulated annealing.
*J. Assoc. Comput. Mach.*35: 387–403MathSciNetGoogle Scholar - Sechen C T, Sangovianni-Vincentelli A L 1984 The Timber-Wolf placement and routing package.
*Proceedings of the 1984 Custom Integrated Circuit Conference*, RochesterGoogle Scholar - Seneta E 1980
*Nonnegative matrices and Markov chains*. (New York: Springer Verlag)Google Scholar - Sinclair A, Jerrum M 1989 Approximate counting, uniform generation and rapidly mixing Markov chains.
*Inf. Comput.*82: 93–133MATHCrossRefMathSciNetGoogle Scholar - Tsitsiklis J N 1989 Markov chains with rare transitions and simulated annealing.
*Math. Oper. Res.*14: 70–90MATHMathSciNetGoogle Scholar - Vecchi M P, Kirkpatrick S 1983 Global wiring by simulated annealing.
*IEEE Trans. Comput.-Aided Design*CAD-2: 215–222CrossRefGoogle Scholar - Ventcel A D 1972 On the asymptotics of eigenvalues of matrices with elements of order exp(−V
_{ij}/2_{ε}^{2}).*Dokl. Akad. Nauk SSSR*202: 65–68Google Scholar - White S 1984 Concepts of scale in simulated annealing.
*Proceedings of the IEEE International Conference on Computer Design*(Los Alamitos, CA: IEEE Comput. Press) pp 646–651Google Scholar - Wolberg G, Pavlidis T 1985 Restoration of binary images using stochastic relaxation with annealing.
*Pattern Recogn. Lett.*3: 375–388CrossRefGoogle Scholar - Wong D F, Liu C L 1986 A new algorithm for floorplan design.
*Proceedings of the 23rd Design Automation Conference*(New York: ACM Press) pp 1–7Google Scholar - Yao X, Liu C L 1990 PLA logic minimization by simulated annealing.
*Integration, VLSI J.*9: 243–257CrossRefGoogle Scholar