, Volume 24, Issue 4–5, pp 317–337 | Cite as

Some results characterizing the finite time behaviour of the simulated annealing algorithm

  • Madhav P Desai
Chance As Necessity


TheSimulated Annealing algorithm is a probabilistic search technique for finding the minimum cost state in a set Ω. The algorithm has been successfully used to obtain near-optimal solutions for problems for which no other effective algorithms exist. For example, problems in integrated circuit layout and in finite impulse response (FIR) filter design have been solved using annealing. In most applications, Ω is finite set, and the annealing algorithm may be modelled as a time-inhomogeneous Markov chain on Ω with transition probabilities that are powers of a time varying parameter ε. It has been shown by several researchers that if ε is driven to 0 sufficiently slowly, then the algorithm will eventually find a minimum cost state in Ω with probability 1. In this paper, we will focus on the finite-time behaviour of the annealing algorithm. In particular, we will summarize some results relating the number of steps taken by the algorithm to the quality of the solutions obtained. These results provide qualitative as well as quantitative information about the status of the annealing algorithm after a finite number of steps. This will be illustrated using some examples.


Simulated annealing algorithm finite time behaviour minimum cost state probabilistic search technique 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Apostol T M 1976Introduction to analytic number theory (New York: Springer-Verlag)MATHGoogle Scholar
  2. Catthoor F, DeMan H, Vanderwalle J 1986 Investigation of finite word length effects in arbitrary digital filters using simulated annealing.Proceedings of the IEEE International. Symposium on Circuits and Systems (Piscataway, NJ: IEEE Press) pp 1296–1297Google Scholar
  3. Catoni O 1991a Sharp large deviation estimates for simulated annealing algorithms.Ann. Inst. Henri Poincare 27: 291–383MATHMathSciNetGoogle Scholar
  4. Catoni O 1991b Application of sharp large deviation estimates to optimal cooling schedules.Ann. Inst. Henri Poincare 27: 463–518MATHMathSciNetGoogle Scholar
  5. Catoni O 1992 Rough large deviation estimates for simulated annealing algorithms: application to exponential schedules.Ann. Probab. 20: 1109–1140MATHCrossRefMathSciNetGoogle Scholar
  6. Cerny V 1985 Thermodynamical approach to the travelling salesman problem: An efficient simulation algorithm.J. Optim. Theory Appl. 45: 41–51MATHCrossRefMathSciNetGoogle Scholar
  7. Cheeger J 1970 A lower bound for the smallest eigenvalue of the Laplacian. InProblems in analysis (ed.) R C Gunning (Princeton, NJ: University Press) pp 195–199Google Scholar
  8. Chiang T-S, Chow Y 1984 On eigenvalues and annealing rates.Math. Oper. Res. 9: 508–511MathSciNetGoogle Scholar
  9. Chiang T-S, Chow Y 1989 A limit theorem for a class of inhomogeneous Markov processes.Ann. Probab. 17: 1483–1502MATHCrossRefMathSciNetGoogle Scholar
  10. Connors D P, Kumar P R 1988 Balance of recurrence order in time-inhomogeneous Markov chains with application to simulated annealing.Probab. Eng. Inf. Sci. 2: 157–184MATHCrossRefGoogle Scholar
  11. Desai M P 1991 An eigenvalue-based approach to the finite time behavior of simulated annealing. Ph D thesis, Department of Electrical and Computer Engineering, University of Illinois, Urbana Champaign, ILGoogle Scholar
  12. Desai M P, Rao V B 1993 On the convergence of reversible Markov chains.SIAM J. Matrix Anal. 14: 950–966MATHCrossRefMathSciNetGoogle Scholar
  13. Desai M P, Rao V B 1997 Finite time behaviour of slowly cooled annealing chains.Probab. Eng. Inf. Sci. 11: 137–176MATHMathSciNetGoogle Scholar
  14. Desai M P, Sunil Kumar, Kumar P R 1994 Quasi-statically cooled Markov chains.Probab. Eng. Inf. Sci. 8: 1–19Google Scholar
  15. Diaconis P, Stroock D 1991 Geometric bounds for eigenvalues of Markov chains.Ann. Appl. Probab. 1: 36–61MATHCrossRefMathSciNetGoogle Scholar
  16. Dyer M, Frieze A, Kannan R 1989 A random polynomial algorithm for approximating the volume of a convex body.Proceedings of the 21st Symposium on the Theory of Computation, pp 375–381Google Scholar
  17. Fleisher H, Giraldi J, Martin D B, Phoenix R L, Tavel M A 1985 Simulated annealing as a tool for logic optimization in a CAD environment.Proceedings of the IEEE International Conference on Computer-Aided Design, Santa Clara, pp 203–205Google Scholar
  18. Garey M R, Johnson D S 1979Computers and intractability: A guide to the theory of NP-completeness (New York: W H Freeman)MATHGoogle Scholar
  19. Geman S, Geman D 1984 Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images.IEEE Trans. Pattern Anal. Machine Intell. PAMI-6:Google Scholar
  20. Gidas B 1985 Non-stationary Markov chains and convergence of the annealing algorithm.J. Stat. Phys. 39: 73–131MATHCrossRefMathSciNetGoogle Scholar
  21. Hajek B 1988 Cooling schedules for optimal annealing.Math. Oper. Res. 13: 311–329MATHMathSciNetCrossRefGoogle Scholar
  22. Kirkpatrick S, Gelatt C D Jr, Vecchi M P 1983 Optimization by simulated annealing.Science 220: 671–680CrossRefMathSciNetGoogle Scholar
  23. van Laarhoven P J M, Aarts E H 1987Simulated annealing: Theory and applications (Dordrecht: D Reidel)MATHGoogle Scholar
  24. Lawler G, Sokal A 1988 Bounds on theL 2 spectrum for Markov chains and Markov processes: A generalization of Cheeger’s inequality.Trans. Am. Math. Soc. 309: 557–580MATHCrossRefMathSciNetGoogle Scholar
  25. Mazza C 1991 Asymptotic first hitting time distribution of annealing processes. PreprintGoogle Scholar
  26. Metropolis N, Rosenbluth A, Rosenbluth M, Teller A, Teller E 1953 Equation of state calculations by fast computing machines.J. Chem. Phy. 21: 87–92CrossRefGoogle Scholar
  27. Mitra D, Romeo F, Sangiovanni-Vincentelli A L 1986 Convergence and finite-time behavior of simulated annealing.Adv. Appl. Probab. 18: 747–771MATHCrossRefMathSciNetGoogle Scholar
  28. Mohar B 1988 The Laplacian spectrum of graphs.International Graph Theory Conference, Western Michigan UniversityGoogle Scholar
  29. Otten R H J M, Van Ginneken L P P P 1984 Floorplan design using simulated annealing.Proceedings of the International Conference on Computer-Aided Design (Los Alamitos, CA: IEEE Comput. Press) pp 96–98Google Scholar
  30. Papadimitrou C H, Stieglitz K 1981Combinatorial optimization: Algorithms and complexity (Englewood Cliffs, NJ: Prentice-Hall)Google Scholar
  31. Sasaki G, Hajek B 1988 The time-complexity of maximum matching by simulated annealing.J. Assoc. Comput. Mach. 35: 387–403MathSciNetGoogle Scholar
  32. Sechen C T, Sangovianni-Vincentelli A L 1984 The Timber-Wolf placement and routing package.Proceedings of the 1984 Custom Integrated Circuit Conference, RochesterGoogle Scholar
  33. Seneta E 1980Nonnegative matrices and Markov chains. (New York: Springer Verlag)Google Scholar
  34. Sinclair A, Jerrum M 1989 Approximate counting, uniform generation and rapidly mixing Markov chains.Inf. Comput. 82: 93–133MATHCrossRefMathSciNetGoogle Scholar
  35. Tsitsiklis J N 1989 Markov chains with rare transitions and simulated annealing.Math. Oper. Res. 14: 70–90MATHMathSciNetGoogle Scholar
  36. Vecchi M P, Kirkpatrick S 1983 Global wiring by simulated annealing.IEEE Trans. Comput.-Aided Design CAD-2: 215–222CrossRefGoogle Scholar
  37. Ventcel A D 1972 On the asymptotics of eigenvalues of matrices with elements of order exp(−Vij/2ε 2).Dokl. Akad. Nauk SSSR 202: 65–68Google Scholar
  38. White S 1984 Concepts of scale in simulated annealing.Proceedings of the IEEE International Conference on Computer Design (Los Alamitos, CA: IEEE Comput. Press) pp 646–651Google Scholar
  39. Wolberg G, Pavlidis T 1985 Restoration of binary images using stochastic relaxation with annealing.Pattern Recogn. Lett. 3: 375–388CrossRefGoogle Scholar
  40. Wong D F, Liu C L 1986 A new algorithm for floorplan design.Proceedings of the 23rd Design Automation Conference (New York: ACM Press) pp 1–7Google Scholar
  41. Yao X, Liu C L 1990 PLA logic minimization by simulated annealing.Integration, VLSI J. 9: 243–257CrossRefGoogle Scholar

Copyright information

© the Indian Academy of Sciences 1999

Authors and Affiliations

  1. 1.Department of Electrical EngineeringIndian Institute of TechnologyMumbaiIndia

Personalised recommendations