Skip to main content
SpringerLink
Log in
Book cover

Heidelberg Colloquium on Glassy Dynamics pp 287–306Cite as

A pedestrian review of the theory and application of the simulated annealing algorithm

  • II. Optimization
  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Physics ((LNP,volume 275))

Abstract

Simulated annealing is a combinatorial optimization method based on randomization techniques. The method originates from the analogy between the annealing of solids, as described by the theory of statistical physics, and the optimization of large combinatorial problems. Here we review the basic theory of simulated annealing and recite a number of applications of the method. The theoretical review includes concepts of the theory of homogeneous and inhomogeneous Markov chains, an analysis of the asymptotic convergence of the algorithm, and a discussion of the finite-time behaviour. Furthermore, a section is devoted to the relation between statistical physics and the optimization of combinatorial problems. General aspects related to the application of the algorithm are briefly discussed. The list of applications includes combinatorial optimization problems related to VLSI design, image processing, code design and neural networks.

Other names to demote the method are Statistical Cooling 111, Monte Carlo Annealing 1321 and Probabilistic Hill Climbing [52]

This is a preview of subscription content, log in via an institution.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aarts, E.H.L and P.J.M. van Laarhoven, Statistical Cooling: A General Approach to Combinatorial Optimization Problems, Philips J. of Research, 40 (1985)193–226.

    Google Scholar 

  2. Aarts, E.H.L and P.J.M. van Laarhoven, A New Polynomial Time Cooling Schedule, Proc. Int. Conf. on Computer-Aided Design, Santa Clara, November 1985, pp. 206–208.

    Google Scholar 

  3. Aarts, E.H.L, F.M.J. de Bont, J.H.A. Habers and P.J.M. van Laarhoven, A Parallel Statistical Cooling Algorithm, Proc. Symposium on Theoretical Aspects of Computer Science, 1986, Springer Lecture Notes in Computer Science, 210 (1986)87–97.

    MATH  Google Scholar 

  4. Aarts, E.H.L. and P.J.M. van Laarhoven, Quantitative Analysis of the Statistical Cooling Algorithm, to appear in Philips J. of Research, 1986.

    Google Scholar 

  5. Ackley, D.H., G.E. Hinton and T.J. Sejnowski, A Learning Algorithm for Boltzmann Machines, Cognitive Science, 9 (1985)147–169.

    Article  Google Scholar 

  6. Anily, S. and A. Federgruen, Probabilistic Analysis of Simulated Annealing Methods, preprint, 1985.

    Google Scholar 

  7. Aragon C.R., D.S. Johnson, L.A. McGeoch and C. Schevon, Optimization by Simulated Annealing: an Experimental Evaluation, working paper, 1984.

    Google Scholar 

  8. Barker, J.A. and D. Henderson, What is “liquid”? Understanding the states of matter, Reviews of Modern Physics, 48-4 (1976)587–671.

    Article  MathSciNet  ADS  Google Scholar 

  9. Beenker G.F.M., T.A.C.M. Claasen and P.W.C. Hermens, Binary Sequences with Maximally Flat Amplitude Spectrum, Philips J. of Research, 40 (1985)289–304.

    MATH  Google Scholar 

  10. Binder, K., ed., Monte Carlo Methods in Statistical Physics, (Springer, New York, 1978).

    Google Scholar 

  11. Bonomi, E. and J.-L. Lutton, The N-city Travelling Salesman Problem: Statistical Mechanics and the Metropolis Algorithm, SIAM Rev., 26 (1984)551–568.

    Article  MATH  MathSciNet  Google Scholar 

  12. Burkard, R.E. and F. Rendl, A Thermodynamically Motivated Simulation Procedure for Combinatorial Optimization Problems, European J. of Oper. Res., 17 (1984)169–174.

    Article  MATH  Google Scholar 

  13. Carnevalli, P., L. Coletti and S. Paternello, Image Processing by Simulated Annealing, IBM J. Res. Develop., 29 (1985)569–579.

    Article  Google Scholar 

  14. Catthoor, F., H. DeMan and J. Vanderwalle, SAILPLANE: A Simulated Annealing based CAD-tool for the Analysis of Limit-Cycle Behaviour, Proc. IEEE Int. Conf. on Computer Design, Port Chester, October 1985, pp. 244–247.

    Google Scholar 

  15. Černy, V., Thermodynamical Approach to the Traveling Salesman Problem: An Efficient Simulation Algorithm, J. Opt. Theory Appl., 45 (1985)41–51.

    Article  MATH  Google Scholar 

  16. Devada, S. and A.R. Newton, GENIE: A Generalized Array Optimizer for VLSI Synthesis, Proc. 23rd Des. Automation Conf., Las Vegas, June 1986, pp. 631–637.

    Google Scholar 

  17. Feller, W., An Introduction to Probability Theory and Applications, vol. 1, (Wiley, New York, 1950).

    Google Scholar 

  18. Fleisher, H., J. Giraldi, D.B. Martin, R.L. Phoenix and M.A. Tavel, Simulated Annealing as a Tool for Logic Optimization in a CAD Environment, Proc. Int. Conf. on Computer-Aided Design, Santa Clara, November 1985, pp. 203–205.

    Google Scholar 

  19. Garey, M.R. and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, (W.H. Freeman and Co., San Francisco, 1979).

    MATH  Google Scholar 

  20. Geman, S. and D. Geman, Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images, IEEE Proc. Pattern Analysis and Machine Intelligence, PAM-6 (1984)721–741.

    Article  Google Scholar 

  21. Gidas, B., Nonstationary Markov Chains and Convergence of the Annealing Algorithm, J. Statis. Phys., 39 (1985)73–131.

    Article  MATH  MathSciNet  ADS  Google Scholar 

  22. Gidas, B., Global Optimization via the Langevin Equation, Proc. 24th Conf. on Decision and Control, Ft. Lauderdale, December 1985, pp. 774–778.

    Google Scholar 

  23. Grest, G.S., C.M. Soukoulis and K. Levin, Cooling-Rate Dependence for the Spin-Glass Ground-State Energy: Implications for Optimization by Simulated Annealing, Phys. Rev. Letters, 56 (1986)1148–1151, and these proceedings.

    Article  ADS  Google Scholar 

  24. Hajek, B., Cooling Schedules for Optimal Annealing, preprint, 1985.

    Google Scholar 

  25. Hajek, B. A Tutorial Survey of Theory and Applications of Simulated Annealing, Proc. 24th Conf. on Decision and Control, Ft. Lauderdale, December 1985, pp. 755–760.

    Google Scholar 

  26. Hemachandra, L.A. and V.K. Wei, Simulated Annealing and Error Correcting Codes, preprint, 1984.

    Google Scholar 

  27. Hinton, G.E., T.J. Sejnowski and D.H. Ackley, Boltzmann Machines: Constraint Satisfaction Networks that Learn, Department of Computer Science, Carnegie-Mellon University, Technical Report No. CMU-CS-86-119, May 1984.

    Google Scholar 

  28. Holley, R., Rapid Convergence to Equilibrium in One Dimensional Stochastic Ising Models, Annals of Probability, 13 (1985)72–89.

    Article  MATH  MathSciNet  ADS  Google Scholar 

  29. Hopfield, J.J., Neural Networks and Physical Systems with Emergent Collective Computational Abilities, Proc. Nat. Academy of Sciences, USA, 79 (1982)2554–2558.

    Article  MathSciNet  ADS  Google Scholar 

  30. Hopfield, J.J. and D.W. Tank, Neural Computation of Decisions in Optimization Problems, Biological Cybernetics, 52 (1985)141–152.

    MATH  MathSciNet  Google Scholar 

  31. Isaacson, D. and R. Madsen, Markov Chains, (Wiley, New York, 1976).

    MATH  Google Scholar 

  32. Jepsen, D.W. and C.D. Gelatt Jr., Macro Placement by Monte Carlo Annealing, Proc. Int. Conf. on Computer Design, Port Chester, November 1983, pp. 495–498.

    Google Scholar 

  33. Kirkpatrick, S., C.D. Gelatt Jr. and M.P. Vecchi, Optimization by Simulated Annealing, Science, 220 (1983)671–680.

    Article  MathSciNet  ADS  Google Scholar 

  34. Kirkpatrick, S., Optimization by Simulated Annealing: Quantitative Studies, J. Statis. Phys., 34 (1984)975–986.

    Article  MathSciNet  ADS  Google Scholar 

  35. Kirkpatrick, S. and G. Toulouse, Configuration Space Analysis of Travelling Salesman Problems, J. Physique, 46 (1985)1277–1292.

    Article  MathSciNet  Google Scholar 

  36. Laarhoven, P.J.M. van and E.H.L. Aarts, Simulated Annealing: A Review of the Theory and Applications, submitted for publication in Acta Applicandae Mathematicae, 1986.

    Google Scholar 

  37. Lawler, E.L., J.K. Lenstra, A.H.G. Rinnooy Kan and D.B. Shmoys, The Traveling Salesman Problem, (Wiley, Chichester, 1985).

    MATH  Google Scholar 

  38. Leong, H.W., D.F. Wong and C.L. Liu, A Simulated-Annealing Channel Router, Proc. ICCAD, Santa Clara, November 1985, pp. 226–229.

    Google Scholar 

  39. Ligthart, M.M., E.H.L. Aarts and F.P.M. Beenker, A Design-for-testability of PLA's using Statistical Cooling, to appear in Proc. 23rd Des. Automation Conf., Las Vegas, June 1986.

    Google Scholar 

  40. Lundy, M. and A. Mees, Convergence of an Annealing Algorithm, Math. Prog., 34 (1986)111–124.

    Article  MATH  MathSciNet  Google Scholar 

  41. M. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller and E. Teller, Equation of State Calculations by Fast Computing Machines, J. Chem. Phys. 21 (1953)1087–1092.

    Article  ADS  Google Scholar 

  42. Mézard, M., G. Parisi, N. Sourlas, G. Toulouse and M. Virasoro, Nature of the Spin-Glass Phase, Phys. Rev. Letters, 52 (1984)1156–1159.

    Article  ADS  Google Scholar 

  43. Mézard, M. and G. Parisi, Replicas and Optimization, J. Physique Lett., 46 (1985)L–771–L–778.

    Google Scholar 

  44. Mitra, D., Romeo, F. and A.L. Sangiovanni-Vincentelli, Convergence and Finite-Time Behavior of Simulated Annealing, Proc. 24th Conf. on Decision and Control, Ft. Lauderdale, December 1985, pp. 761–767.

    Google Scholar 

  45. Moore, T.P. and A.J. de Geus, Simulated Annealing Controlled by a Rule-Based Expert System, Proc. Int. Conf. on Computer Aided Design, Santa Clara, November 1985, pp. 200–202.

    Google Scholar 

  46. Morgenstern, I., Phase Transitions in Spin Glasses and Chip Design, these proceedings.

    Google Scholar 

  47. Otten, R.H.J.M. and L.P.P.P. van Ginneken, Floorplan Design using Simulated Annealing, Proc. Int. Conf. on Computer-Aided Design, Santa Clara, November 1984, pp. 96–98.

    Google Scholar 

  48. Papadimitriou, C.H. and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity, (Prentice Hall, New York, 1982).

    MATH  Google Scholar 

  49. Parisi, G., Order Parameter for Spin-Glasses, Phys. Rev. Letters, 50 (1983)1946–1948.

    Article  MathSciNet  ADS  Google Scholar 

  50. These Proceedings.

    Google Scholar 

  51. Ripley, B.D., Statistics, Images and Pattern Recognition, preprint, 1985.

    Google Scholar 

  52. Romeo, F. and A.L. Sangiovanni-Vincentelli, Probabilistic Hill Climbing Algorithms: Properties and Applications, Proc. 1985 Chapel Hill Conf. on VLSI, May 1985, pp. 393–417.

    Google Scholar 

  53. Rowen, C. and J.L. Hennessy, Logic Minimization, Placement and Routing in SWAMI, Proc. 1985 Custom IC Conf., Portland, May 1985.

    Google Scholar 

  54. Sechen, C. and A.L. Sangiovanni-Vincentelli, TimberWolf 3.2: A New Standard Cell Placement and Global Routing Package, to appear in Proc. 23rd Des. Automation Conf., Las Vegas, June 1986, pp. 432–439.

    Google Scholar 

  55. Seneta, E., Non-negative Matrices and Markov Chains, (Springer Verlag, New York, 2nd ed., 1981).

    MATH  Google Scholar 

  56. Smith, W.E., H.H. Barrett and R.G. Paxman, Reconstruction of Objects from Coded Images by Simulated Annealing, Optics Letters, 8 (1983)199–201.

    Article  ADS  Google Scholar 

  57. Smith, W.E., R.G. Paxman and H.H. Barrett, Application of Simulated Annealing to Coded-aperture Design and Tomographic Reconstruction, IEEE Trans. Nuclear Science, NS-32 (1985)758–761.

    Article  ADS  Google Scholar 

  58. Solla, S., G. Sorkin and S. White, Configuration Space Analysis for Optimization Problems, in: Disordered Systems and Biological Organization, (Springer Verlag, 1985).

    Google Scholar 

  59. Sontag, E.D. and H.J. Sussmann, Image Restoration and Segmentation using the Annealing Algorithm, Proc. 24th Conf. on Decision and Control, Ft. Lauderdale, December 1985, pp. 768–773.

    Google Scholar 

  60. Toda, M., R. Kubo and N. Saitô, Statistical Physics, (Springer, Berlin, 1983).

    Google Scholar 

  61. Vecchi, M.P. and S. Kirkpatrick, Global wiring by Simulated Annealing, IEEE Trans. on Computer-Aided Design, 2 (1983)215–222.

    Article  Google Scholar 

  62. White, S.R., Concepts of Scale in Simulated Annealing, Proc. ICCD, Port Chester, November 1984, pp. 646–651.

    Google Scholar 

  63. Wolberg, G. and T. Pavlidis, Restoration of Binary Images using Stochastic Relaxation with Annealing, Pattern Recognition Letters, 3 (1985)375–388.

    Article  Google Scholar 

  64. Wong, D.F., H.W. Leong and C.L. Liu, Multiple PLA Folding by the Method of Simulated Annealing, Proc. 1986 Custom IC Conf., Rochester, May 1986, pp. 351–355.

    Google Scholar 

  65. Wong, D.F. and C.L. Liu, A New Algorithm for Floorplan Design, Proc. 23rd Des. Automation Conf., Las Vegas, June 1986, pp. 101–107.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Philips Research Laboratories, P.O. Box 80000, 5600, JA Eindhoven, the Netherlands

    Emile H. L. Aarts & Peter J. M. van Laarhoven

Authors
  1. Emile H. L. Aarts
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter J. M. van Laarhoven
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

J. L. van Hemmen I. Morgenstern

Rights and permissions

Reprints and permissions

Copyright information

© 1987 Springer-Verlag

About this paper

Cite this paper

Aarts, E.H.L., van Laarhoven, P.J.M. (1987). A pedestrian review of the theory and application of the simulated annealing algorithm. In: van Hemmen, J.L., Morgenstern, I. (eds) Heidelberg Colloquium on Glassy Dynamics. Lecture Notes in Physics, vol 275. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0057524

Download citation

  • DOI: https://doi.org/10.1007/BFb0057524

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-17777-7

  • Online ISBN: 978-3-540-47819-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation