Thermo-Inspired Machine Learning Algorithm: Simulated Annealing

Abstract

Simulated Annealing (SA) is a global optimization algorithm. It belongs to the stochastic optimization algorithms. The idea of SA was published in a paper by Metropolis [1], this idea was inspired by the annealing process in metallurgy. In this process, a material is heated to a very high temperature so that the particles of the material are freely moving, and then the material is gradually cooled under certain conditions so that the particles begin to take the narrower paths till they collectively reach the minimum energy state level in the material as depicted in Fig. 3.1. In other words, it is a physical phenomenon that occurs when the metal is heated at a very high temperature, and then slowly cooled [2–9]. By analogy with this physical process, each step of the SA algorithm attempts to replace the current solution by a random solution till the desired output is obtained. At each step of the algorithm, SA decides between moving the system to a new state and staying in the current state. This decision is done probabilistically. The analogy between simulated annealing and physical system can be shown in Table 3.1.