Simulated Annealing: “Smooth” (Local) Discrete Optimization

Abstract

In many real-life problems, we know what will happen if we make a decision, and what we want. Such problems are naturally formalized as optimization problems. In Lessons 5 and 6, we showed that when an optimization problem is continuous in nature, continuous mathematics is useful in designing an algorithm for this problem. In this lesson (and in the follow-up Lesson 8), we will show that continuous mathematics can help in designing algorithms for discrete optimization as well. Namely, in Lesson 7, we will show this on the example of simulated annealing, which is a good method for “almost smooth” discrete objective functions. In Lesson 8, the same idea will be applied to genetic algorithms, which optimize “non-smooth” objective functions.