Chains

Abstract

The inner loop of an annealing algorithm is a markov process, and since the value of t is not changed within this loop the process is also homogeneous. A markov process is called a markov chain when a sequence of events rather than a process defined in continuous time is described. Finally, the fact that the number of states is finite makes the loop a finite homogeneous markov chain. Matrices provide a convenient formalism for discussing these chains, and we therefore describe the inner loop of the annealing algorithm as a sequence of matrix multiplications. Many results are derived using the matrix formulation of the chain, but the central result of this chapter is that under certain mild conditions the frequencies of the states in the loop will tend to a stationary density function, called the equilibrium density of the chain. The equilibrium density depends on t.