Control Optimization with Learning Automata

Abstract

In this chapter, we will discuss an alternative to Reinforcement Learning for solving Markov decision problems (MDPs) and Semi-Markov decision problems (SMDPs). The methodology that we will discuss in this chapter is generally referred to as Learning Automata. We have already discussed the theory of learning automata in the context of parametric optimization. It turns out that in control optimization too, in particular for solving problems modeled with Markov chains, learning automata methods can be useful.

If a man does not keep pace with his companion, perhaps it is because he hears a different drummer. Let him step to the music which he hears, however measured and far away.

— H.D. Thoreau (1817–1862)