On-Line Evolutionary Computation

Abstract

Sutton and Barto’s criticism of evolutionary methods rests on the fact that such methods do not exploit the specific structure of the reinforcement learning problem. Instead, they just treat it like any other optimization problem, using total reward accrued as a fitness function. Much of this book focuses on eliminating this shortcoming by customizing such techniques to the unique characteristics of the reinforcement learning problem. As a result, the representation-learning power of methods like NEAT can be harnessed without sacrificing the advantages of other reinforcement learning approaches, such as temporal difference methods. The heart of this customization is presented in Chapter 4, which describes how to synthesize evolutionary and temporal difference methods so as to evolve representations for value functions.