Unified Criterion of State Generalization for Reactive Autonomous Agents

Abstract

Autonomous state generalization problem is a key issue in the research field of behavior learning of reactive agents, and many approaches have been proposed in recent years. However, those existing methods have a diversity in their criteria of state generalization or “how to define the similarity or distance between different sensor inputs”, while it is not yet clear how this difference in the criteria would affect the entire learning process. In this paper, we first classify and examine those conventional heuristic criteria of state generalization, and then propose a new general framework for unifying all of them. This novel general criterion is based on minimization of weighted sum of entropies in multiple behavior outcomes of agents. An experimental study in the latter part suggests that this state generalization criterion enables a reactive agent to construct or reconstruct its state space in a more efficient and flexible way.