Development of Self-Learning Vision-Based Mobile Robots for Acquiring Soccer Robots Behaviors

Abstract

An input generalization problem is one of the most important ones in applying reinforcement learning to real robot tasks. To cope with this problem, we propose a self-partitioning state space algorithm which can make non-uniform quantization of the multidimensional continuous state space. This method recursively splits its continuous state space into some coarse spaces called tentative states based on the relevance test for immediate reward r and discounted future reward Q which are collected during Q-learning process. When it finds out that a tentative state is relevant by the statistical test on a minimum description length (hereafter, MDL), it partitions this coarse space into finer spaces. To show that our algorithm has generalization capability, we apply our method to two tasks in which a soccer robot shoots a ball into a goal and prevent a ball from entering a goal. To show the validity of this method, the experimental results for computer simulation and a real robot are shown.