Design of Adaptive Self-Navigated Airship in Simulated Environment
The final goal of this research is to realize a small airship robot that can automatically achieve a given task. The airship is subjected to strong inertial forces and air resistance. Although reinforcement learning methods could be expected to control a small airship, the unstable property of the airship prevents the learning methods from achieving control of it.
In order to design an automatically controlled airship, sensory information is especially important. We assume using like ultrasonic transducers which have been widely used as a cheap and light way to provide mobile robots with accurate range finders. This paper verifies the difference in control performance of the airship between a variety of sensory setup. We simulated a small airship with the Cerebellar Model Articulation Controller (CMAC) as a reinforcement learning method which is enabled to deal with generalization problems, on the assumption that we use ultrasonic transducers afterward.
The experimental results showed that the learning performance was not always proportional to the amount of sensory information, and different behavior was acquired according to differences in the sensory setup.
KeywordsReinforcement Learning Airship Control
Unable to display preview. Download preview PDF.
- Lin, L. -J. Scaling Up Reinforcement Learning for Robot Control. Proceedings of the 10th International Conference on Machine Learning 1993: 182–189Google Scholar
- H. Murao, I. Kitagawa and S. Kitamura. Adaptive State Segmentation for Q-learning (in Japanese). Proceedings of SICE00′97 1997: 45–48Google Scholar
- N. Ono and Y. Fukuta. Learning to Coordinate in a Continuous Environment. Proceedings of the Second International Conference on Multi-agent Systems, 1996Google Scholar
- Singh, S. P. Reinforcement Learning with Replacing Eligibility Traces. Machine Learning 1996;22,l/2/3: 123–158Google Scholar
- Sutton, R. S. Learning to predict by the methods of temporal differences. Machine Learning 1988;3: 9–44Google Scholar
- Sutton, R. S. Generalization in reinforcement learning: Successful examples using sparce coarse coding. In D.S.Touretzky, M.C.Mozer and M.E.Hasselmo(eds.), Advances in Neural Information Processing Systems 1996: 1038–1044Google Scholar
- Y. Takahashi, M. Asada and K. Hosoda. Reasonable Performance in Less Learning Time by Real Robot Based on Incremental State Space Segmentation. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems 1996: 1038–1044Google Scholar
- Ming Tan. Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents. Proceedings of the Tenth International Conference on Machine Learning 1993: 330–337Google Scholar
- Wiering, M. and Schmidhuber, J. Speeding up Q(λ)-learning. Proceedings of the Tenth European Conference on Machine Learning 1998: 352–363Google Scholar