Advertisement

Design of Adaptive Self-Navigated Airship in Simulated Environment

  • Keiko Motoyama
  • Keiji Suzuki
  • Hidenori Kawamura
  • Masahito Yamamoto
  • Azuma Ohuchi
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 43)

Abstract

The final goal of this research is to realize a small airship robot that can automatically achieve a given task. The airship is subjected to strong inertial forces and air resistance. Although reinforcement learning methods could be expected to control a small airship, the unstable property of the airship prevents the learning methods from achieving control of it.

In order to design an automatically controlled airship, sensory information is especially important. We assume using like ultrasonic transducers which have been widely used as a cheap and light way to provide mobile robots with accurate range finders. This paper verifies the difference in control performance of the airship between a variety of sensory setup. We simulated a small airship with the Cerebellar Model Articulation Controller (CMAC) as a reinforcement learning method which is enabled to deal with generalization problems, on the assumption that we use ultrasonic transducers afterward.

The experimental results showed that the learning performance was not always proportional to the amount of sensory information, and different behavior was acquired according to differences in the sensory setup.

Keywords

Reinforcement Learning Airship Control 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Lin, L. -J. Scaling Up Reinforcement Learning for Robot Control. Proceedings of the 10th International Conference on Machine Learning 1993: 182–189Google Scholar
  2. H. Murao, I. Kitagawa and S. Kitamura. Adaptive State Segmentation for Q-learning (in Japanese). Proceedings of SICE00′97 1997: 45–48Google Scholar
  3. N. Ono and Y. Fukuta. Learning to Coordinate in a Continuous Environment. Proceedings of the Second International Conference on Multi-agent Systems, 1996Google Scholar
  4. Singh, S. P. Reinforcement Learning with Replacing Eligibility Traces. Machine Learning 1996;22,l/2/3: 123–158Google Scholar
  5. Sutton, R. S. Learning to predict by the methods of temporal differences. Machine Learning 1988;3: 9–44Google Scholar
  6. Sutton, R. S. Generalization in reinforcement learning: Successful examples using sparce coarse coding. In D.S.Touretzky, M.C.Mozer and M.E.Hasselmo(eds.), Advances in Neural Information Processing Systems 1996: 1038–1044Google Scholar
  7. Y. Takahashi, M. Asada and K. Hosoda. Reasonable Performance in Less Learning Time by Real Robot Based on Incremental State Space Segmentation. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems 1996: 1038–1044Google Scholar
  8. Ming Tan. Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents. Proceedings of the Tenth International Conference on Machine Learning 1993: 330–337Google Scholar
  9. Wiering, M. and Schmidhuber, J. Speeding up Q(λ)-learning. Proceedings of the Tenth European Conference on Machine Learning 1998: 352–363Google Scholar

Copyright information

© Springer Science+Business Media New York 2002

Authors and Affiliations

  • Keiko Motoyama
    • 1
  • Keiji Suzuki
    • 2
  • Hidenori Kawamura
    • 1
  • Masahito Yamamoto
    • 1
  • Azuma Ohuchi
    • 1
  1. 1.Graduate School of EngeneeringHokkaido UniversitySapporo, HokkaidoJapan
  2. 2.Faculty of System information ScienceHakodate Future UniversityHakodate, HokkaidoJapan

Personalised recommendations