A Modular Reinforcement Learning Architecture for Mobile Robot Control

  • R. M. Rylatt
  • C. A. Czarnecki
  • T. W. Routen
Conference paper


The paper presents a way of extending complementary reinforcement backpropagation learning (CRBP) to modular architectures using a new version of the gating network approach in the context of reactive navigation tasks for a simulated mobile robot. The gating network has partially recurrent connections to enable the co-ordination of reinforcement learning across both modules successive time steps. The experiments reported explore the possibility that architectures based on this approach can support concurrent acquisition of different reactive navigation related competences while the robot pursues light-seeking goals.


Mobile Robot Autonomous Agent Recurrent Connection Modular Neural Network Credit Assignment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    D. H. Ackley and M. L. Littman. Generalisation and scaling in reinforcement learning. In D. S. Touretsky, editor, Advances in Neural Information Processing Systems, pages 550–557. Morgan Kaufmann, San Mateo, CA, 1990.Google Scholar
  2. [2]
    R. A. Brooks. A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, RA-2:14–23, 1986.MathSciNetCrossRefGoogle Scholar
  3. [3]
    R. A. Brooks. Intelligence without representation. Artificial Intelligence, 47:131–159, 1991.CrossRefGoogle Scholar
  4. [4]
    J. Elman. Finding structure in time. Cognitive Science, 14:179–192, 1990.CrossRefGoogle Scholar
  5. [5]
    R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton. Adaptive mixtures of local experts. Neural Computation, 3:337–345, 1991.CrossRefGoogle Scholar
  6. [6]
    L. Meeden, G. McGraw, and D. Blank. Emergent control and planning in an autonomous vehicle. In Proceedings of the Fifteenth Annual Conference of the Cognitive Science Society, 1994.Google Scholar
  7. [7]
    R. Molland, T. Scutt, and P. Green. Extending low-level reactive behaviours using primitive behavioural memory. In Proceedings of the International Conference on Recent Advances in Mechatronics, pages 510–516, 1995.Google Scholar
  8. [8]
    R. M. Rylatt, C. A. Czaxnecki, and T. W. Routen. Connectionist learning in behaviour-based mobile robots: A survey. In Artificial Intelligence Review. Kluwer Academic Publishers. (to appear).Google Scholar
  9. [9]
    R. M. Rylatt, C. A. Czarnecki, and T. W. Routen. A perspective on the future of behaviour-based robotics. In Mobile Robotics Workshop Notes — Tenth Biennial Conference on Artificial Intelligence and Simulated Behaviour, 1995.Google Scholar
  10. [10]
    R. M. Rylatt, C. A. Czarnecki, and T. W. Routen. Learning behaviours in a modular neural net architecture for a mobile autonomous agent. In Proceedings of the First Euromicro Workshop on Advanced Mobile Robots, pages 82–86, 1996.Google Scholar
  11. [11]
    G. M. Saunders, J. F. Kolen, and J. B. Pollack. The importance of leaky levels for behaviour based A.I. In From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behaviour, pages 275–281. MIT Press, 1994.Google Scholar
  12. [12]
    R. J. Williams. On the use of backpropagation in associative reinforcement learning. In Proceedings of the IEEE International Conference on Neural Networks, pages 263–270, 1988.Google Scholar
  13. [13]
    T. Ziemke. Towards adaptive perception in autonomous robots using second-order recurrent networks. In Proceedings of the First Euromicro Workshop on Advanced Mobile Robots, pages 89–98, 1996.Google Scholar

Copyright information

© Springer-Verlag Wien 1998

Authors and Affiliations

  • R. M. Rylatt
    • 1
  • C. A. Czarnecki
    • 1
  • T. W. Routen
    • 1
  1. 1.Department of Computer ScienceDe Montfort UniversityLeicesterUK

Personalised recommendations