Designing Neural Control Architectures for an Autonomous Robot Using Vision to Solve Complex Learning Tasks

  • A. Revel
  • P. Gaussier
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 109)


In this chapter, we intend to present a way to design Neural Network architectures to control an autonomous mobile robot using vision as the main sensor. We start discussing the notion of autonomy. In particular we show how it constrains the learning and the architecture of the control system. We propose a set of neural tools developed to solve problems linked with autonomous learning: the Perception-Action (PerAc) architecture, the Probabilistic Conditioning Rule (PCR) and a system allowing to plan actions (integrating a system for transition learning and prediction). Illustrations on different examples (visual homing, maze problem and planning) of how the tools that has been elaborated can be assembled to form a generic control system reusable for several tasks are presented along the description of the tools. Finally, future developments and the way to integrate these works in a general cognitive science framework will be discussed.


Path Integration Synaptic Weight Autonomous Robot Reinforcement Signal Motivational Neuron 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    R. Chatila, Deliberation and Reactivity in Autonomous Mobile Robots, Robotics and Autonomous System, 1995, December, 16, 2–4, 197–211Google Scholar
  2. 2.
    R.A. Brooks, A Robust Layered Control System for a Mobile Robot, IEEE Journal of Robotics and Automation, 1981, 40, 201–211Google Scholar
  3. 3.
    R. Pfeifer and P. Verschure, The Artificial Life Route to Artificial Intelligence, The challenge of autonomous systems: Pitfalls and how to avoid them, 1994, L. Steels and R. Brooks, MIT Press, Cambridge, MAGoogle Scholar
  4. 4.
    R. Pfeifer and C. Scheier, Sensory-motor coordination: the metaphor and beyond, Robotics and Autonomous Systems, R.Pfeifer and R.A. Brooks, 1996Google Scholar
  5. 5.
    D.C. Dennett, Consciousness Explained, 1991, Little, Brown, Boston, MassachusettsGoogle Scholar
  6. 6.
    David McFarland, Animal Robotics - From Self-sufficiency to Autonomy, From Perception to Action, IEEE, 1994, September, P. Gaussier and J.D. Nicoud, IEEE Computer Society Press, Lausanne, SwitzerlandGoogle Scholar
  7. 7.
    Luc Steels, When are robots intelligent autonomous agents?, Robotics and Autonomous Systems, 1995, 15, 3–9CrossRefGoogle Scholar
  8. 8.
    G.A. Carpenter and S. Grossberg, Invariant Pattern Recognition and Recall by an Attentive Self-Organizing ART Architecture in a Nonstationary World, Proceeding of Neural Network, 1987, 2, 737–745Google Scholar
  9. 9.
    P. Gaussier and S. Zrehen, Avoiding the World Model Trap: An Acting Robot Does Not Need to Be So Smart!, Robotics and Computer-Integrated Manufacturing, 1995, 11, 4, 279–286Google Scholar
  10. 10.
    V. Braitenberg, Vehicles: Experiments in Synthetic Psychology, 1984, MIT Press, Bradford Books, CambridgeGoogle Scholar
  11. 11.
    F. Varela and E. Thompson and E. Rosch, The Embodied Mind, 1993, MIT PressGoogle Scholar
  12. 12.
    S. Hamad, The Symbol Grounding Problem, Physica D, 1990, 42, 335–346CrossRefGoogle Scholar
  13. 13.
    J.R. Searle, Du cerveau au savoir, 1987, HermannGoogle Scholar
  14. 14.
    J.A. Meyer and S.W. Wilson, From Animals to Animats, First International Conference on Simulation of Adaptive Behavior, 1991, MIT Press, Bardford BooksGoogle Scholar
  15. 15.
    J. Stewart, The Implication for Understanding High-level Cognition of a Grounding in Elementary Adaptive Systems, Robotics and Autonomous Systems, 1995, December, 16, 2–4, 107–116Google Scholar
  16. 16.
    J. Piaget, La naissance de l’intelligence chez l’enfant, 1936, Delachaux et Niestle Editions, Neuchâtel-Paris, GeneveGoogle Scholar
  17. 17.
    I.P. Pavlov, Conditioned Reflexes, 1927, Oxford University PressGoogle Scholar
  18. 18.
    E.C. Tolman, Purposive behavior of animals and men, Irvington, 1932, New York, 189–208Google Scholar
  19. 19.
    E.C. Tolman, Cognitive maps in rats and men, The Psychological Review, 1948, 55, 4Google Scholar
  20. 20.
    P. Gaussier and S. Zrehen, PerAc: A Neural Architecture to Control Artificial Animals, Robotics and Autonomous System, 1995, December, 16, 2–4, 291–320Google Scholar
  21. 21.
    J.S. Albus, Outline for a Theory of Intelligence, IEEE trans. on syst. and cybern., 1991, may/june, 21, 3, 473–509Google Scholar
  22. 22.
    Rodney A. Brooks, A Robust Layered Control System for a Mobile Robot, IEEE Journal of Robotics and Automation, 1986, March, R.A. 2, 1, 14–23Google Scholar
  23. 23.
    R. Hecht-Nielsen, Counterpropagation Networks, Applied Optics, 1987, 26, 23, 4979–4984CrossRefGoogle Scholar
  24. 24.
    G.A. Carpenter and S. Grossberg, A massively parallel architecture for self-organizing neural pattern recognition machine, Computer Vision, Graphics, and Image Processing, 1987, 37, 54–115MATHCrossRefGoogle Scholar
  25. 25.
    P. Gaussier and S. Zrehen, PerAc: A neural architecture to control artificial animals, Robotics and Autonomous Systems, 1995, 16, 2–4, 291–320Google Scholar
  26. 26.
    P. Gaussier and S. Zrehen, Avoiding the World Model Trap: An Acting Robot Does Not Need to Be So Smart!, Journal of Robotics and Computer-Integrated Manufacturing, 1994, 11, 4, 279–286CrossRefGoogle Scholar
  27. 27.
    A.G. Barto and R.S. Sutton and D.S. Brouwer, Associative search network: A reinforcement learning associative memory, Biological cybernetics, 1981, 40, 201–211MATHCrossRefGoogle Scholar
  28. 28.
    A.G. Barto and R.S. Sutton and C.W. Anderson, Neuronlike adaptive elements that can solve difficult control problems, IEEE transactions on system, man and cybernetics, 1983, Sep/Oct, SMC-13, 5, 834–846Google Scholar
  29. 29.
    S. Mahadevan and J. Connell, Automatic programming of behavior -based robots using reinforcement learning, Ninth National Conference on Artificial Intelligence, 1991, Menlo Park, CAGoogle Scholar
  30. 30.
    J R Milian, Learning Efficient Reactive Behavioral Sequences from Basic Reflexes in a Goal-Directed Autonomous Robot, From Animals to Animats: SAB’94, 1994, D.Cliff and P. Husbands and J.A. Meyer and S.W. Wilson, 266–274Google Scholar
  31. 31.
    J.L. McClelland and D.E. Rumeihart and G.E. Hinton, PDP, The Appeal of Parallel Distributed Processing, 1986, MIT Press, CambridgeGoogle Scholar
  32. 32.
    U. Nehmzow and T.Smithers, Mapbuilding using self-organising networks, From Animals to Animats: SAB’91, 1991, J.A. Meyer and S. Wilson, MIT Press, Cambridge, MAGoogle Scholar
  33. 33.
    I. Krechevsky, The genesis of “hypotheses” in rats., Univ. Calif. Publ. Psychol., 1932, 6, 4, 46Google Scholar
  34. 34.
    M. Levine, Hypothesis Theory and Nonlearning Despite Ideal S-RReinforcement Contingencies, Psychological Review, 1971, 78, 2, 130–140CrossRefGoogle Scholar
  35. 35.
    T. Trabasso, Stimulus emphasis and all-or-none learning of concept identification, Journal of Experimental Psychology, 1963, 65, 395–406CrossRefGoogle Scholar
  36. 36.
    S.E. Weaver and A.H. Klopf and J.S. Morgan, A hierarchical network of control systems that learn: modeling nervous system function during classical and instrumental conditioning, Adaptive behavior, 1993, 1, 3, 263–319CrossRefGoogle Scholar
  37. 37.
    M. Levine, A Model of Hypothesis Behavior in Discrimination Learning Set, Psychological Review, 1959, 66, 6, 353–366CrossRefGoogle Scholar
  38. 38.
    J.P. Changeux, Neuronal Man: The Biology of Mind, 1985, Oxford University PressGoogle Scholar
  39. 39.
    G. Edelman, Neural Darwinism: The Theory of Neuronal Group Selection, 1987, Basic Books, New YorkGoogle Scholar
  40. 40.
    G.N. Reeke and O. Sporns and G.M. Edelman, Synthetic Neural Modeling: The “Darwin” Series of Recognition Automata, IEEE Proceedings, Special issue on Neural Networks, IEEE, 1990, September, C. Lau and B. Widrow, 1498–1530Google Scholar
  41. 41.
    A.G. Barto and R.S. Sutton, Landmark learning: an illustration of associative search, Biological cybernetics, 1981, 42, 1–8MATHCrossRefGoogle Scholar
  42. 42.
    C. Joulain and P. Gaussier, What can robots take for free ? Learning to build visual categories from sensori-motor associations, ETIS-ENSEA, 1996Google Scholar
  43. 43.
    P. Gaussier and J.P. Cocquerez, Neural networks for complex scene recognition: simulation of a visual system with several cortical areas, IJCNN Baltimore, 1992, 233–259Google Scholar
  44. 44.
    S. Zrehen and P. Gaussier, Why topological maps are useful for learning in an autonomous agent, From perception to action conference, 1994, J.D. Nicoud and P. Gaussier, IEEE Press, Los Alamitos, CAGoogle Scholar
  45. 45.
    C.R. Gallistel, The organization of learning, 1993, MIT PressGoogle Scholar
  46. 46.
    O. Trullier and S.I. Wiener and A. Berthoz and J.A. Meyer, Biologically based artificial navigation systems: review and prospects, Progress in Neurobiology, 1997, 51, 483–544CrossRefGoogle Scholar
  47. 47.
    N.A. Schmajuk and A.D. Thieme, Purposive behavior and cognitive mapping: a neural network model, Biological Cybernetics, 1992, 67, 165–174MATHCrossRefGoogle Scholar
  48. 48.
    G. Bugmann and J.G. Taylor and M J Denham, Neural Networks, Route finding by neural nets, Alfred Waller Ltd., 1995, J.G. Taylor, 217–230, Henley-on-ThamesGoogle Scholar
  49. 49.
    R.E. Bellman, On a routing problem, Quarterly of Applied Mathematics, 1958, 16, 87–90MathSciNetMATHGoogle Scholar
  50. 50.
    Y. Burnod, An adaptive neural network: the cerebral cortex, 1989, Collection biologie théorique, MassonGoogle Scholar
  51. 51.
    J.P. Banquet and J.L. Contreras-Vidal and P. Gaussier and Y. Burnod, Fundamentals of neural network modelling for neuropsychologists, The corticalhippocampal system as a multirange temporal processor: A neural model, 1996, R. Park and D. Levin, MIT Press, BostonGoogle Scholar
  52. 52.
    P. Gaussier and C. Joulain and S. Zrehen and J.P. Banquet and A. Revel, Visual Navigation in an open environment without map, International Conference on Intelligent Robots and Systems–IROS’97, 1997, IEEE/RSJ, Grenoble, France, September, 545–550Google Scholar
  53. 53.
    C. Thinus-blanc, Animal saptial cognition, 1996, World ScientificGoogle Scholar
  54. 54.
    Alexei Samsonovich and Bruce McNaughton, Path integration and cognitive mapping in a continuous attractor neural network model, Journal of Neuroscience, 1997, 17, 5900–5920Google Scholar
  55. 55.
    R. Wehner and B. Michel and P. Antonsen, Visual navigation in insects: coupling of egocentric and geocentric information, Journal of Experimental Biology, 1996, 199, 129–140Google Scholar
  56. 56.
    A. Redish and D. Touretzky, Separating hippocampal maps, The hippocampus and parietal foundations of spatial cognition, 1999, K. Jeffery and N. Burgess and J. O’Keefe, Monte Verita, Ticino, Switzerland, Oxford University PressGoogle Scholar
  57. 57.
    A. Etienne, Mammalian Navigation, Neural Models and Biorobotics, Connection Science, 1998, 10, 3–4, 271–289Google Scholar
  58. 58.
    C.J.C.H. Watkins, Learning from delayed rewards, 1989, Cambridge, England, Psychology Department, Cambridge UniversityGoogle Scholar
  59. 59.
    L. Pack Kaelbling and M.L. Littman and A.W. Moore, Reinforcement Learning: A Survey, Journal of Artificial Intelligence Research, 1996, Practice and Future of Autonomous Agents, R. Pfeifer, Monte Verita, Ticino, SwitzerlandGoogle Scholar
  60. 60.
    M.L. Littman, Memoryless policies: Theoritical limitations and practical results., From Animals to Animats: SAB’94, 1994, D. Cliff and P. Husbands and J.A.Meyer and S.W.Wilson, MIT Press, Cambridge, MA, 238–245Google Scholar
  61. 61.
    Steven D. Whitehead, Complexity and cooperation in Q-learning, Eight International Conference on Machine Learning, 1991, Morgan Kaufman, Evanston, IL, 363–367Google Scholar
  62. 62.
    S. Thrun and T.M. Mitchell, Lifelong Robot Learning, Robotics and Autonomous Systems, 1995, 15, 25–46CrossRefGoogle Scholar
  63. 63.
    I.A. Bachelor and A.M. Waxman, Mobile robot visual mapping and localization: a view-based neuro computational architecture that emulates hippocampal place learning, Neural Networks, 1994, 6 /7, 1083–1099CrossRefGoogle Scholar
  64. 64.
    G. Schöner and M. Dose and C. Engels, Dynamics of behavior: theory and applications for autonomous robot architectures, Robotics and Autonomous System, 1995, December, 16, 2–4, 213–245Google Scholar
  65. 65.
    J.Y. Donnant and J.A. Meyer, Learning reactive and planning rules in a motivationnally autonomous animat, IEEE Transactions on Systems, Man and Cybernetics-Part B, 1996, 26, 3, 381–395CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • A. Revel
    • 1
  • P. Gaussier
    • 1
  1. 1.Neurocybernetic Team - ETIS ENSEA/UCPCergy-Pontoise CedexFrance

Personalised recommendations