Artificial Evolution of Plastic Neural Networks: A Few Key Concepts

  • Jean-Baptiste MouretEmail author
  • Paul Tonelli
Part of the Studies in Computational Intelligence book series (SCI, volume 557)


This chapter introduces a hierarchy of concepts to classify the goals and the methods used in articles that mix neuro-evolution and synaptic plasticity. We propose definitions of “behavioral robustness” and oppose it to “reward-based behavioral changes”; we then distinguish the switch between behaviors and the acquisition of new behaviors. Last, we formalize the concept of “synaptic General Learning Abilities” (sGLA) and that of “synaptic Transitive learning Abilities (sTLA)”. For each concept, we review the literature to identify the main experimental setups and the typical studies.


Synaptic Plasticity Artificial Agent Synaptic Weight Structural Plasticity Reward Signal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was funded by the EvoNeuro project (ANR-09-EMER-005-01) and the Creadapt project (ANR-12-JS03-0009).


  1. 1.
    C.H. Bailey, M. Giustetto, Y.Y. Huang, R.D. Hawkins, E.R. Kandel, Is heterosynaptic modulation essential for stabilizing Hebbian plasticity and memory? Nature Rev. Neurosci. 1(1), 11–20 (2000)CrossRefGoogle Scholar
  2. 2.
    J. Blynel, D. Floreano, Levels of dynamics and adaptive behavior in evolutionary neural controllers, in Conference on Simulation of Adaptive Behavior (SAB) (2002), pp. 272–281Google Scholar
  3. 3.
    D.J. Chalmers, The Evolution of Learning: An Experiment in Genetic Connectionism (Connectionist Models Summer School, 1990)Google Scholar
  4. 4.
    J. Clune, K.O. Stanley, R.T. Pennock, C. Ofria, On the performance of indirect encoding across the continuum of regularity. IEEE Trans. Evol. Comput. 15(3), 346–367 (2011)Google Scholar
  5. 5.
    N.D. Daw, Y. Niv, P. Dayan, Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neurosci. 8(12), 11–1704 (2005)CrossRefGoogle Scholar
  6. 6.
    D. Floreano, Evolution of plastic neurocontrollers for situated agents, in Conference on Simulation of Adaptive Behavior (SAB) (1996)Google Scholar
  7. 7.
    D. Floreano, P. Dürr, C. Mattiussi, Neuroevolution: from architectures to learning. Evol. Intell. 1(1), 47–62 (2008)CrossRefGoogle Scholar
  8. 8.
    D. Floreano, C. Mattiussi, Bio-inspired Artificial Intelligence: Theories, Methods, and Technologies (The MIT Press, 2008)Google Scholar
  9. 9.
    S. Haykin, Neural networks: a comprehensive foundation (Prentice Hall, Upper Saddle River, 1999)Google Scholar
  10. 10.
    J. Kodjabachian, J.-A. Meyer, Evolution and development of neural controllers for locomotion, gradient-following, and obstacle-avoidance in artificial insects. IEEE Trans. Neural Networks 9(5), 796–812 (1998)CrossRefGoogle Scholar
  11. 11.
    T. Kondo, Evolutionary design and behavior analysis of neuromodulatory neural networks for mobile robots control. Appl. Soft Comput. 7(1), 189–202 (2007)CrossRefGoogle Scholar
  12. 12.
    J.-B. Mouret, S. Doncieux, Using behavioral exploration objectives to solve deceptive problems in neuro-evolution, in Conference on genetic and evolutionary computation (GECCO) (2009)Google Scholar
  13. 13.
    J.-B. Mouret, S. Doncieux, B. Girard, Importing the computational neuroscience toolbox into neuro-evolution—application to basal ganglia, in Conference on genetic and evolutionary computation (GECCO) (2010)Google Scholar
  14. 14.
    Y. Niv, D. Joel, I. Meilijson, E. Ruppin, Evolution of reinforcement learning in uncertain environments: a simple explanation for complex foraging behaviors. Adapt. Behav. 10(1), 5–24 (2002)CrossRefGoogle Scholar
  15. 15.
    S. Nolfi, How learning and evolution interact: the case of a learning task which differs from the evolutionary task. Adapt. Behav. 4(1), 81–84 (1999)Google Scholar
  16. 16.
    S. Nolfi, O. Miglino, D. Parisi, Phenotypic plasticity in evolving neural networks. in From Perception to Action Conference (IEEE, 1994), pp. 146–157Google Scholar
  17. 17.
    S. Risi, K.O. Stanley, Indirectly encoding neural plasticity as a pattern of local rules, in Conference on Simulation of Adaptive Behavior (SAB) (2010)Google Scholar
  18. 18.
    S. Risi, S.D. Vanderbleek, C.E. Hughes, K.O. Stanley, How novelty search escapes the deceptive trap of learning to learn, in Conference on genetic and evolutionary computation (GECCO) (2009)Google Scholar
  19. 19.
    W. Schultz, P. Dayan, P.R. Montague, A neural substrate of prediction and reward. Science 275(5306), 1593–1599 (1997)CrossRefGoogle Scholar
  20. 20.
    B.F. Skinner, Operant behavior. Am. Psychol. 18(8), 503 (1963)CrossRefGoogle Scholar
  21. 21.
    A. Soltoggio, J.A. Bullinaria, C. Mattiussi, P. Dürr, D. Floreano, Evolutionary advantages of neuromodulated plasticity in dynamic, reward-based scenarios. Artif. Life 11, 569 (2008)Google Scholar
  22. 22.
    A. Soltoggio, P. Dürr, C. Mattiussi, D. Floreano, Evolving neuromodulatory topologies for reinforcement learning-like problems, in IEEE Congress on Evolutionary Computation (CEC) (2007)Google Scholar
  23. 23.
    A. Soltoggio, B. Jones, Novelty of behaviour as a basis for the neuro-evolution of operant reward learning, in Conference on genetic and evolutionary computation (GECCO) (2009)Google Scholar
  24. 24.
    K.O. Stanley, B.D. Bryant, R. Miikkulainen, Evolving adaptive neural networks with and without adaptive synapses, in IEEE Congress on Evolutionary Computation (CEC) (2003)Google Scholar
  25. 25.
    K.O. Stanley, D. D’Ambrosio, J. Gauci, A hypercube-based indirect encoding for evolving large-scale neural networks. Artif. Life 15(2), 185–212 (2009)CrossRefGoogle Scholar
  26. 26.
    K.O. Stanley, R. Miikkulainen, Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)CrossRefGoogle Scholar
  27. 27.
    R.S. Sutton, A.G. Barto, Reinforcement learning: An introduction (The MIT press, 1998)Google Scholar
  28. 28.
    P. Tonelli, J.-B. Mouret, On the relationships between synaptic plasticity and generative systems, in Conference on genetic and evolutionary computation (GECCO) (2011)Google Scholar
  29. 29.
    P. Tonelli, J.-B. Mouret, On the relationships between generative encodings, regularity, and learning abilities when evolving plastic artificial neural networks. PLoS One. 8(11), e79138 (2013)Google Scholar
  30. 30.
    J. Urzelai, D. Floreano, Evolution of adaptive synapses: robots with fast adaptive behavior in new environments. Evol. Comput. 9(4), 495–524 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Institut des Systèmes Intelligents et de Robotique (ISIR), UMR 7222Sorbonne UniversitésParisFrance
  2. 2.UMR 7222, ISIRParisFrance

Personalised recommendations