Countering Poisonous Inputs with Memetic Neuroevolution

  • Julian Togelius
  • Tom Schaul
  • Jürgen Schmidhuber
  • Faustino Gomez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5199)


Applied to certain problems, neuroevolution frequently gets stuck in local optima with very low fitness; in particular, this is true for some reinforcement learning problems where the input to the controller is a high-dimensional and/or ill-chosen state description. Evidently, some controller inputs are “poisonous”, and their inclusion induce such local optima. Previously, we proposed the memetic climber, which evolves neural network topology and weights at different timescales, as a solution to this problem. In this paper, we further explore the memetic climber, and introduce its population-based counterpart: the memetic ES. We also explore which types of inputs are poisonous for two different reinforcement learning problems.


Random Search Memetic Algorithm Standard Input Neural Network Weight Reinforcement Learning Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lucas, S.M., Togelius, J.: Point-to-point car racing: an initial study of evolution versus temporal difference learning. In: Proceedings of the IEEE Symposium on Computational Intelligence and Games (2007)Google Scholar
  2. 2.
    Igel, C.: Neuroevolution for reinforcement learning using evolution strategies. In: Proceedings of the Congress on Evolutionary Computation (CEC) (2003)Google Scholar
  3. 3.
    De Nardi, R., Togelius, J., Holland, O., Lucas, S.M.: Evolution of neural networks for helicopter control: Why modularity matters. In: Proceedings of the IEEE Congress on Evolutionary Computation (2006)Google Scholar
  4. 4.
    Togelius, J., Gomez, F., Schmidhuber, J.: Learning what to ignore: memetic climbing in weight and topology space. In: Congress on Evolutionary Computation (CEC) (to be presented, 2008)Google Scholar
  5. 5.
    Yao, X.: Evolving artificial neural networks. Proceedings 1447, 87(9) (1999)Google Scholar
  6. 6.
    Siebel, N.T., Sommer, G.: Evolutionary reinforcement learning of artificial neural networks. International Journal of Hybrid Intelligent Systems 4(3), 171–183 (2007)zbMATHGoogle Scholar
  7. 7.
    Krasnogor, N., Pacheco, A.A.: Memetic algorithms. In: Metaheuristics in Neural Networks Learning, pp. 225–247. Springer, Heidelberg (2006)Google Scholar
  8. 8.
    Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 10(2), 99–127 (2002)CrossRefGoogle Scholar
  9. 9.
    Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)CrossRefGoogle Scholar
  10. 10.
    Togelius, J.: Optimization, Imitation and Innovation: Computational Intelligence and Games. PhD thesis, Department of Computing and Electronic Systems, University of Essex, Colchester, UK (2007)Google Scholar
  11. 11.
    Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. Journal of Machine Learning Research 9, 937–965 (2008)zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Julian Togelius
    • 1
  • Tom Schaul
    • 1
  • Jürgen Schmidhuber
    • 1
    • 2
  • Faustino Gomez
    • 1
  1. 1.IDSIAManno-LuganoSwitzerland
  2. 2.TU MunichGarching, MünchenGermany

Personalised recommendations