Skip to main content
SpringerLink
Log in

An evolutionary cognitive architecture made of a bag of networks

  • Special Issue
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

A cognitive architecture is presented for modelling some properties of sensorimotor learning in infants, namely the ability to accumulate adaptations and skills over multiple tasks in a manner which allows recombination and re-use of task specific competences. The control architecture invented consists of a population of compartments (units of neuroevolution) each containing networks capable of controlling a robot with many degrees of freedom. The nodes of the network undergo internal mutations, and the networks undergo stochastic structural modifications, constrained by a mutational and recombinational grammar. The nodes used consist of dynamical systems such as dynamic movement primitives, continuous time recurrent neural networks and high-level supervised and unsupervised learning algorithms. Edges in the network represent the passing of information from a sending node to a receiving node. The networks in a compartment operate in parallel and encode a space of possible subsumption-like architectures that are used to successfully evolve a variety of behaviours for a NAO H25 humanoid robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. Supplementary material is available at http://www.alexchurchill.com/papers/evin2014.

References

  1. Baldassarre G, Mirolli M (2013) Intrinsically motivated learning in natural and artificial systems. Springer, New York

    Book  Google Scholar 

  2. Baranes A, Oudeyer PY (2013) Active learning of inverse models with intrinsically motivated goal exploration in robots. Robot Auton Syst 61(1):49–73

    Article  Google Scholar 

  3. Bellas F, Duro RJ, Faiña A, Souto D (2010) Multilevel darwinist brain (mdb): artificial evolution in a cognitive architecture for real robots. IEEE Trans Auton Mental Dev 2(4):340–354

    Article  Google Scholar 

  4. Brooks RA (1990) Elephants don’t play chess. Robot Auton Syst 6(1):3–15

    Article  Google Scholar 

  5. Bull L, Kovacs T (2005) Foundations of learning classifier systems, vol 183. Springer, New York

    MATH  Google Scholar 

  6. Butz MV, Herbort O (2008) Context-dependent predictions and cognitive arm control with xcsf. In: Proceedings of the 10th annual conference on Genetic and evolutionary computation, ACM, New York, pp 1357–1364

  7. Calabretta R, Nolfi S, Parisi D, Wagner GP (2000) Duplication of modules facilitates the evolution of functional specialization. Artif Life 6(1):69–84

    Article  Google Scholar 

  8. Chklovskii D, Mel B, Svoboda K (2004) Cortical rewiring and information storage. Nature 431:782–788

    Article  Google Scholar 

  9. Cliff D, Ross S (1994) Adding temporary memory to zcs. Adapt Behav 3(2):101–150

    Article  Google Scholar 

  10. Clune J, Mouret JB, Lipson H (2013) The evolutionary origins of modularity. Proc R Soc B Biol Sci 280(1755):20122,863

    Article  Google Scholar 

  11. Crapse TB, Sommer MA (2008) Corollary discharge across the animalkingdom. Nat Rev Neurosci 9(8):587–600 [Crapse, Trinity B Sommer, Marc A R01-EY017592/EY/NEI NIH HHS/United States Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov’t Review England Nature reviews]. Neuroscience Nat Rev Neurosci. 2008 Aug; 9(8):587–600

  12. Dearden A, Demiris Y (2005) Learning forward models for robots. In: IJCAI, vol 5, p 1440

  13. Der R, Martius G (2012) The playful machine. Cognitive systems monographs. Springer, New York

    Google Scholar 

  14. Dorigo M (1998) Robot shaping: an experiment in behavior engineering. MIT press, Cambridge

    Google Scholar 

  15. Durr P, Mattiussi C, Floreano D (2010) Genetic representation and evolvability of modular neural controllers. IEEE Comput Intell Mag 5(3):10–19

    Article  Google Scholar 

  16. Eaton M (2013) An approach to the synthesis of humanoid robot dance using non-interactive evolutionary techniques. In: IEEE international conference on systems, man, and cybernetics (SMC), pp 3305–3309. IEEE

  17. Eaton M, Davitt TJ (2007) Evolutionary control of bipedal locomotion in a high degree-of-freedom humanoid robot: first steps. Artif Life Robot 11(1):112–115

    Article  Google Scholar 

  18. Farchy A, Barrett S, MacAlpine P, Stone P (2013) Humanoid robots learning to walk faster: from the real world to simulation and back. In: Proceedings of the 2013 international conference on autonomous agents and multi-agent systems, pp 39–46. International foundation for autonomous agents and multiagent systems

  19. Fernando C, Zachar I, Szathmry E (2010) Linguistic constructions as neuronal replicators. In: Steels L (ed) Fluid construction grammar. John Bengamin, Oxford

    Google Scholar 

  20. Fodor J, Pylyshyn Z (1988) Connectionism and cognitive architecture: a critical analysis. Cognition 28:3–71

    Article  Google Scholar 

  21. Gomez F, Miikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5(3–4):317–342

    Article  Google Scholar 

  22. Gordon G, Ahissar E (2012) A curious emergence of reaching, lecture notes in computer science, vo. 7429, chap 1. Springer, Berlin, pp 1–12

  23. Gordon G, Ahissar E (2012) Hierarchical curiosity loops and active sensing. Neural Netw 32:119–129 [Gordon, Goren Ahissar, Ehud Neural Netw. 2012 Aug; 32:119–129. Epub 2012 Feb 14]

  24. Haruno M, Wolpert D, Kawato M (2001) Mosaic model for sensorimotor learning and control. Neural Comput 13:2201–2220

    Article  MATH  Google Scholar 

  25. Harvey I (2011) The microbial genetic algorithm. In: Advances in artificial life. Darwin meets von Neumann. Springer, New York, pp 126–133

  26. Harvey I, Husbands P, Cliff D (1994) Seeing the light: artificial evolution, real vision. School of Cognitive and Computing Sciences, University of Sussex, Falmer

    Google Scholar 

  27. Hurst J, Bull L (2006) A neural learning classifier system with self-adaptive constructivism for mobile robot control. Artif Life 12(3):353–380

    Article  Google Scholar 

  28. Husbands P, Harvey I, Cliff D (1993) An evolutionary approach to situated ai. In: Proceedings of the 9th Bi-annual conference of the society for the study of artificial intelligence and the simulation of behaviour (AISB 93), pp 61–70

  29. Izhikevich EM (2006) Polychronization: computation with spikes. Neural Comput 18(2):245–282 [Izhikevich, Eugene M. Neural Comput.18(2), 245–82 (2006)]

  30. Jordan MI, Rumelhart DE (1992) Forward models: supervised learning with a distal teacher. Cogn Sci 16(3):307–354

    Article  Google Scholar 

  31. Koos S, Cully A, Mouret JB (2013) Fast damage recovery in robotics with the t-resilience algorithm. Int J Robot Res 32(14):1700–1723. doi:10.1177/0278364913499192

    Article  Google Scholar 

  32. Koza JR (1992) Evolution of subsumption using genetic programming. In: Proceedings of the 1st European conference on artificial life, pp 110–119

  33. Koza JR (1999) Genetic programming III: darwinian invention and problem solving. Morgan Kaufmann, San Francisco. 99010099 [John R. Koza et al. ill.; 25 cm. Includes bibliographical references (p. [1081]–1114)

  34. Lalazar H, Vaadia E (2008) Neural basis of sensorimotor learning: modifying internal models. Curr Opin Neurobiol 18((6)):573–581

    Article  Google Scholar 

  35. Lehman J, Stanley KO (2011) Abandoning objectives: evolution through the search for novelty alone. Evol Comput 19(2):189–223

    Article  Google Scholar 

  36. Lewis FL, Vrabie D, Syrmos VL (2012) Optimal control, 3rd edn. Wiley, Hoboken

  37. Li H, Liao X, Carin L (2009) Multi-task reinforcement learning in partially observable stochastic environments. J Mach Learn Res 10(1131–1186):1577109

    MathSciNet  Google Scholar 

  38. Mckay RI, Hoai NX, Whigham PA, Shan Y, ONeill M (2010) Grammar-based genetic programming: a survey. Genet Program Evol Mach 11(3–4):365–396

    Article  Google Scholar 

  39. Miller JF, Khan GM (2011) Where is the brain inside the brain? Memet Comput 3(3):217–228

    Article  Google Scholar 

  40. Miller JF, Thomson P (2000) Cartesian genetic programming. In: Genetic programming. Springer, New York, pp 121–132

  41. Mohamed Z, Kitani M, Kaneko Si, Capi G (2014) Humanoid robot arm performance optimization using multi objective evolutionary algorithm. Int J Control Autom Syst 12(4):870–877

    Article  Google Scholar 

  42. Moriguchi H, Lipson H (2011) Learning symbolic forward models for robotic motion planning and control. In: Proceedings of European conference of artificial life (ECAL 2011), pp 558–564

  43. Mouret JB, Doncieux S (2009) Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity. In: IEEE congress on evolutionary computation, 2009. CEC’09, pp 1161–1168. IEEE

  44. Oudeyer PY, Kaplan F, Hafner VV (2007) Intrinsic motivation systems for autonomous mental development. IEEE Trans Evol Comput 11(2):265–286

    Article  Google Scholar 

  45. Pape L, Oddo CM, Controzzi M, Cipriani C, Frster A, Carrozza MC, Schmidhuber J(2012) Learning tactile skills through curious exploration. Frontiers in neurorobotics 6

  46. Poli R (1996) Parallel distributed genetic programming. Citeseer

  47. Rozenberg G (1997) Handbook of graph grammars and computing by graph transformation. World Scientific, Singapore. 96037597 edited by Grzegorz Rozenberg. ill.; 23 cm. Includes bibliographical references and indexes. v. 1. Foundations

  48. Savastano P, Nolfi S (2013) A robotic model of reaching and grasping development

  49. Schaal S (2006) Dynamic movement primitives-a framework for motor control in humans and humanoid robotics. In: Adaptive motion of animals and machines. Springer, New York, pp 261–280

  50. Schaal S, Peters J, Nakanishi J, Ijspeert A (2005) Learning movement primitives. In: Robotics research. Springer, New York, pp 561–572

  51. Schmidhuber J (2009) Driven by compression progress: a simple principle explains essential aspects of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes. In: Anticipatory behavior in adaptive learning systems. Springer, New York, pp 48–76

  52. Schrum J, Miikkulainen R (2014) Evolving multimodal behavior with modular neural networks in ms. pac-man. In: Proceedings of the genetic and evolutionary computation conference (GECCO 2014). Vancouver, BC, Canada, pp 325–332. http://nn.cs.utexas.edu/?schrum:gecco2014. Best Paper: Digital Entertainment and Arts

  53. Shadmehr R (2004) Generalization as a behavioral window to the neural mechanisms of learning internal models. Hum Mov Sci 23:543–568

    Article  Google Scholar 

  54. Sporns O, Edelman GM (1993) Solving bernstein’s problem: a proposal for the development of coordinated movement by selection. Child Dev 64:960–981

    Article  Google Scholar 

  55. Stanley KO, D’Ambrosio DB, Gauci J (2009) A hypercube-based encoding for evolving large-scale neural networks. Artif Life 15(2):185–212

    Article  Google Scholar 

  56. Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99–127

    Article  Google Scholar 

  57. Steels L, De Beule J (2006) Unify and merge in fluid construction grammar

  58. Stolle M, Precup D (2002) Learning options in reinforcement learning, lecture notes in computer science, vol 2371, chap 16. Springer, Berlin, pp 212–223

  59. Studley M, Bull L (2005) X-tcs: accuracy-based learning classifier system robotics. In: The 2005 IEEE congress on evolutionary computation, vol 3, pp 2099–2106. IEEE

  60. Sturm J, Plagemann C, Burgard W (2008) Unsupervised body scheme learning through self-perception. In: IEEE international conference on robotics and automation, 2008. ICRA 2008, pp 3328–3333. IEEE

  61. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Adaptive computation and machine learning. MIT Press, Cambridge, MA. [97026416 Richard S. Sutton and Andrew G. Barto. ill.; 24 cm. Includes bibliographical references (p. [291]–312) and index]

  62. Teller A, Veloso M (1995) Program evolution for data mining. Int J Exp Syst Res Appl 8:213–236

    Article  Google Scholar 

  63. Teller A, Veloso M (1996) Neural programming and an internal reinforcement policy. In: Late breaking papers at the genetic programming 1996 conference, pp 186–192. Citeseer, New Jersey

  64. Thelen E (1995) Motor development: a new synthesis. Am Psychol 50(2):79

    Article  Google Scholar 

  65. Todorov E, Jordan MI (2002) Optimal feedback control as a theory of motor coordination. Nat Neurosci 5(11):1226–1235. doi:10.1038/nn963

    Article  Google Scholar 

  66. Togelius J (2004) Evolution of a subsumption architecture neurocontroller. J Intell Fuzzy Syst 15(1):15–20

    Google Scholar 

  67. Urzelai J, Floreano D, Dorigo M, Colombetti M (1998) Incremental robot shaping. Connect Sci 10(3–4):341–360

    Article  Google Scholar 

  68. Wilson SW (2000) Get real! xcs with continuous-valued inputs. In: Learning classifier systems. Springer, New York, pp 209–219

Download references

Acknowledgments

The work is funded by the FQEB Templeton grant “Bayes, Darwin and Hebb”, and the FP-7 FET OPEN Grant INSIGHT.

Author information

Authors and Affiliations

  1. School of Electronic Engineering and Computer Science, Queen Mary, University of London, London, UK

    Alexander W. Churchill & Chrisantha Fernando

Authors
  1. Alexander W. Churchill
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chrisantha Fernando
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander W. Churchill.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Churchill, A.W., Fernando, C. An evolutionary cognitive architecture made of a bag of networks. Evol. Intel. 7, 169–182 (2014). https://doi.org/10.1007/s12065-014-0121-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-014-0121-7

Keywords

Search

Navigation