Skip to main content
SpringerLink
Log in

Artificial Neurogenesis: An Introduction and Selective Review

  • Chapter
  • First Online:
Book cover Growing Adaptive Machines

Part of the book series: Studies in Computational Intelligence ((SCI,volume 557))

Abstract

In this introduction and review—like in the book which follows—we explore the hypothesis that adaptive growth is a means of producing brain-like machines. The emulation of neural development can incorporate desirable characteristics of natural neural systems into engineered designs. The introduction begins with a review of neural development and neural models. Next, artificial development—the use of a developmentally-inspired stage in engineering design—is introduced. Several strategies for performing this “meta-design” for artificial neural systems are reviewed. This work is divided into three main categories: bio-inspired representations; developmental systems; and epigenetic simulations. Several specific network biases and their benefits to neural network design are identified in these contexts. In particular, several recent studies show a strong synergy, sometimes interchangeability, between developmental and epigenetic processes—a topic that has remained largely under-explored in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Further complicating this picture are recent results showing that these connections might themselves be information processing units, which would increase this estimation by several orders of magnitude [196].

  2. 2.

    By epigenetic, we mean here any heritable and non-genetic changes in cellular expression. (The same term is also used in another context to refer strictly to DNA methylation and transcription-level mechanisms.) This includes processes such as learning for an animal, or growing toward a light source for a plant. The mentioned time scale represents a rough average over cellular responses to environmental stimuli.

  3. 3.

    The authors point out a similarity between their developed NNs and natural networks, specifically the existence of higher-order network triads. However, Milo et al. [202] attribute the existence of such triads in natural networks to the minimization of information processing time, a factor which was not relevant to the NNs. Hence, we consider this similarity unexplained.

  4. 4.

    Caveat: while neuroevolution is known to be more versatile than, for instance, classic CoNN algorithms, it is also known that evolutionary computation will be often hindered by local optima in the fitness landscape, suggesting a possibly different sort of suboptimality.

References

  1. M. Abeles, Local Cortical Circuits: An Electrophysiological Study, vol. 6 (Springer, New York, 1982)

    Google Scholar 

  2. W.C. Abraham, M.F. Bear, Metaplasticity: the plasticity of synaptic plasticity. Trends Neurosci. 19(4), 126–130 (1996)

    Google Scholar 

  3. I. Aho, H. Kemppainen, K. Koskimies, E. Makinen, T. Niemi, Searching neural network structures with l systems and genetic algorithms. Int. J. Comput. Math. 73(1), 55–75 (1999)

    MATH  MathSciNet  Google Scholar 

  4. U. Alon, An Introduction to Systems Biology: Design Principles of Biological Circuits (CRC Press, Boca Raton, 2007)

    Google Scholar 

  5. T. Andersen, R. Newman, T. Otter, Development of virtual embryos with emergent self-repair. in AAAI Fall Symposium (2006)

    Google Scholar 

  6. P.J. Angeline, G.M. Saunders, J.B. Pollack, An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans. Neural Netw. 5(1), 54–65 (1994)

    Google Scholar 

  7. W. Arthur, The effect of development on the direction of evolution: toward a twenty-first century consensus. Evol. Dev. 6(4), 282–288 (2004)

    MathSciNet  Google Scholar 

  8. F.A.C. Azevedo, L.R.B. Carvalho, L.T. Grinberg, J.M. Farfel, R.E.L. Ferretti, R.E.P. Leite, W.J. Filho, R. Lent, S. Herculano-Houzel, Equal numbers of neuronal and nonneuronal cells make the human brain an isometrically scaled-up primate brain. J. Comp. Neurol. 513(5), 532–541 (2009)

    Google Scholar 

  9. J.M. Baldwin, A new factor in evolution. Am. Nat. 30, 441–451 (1896)

    Google Scholar 

  10. W. Banzhaf, On the dynamics of an artificial regulatory network. in European Conference on Artificial Life (ECAL 2003) (Springer, Berlin, 2003), pp. 217–227

    Google Scholar 

  11. W. Banzhaf, N. Pillay, Why complex systems engineering needs biological development. Complexity 13(2), 12–21 (2007)

    Google Scholar 

  12. J. Beal, Functional blueprints: an approach to modularity in grown systems. Swarm Intell. 5(3–4), 257–281 (2011)

    Google Scholar 

  13. J.A. Bednar, Constructing Complex Systems Via Activity-Driven Unsupervised Hebbian Self-organization. in ed. by Kowaliw et al. [160], pp. 216–241

    Google Scholar 

  14. Y. Bengio, Evolving Culture Versus Local Minima. in ed. by Kowaliw et al. [160], pp. 112–143

    Google Scholar 

  15. Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, Greedy layer-wise training of deep networks. in Advances in Neural Information Processing Systems (2007)

    Google Scholar 

  16. Y. Bengio, Y. LeCun, Scaling learning algorithms towards AI. in Large Scale Kernel Machines (MIT Press, Cambridge, 2007)

    Google Scholar 

  17. P. Bentley, Three ways to grow designs: a comparison of embryogenies for an evolutionary design problem. in Conference on Genetic and Evolutionary Computation (1999), pp. 35–43

    Google Scholar 

  18. P. Bentley, Investigations into graceful degradation of evolutionary developmental software. Nat. Comput. 4(4), 417–437 (2005)

    MATH  MathSciNet  Google Scholar 

  19. E. Bienenstock, R. Doursat, Spatio-temporal coding and the compositionality of cognition. in Temporal Correlations and Temporal Coding in the Brain (1990), pp. 42–47

    Google Scholar 

  20. E. Bienenstock, C. von der Malsburg, A neural network for invariant pattern recognition. Europhys. Lett. 4(1), 121–126 (1987)

    Google Scholar 

  21. E. Bienenstock, R. Doursat, A shape-recognition model using dynamical links. Netw. Comput. Neural Syst. 5(2), 241–258 (1994)

    MATH  Google Scholar 

  22. E.J.W. Boers, H. Kuiper, Biological Metaphors and the Design of Modular Artificial Neural Networks Technical report (Leiden University, 1992)

    Google Scholar 

  23. J.C. Bongard, Evolving modular genetic regulatory networks. in IEEE Congress on Evolutionary Computation (CEC) (2002), pp. 1872–1877

    Google Scholar 

  24. J.C. Bongard, Spontaneous evolution of structural modularity in robot neural network controllers. in Conference on Genetic and Evolutionary Computation (GECCO) (Springer, 2011)

    Google Scholar 

  25. J.C. Bongard, R. Pfeifer, Evolving complete agents using artificial ontogeny. in ed. by F. Hara, R. Pfeifer Morpho-functional Machines: The New Species (Designing Embodied Intelligence) (Springer, 2003), pp. 237–258

    Google Scholar 

  26. M. Bortman, M. Aladjem, A growing and pruning method for radial basis function networks. IEEE Trans. Neural Netw. 20(6), 1039–1045 (2009)

    Google Scholar 

  27. R. Brette, M. Rudolph, N.T. Carnevale, M.L. Hines, D. Beeman, J. Bower, M. Diesmann, A. Morrison, P. Goodman, F. Harris Jr, M. Zirpe, T. Natschläger, D. Pecevski, G.B. Ermentrout, M. Djurfeldt, A. Lansner, O. Rochel, T. Viéville, E. Muller, A.P. Davison, S. El Boustani, A. Destexhe, Simulation of networks of spiking neurons: A review of tools and strategies. J. Comput. Neurosci. 23(3), 349–398 (2007)

    Google Scholar 

  28. D.J. Cahalane, B. Clancy, M.A. Kingsbury, E. Graf, O. Sporns, B.L. Finlay, Network structure implied by initial axon outgrowth in rodent cortex: empirical measurement and models. PLoS ONE 6(1), 01 (2011)

    Google Scholar 

  29. W. Callebaut, D. Rasskin-Gutman, Modularity: Understanding the Development and Evolution of Natural Complex Systems (MIT Press, 2005)

    Google Scholar 

  30. A. Cangelosi, D. Parisi, S. Nolfi. Cell division and migration in a genotype for neural networks. Conf. Comput. Netw. 497–515 (1994)

    Google Scholar 

  31. S. Carroll, J. Grenier, S. Weatherbee, From DNA to Diversity: Molecular Genetics and the Evolution of Animal Design 2nd edn (Blackwell Publishing, 2005)

    Google Scholar 

  32. R. Cattell, A. Parker, Challenges for brain emulation: why is it so difficult? Nat. Intell. INNS Mag. 1(3), 17–31 (2012)

    Google Scholar 

  33. L. Cazenille, N. Bredeche, H. Hamann, J. Stradner, Impact of neuron models and network structure on evolving modular robot neural network controllers. in Conference on Genetic and evolutionary computation (GECCO), (ACM Press, New York, 2012), p. 89

    Google Scholar 

  34. J.P. Changeux, A. Danchin, Nature 264, 705–712 (1976)

    Google Scholar 

  35. N. Cheney, R. Maccurdy, J. Clune, H. Lipson, Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding. in Genetic and Evolutionary Computation Conference (GECCO) (2013), pp. 167–174

    Google Scholar 

  36. N. Chomsky, Aspects of the Theory of Syntax (MIT Press, 1965)

    Google Scholar 

  37. J. Clune, B.E. Beckmann, P.K. McKinley, C. Ofria, Investigating whether hyperNEAT produces modular neural networks. in Conference on Genetic and Evolutionary Computation (GECCO), (ACM Press, New York, 2010), pp. 1523–1530

    Google Scholar 

  38. J. Clune, B.E. Beckmann, R.T. Pennock, C. Ofria, HybrID: A hybridization of indirect and direct encodings for evolutionary computation. in European Conference on Artificial Life (ECAL) (2009), pp. 134–141

    Google Scholar 

  39. J. Clune, J.B. Mouret, H. Lipson, The evolutionary origins of modularity. Proc. Roy. Soc. B Biol. Sci. 280(1755), 20122863 (2013)

    Google Scholar 

  40. J. Clune, K.O. Stanley, R.T. Pennock, C. Ofria, On the performance of indirect encoding across the continuum of regularity. IEEE Trans. Evol. Comput. 15(3), 346–367 (2011)

    Google Scholar 

  41. H. Cuntz, F. Forstner, A. Borst, M. Häusser, One rule to grow them all: a general theory of neuronal branching and its practical application. PLoS Comput. Biol. 6(e1000877), 08 (2010)

    Google Scholar 

  42. S. Cussat-Blanc, H. Luga, Y. Duthen, Cell 2Organ: Self-repairing artificial creatures thanks to a healthy metabolism. in IEEE Congress on Evolutionary Computation (CEC) (2009), pp. 2708–2715

    Google Scholar 

  43. S. Cussat-Blanc, J. Pascalie, S. Mazac, H. Luga, Y. Duthen, A synthesis of the cell2organ developmental model. in Doursat et al. [67], pp. 353–381

    Google Scholar 

  44. G. Cybenko, Approximations by superpositions of sigmoidal functions. Math. Control Sig. Syst. 4(2), 303–314 (1989)

    Google Scholar 

  45. N.M. da Costa, K.A.C. Martin, Whose cortical column would that be? Front. Neuroanat. 4(16), (2010)

    Google Scholar 

  46. G. Dahl, M. Ranzato, A. Mohamed, G.E. Hinton, Phone Recognition with the mean-covariance restricted Boltzmann machine. in Advances in Neural Information Processing Systems (2010), pp. 469–477

    Google Scholar 

  47. K. Dale, P. Husbands, The evolution of reaction-diffusion controllers for minimally cognitive agents. Artif. Life 16, 1–19 (2010)

    Google Scholar 

  48. D.B. DAmbrosio, J. Gauci, K.O. Stanley, HyperNEAT: the first five years. in ed. by Kowaliw et al. [160], pp. 167–197

    Google Scholar 

  49. D.B. D’Ambrosio, K.O. Stanley, A novel generative encoding for exploiting neural network sensor and output geometry. in Conference on Genetic and Evolutionary Computation (GECCO) (ACM Press, New York, 2007), pp. 974–982

    Google Scholar 

  50. H. De Garis, Growing an artificial brain: the genetic programming of million-neural-net-module artificial brains within trillion cell cellular automata machines. in Proceedings of the Third Annual Conference on Evolutionary Programming (1994), pp. 335–343

    Google Scholar 

  51. T.W. Deacon, Rethinking mammalian brain evolution. Am. Zool. 30(3), 629–705 (1990)

    Google Scholar 

  52. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, Q. Le, M. Mao, M. Ranzato, A. Senior, P. Tucker, K. Yang, A. Ng, Large scale distributed deep networks. in ed. by P. Bartlett, F.C.N. Pereira, C.J.C. Burges, L. Bottou, K.Q. Weinberger. Advances in Neural Information Processing Systems 25 (2012), pp. 1232–1240

    Google Scholar 

  53. A.S. Dekaban, D. Sadowsky, Changes in brain weights during the span of human life: Relation of brain weights to body heights and body weights. Ann. Neurol. 4(4), 345–356 (1978)

    Google Scholar 

  54. F. Dellaert, R.D. Beer, Co-evolving Body and Brain in Autonomous Agents Using a Developmental Model. Technical report, Department of Computer Engineering and Science (Case Western Reserve University, Cleveland, 1994)

    Google Scholar 

  55. F. Dellaert, R.D. Beer, Toward an evolvable model of development for autonomous agent synthesis. in Proceedings of the fourth International Workshop on the Synthesis and Simulation of Living Systems (ALIFE Workshop) (1994)

    Google Scholar 

  56. A. Deutsch, S. Dormann, Cellular Automaton Modelling of Biological Pattern Formation: Characterization, Applications and Analysis (Birkhauser, 2005)

    Google Scholar 

  57. A. Devert, N. Bredeche, M. Schoenauer, Robustness and the halting problem for multicellular artificial ontogeny. IEEE Trans. Evol. Comput. 15(3), 387–404 (2011)

    Google Scholar 

  58. M. do Carmo Nicoletti, J. Bertini, D. Elizondo, L. Franco, J. Jerez, Constructive neural network algorithms for feedforward architectures suitable for classification tasks. in ed. by L. Franco, D. Elizondo, J. Jerez. Constructive Neural Networks, Studies in Computational Intelligence, vol. 258 (Springer, Heidelberg, 2009), pp. 1–23

    Google Scholar 

  59. S. Doncieux, J.-B. Mouret, T. Pinville, P. Tonelli, B. Girard, The evoneuro approach to neuro-evolution. in Kowaliw et al. [159], pp. 10–14

    Google Scholar 

  60. R. Doursat, Bridging the mind-brain gap by morphogenetic neuron flocking: The dynamic self-organization of neural activity into mental shapes. in 2013 AAAI Fall Symposium Series (2013)

    Google Scholar 

  61. R. Doursat, Contribution à l’étude des représentations dans le système nerveux et dans les réseaux de neurones formels. PhD thesis, Université Pierre et Marie Curie (Paris 6), 1991

    Google Scholar 

  62. R. Doursat, Facilitating evolutionary innovation by developmental modularity and variability. in Conference on Genetic and Evolutionary Computation (GECCO) (ACM, 2009), pp. 683–690

    Google Scholar 

  63. R. Doursat, Organically grown architectures: creating decentralized, autonomous systems by embryomorphic engineering. in ed. by R.P. Würtz. Organic computing, Understanding Complex Systems (Springer, 2008), pp. 167–199

    Google Scholar 

  64. R. Doursat, The growing canvas of biological development: multiscale pattern generation on an expanding lattice of gene regulatory networks. InterJournal Complex Syst. 1809 (2006)

    Google Scholar 

  65. R. Doursat, E. Bienenstock, Neocortical self-structuration as a basis for learning. in 5th International Conference on Development and Learning (ICDL 2006) (2006), pp. 1–6

    Google Scholar 

  66. R. Doursat, C. Sánchez, R. Dordea, D. Fourquet, T. Kowaliw, Embryomorphic engineering: emergent innovation through evolutionary development. in ed. by Doursat et al. [67], pp. 275–311

    Google Scholar 

  67. R. Doursat, H. Sayama, O. Michel (eds.), Morphogenetic Engineering: Toward Programmable Complex Systems. Understanding Complex Systems (Springer, 2012)

    Google Scholar 

  68. R. Doursat, H. Sayama, O. Michel, A review of morphogenetic engineering. Nat. Comput. 1–19 (2013)

    Google Scholar 

  69. J.E. Dowling, The Great Brain Debate: Nature or Nurture? (Princeton University Press, 2007)

    Google Scholar 

  70. K. Downing, A neural-group basis for evolving and developing neural networks. in AAAI-Devp (2006)

    Google Scholar 

  71. K. Downing, Supplementing evolutionary developmental systems with abstract models of neurogenesis. in 9th Genetic and Evolutionary Computation Conference (GECCO) (2007), pp. 990–996

    Google Scholar 

  72. K. Downing, The Baldwin effect in developing neural networks. in Genetic and Evolutionary Computation Conference (GECCO) (2010), pp. 555–562

    Google Scholar 

  73. P. Durr, C. Mattiussi, D. Floreano, Neuroevolution with analog genetic encoding. in Parallel Problem Solving from Nature (PPSN) (2006), pp. 671–680

    Google Scholar 

  74. S.O.E. Ebbesson, The parcellation theory and its relation to interspecific variability in brain organization, evolutionary and ontogenetic development, and neuronal plasticity. Cell Tissue Res. 213(2), 179–212 (1980)

    Google Scholar 

  75. P. Eggenberger Hotz, Creation of neural networks based on developmental and evolutionary principles. in International Conference on Artificial, Neural Networks (1997), pp. 337–342

    Google Scholar 

  76. P. Eggenberger Hotz, Evolving morphologies of simulated 3D organisms based on differential gene expression. in European Conference on Artificial Life (ECAL) (MIT Press, 1997), pp. 205–213

    Google Scholar 

  77. P. Eggenberger Hotz, Evolving morphologies of simulated 3D organisms based on differential gene expression. in European Conference on Artificial Life (ECAL) (1997), pp. 205–213

    Google Scholar 

  78. P. Eggenberger Hotz, G. Gomez, R. Pfeiffer, Evolving the morphology of a neural network for controlling a foveating retina and its test on a real robot. in Artificial Life 8 (2002), pp. 243–251

    Google Scholar 

  79. A.E. Eiben, J.E. Smith, Introduction to Evolutionary Computing (Springer, 2003)

    Google Scholar 

  80. C. Eliasmith, T.C. Stewart, X. Choo, T. Bekolay, T. DeWolf, Y. Tang, D. Rasmussen, A large-scale model of the functioning brain. Science 338(20), 1202-1205 (2012)

    Google Scholar 

  81. D. Erhan, Y. Bengio, A. Courville, P.A. Manzagol, P. Vincent, S. Bengio, Why does unsupervised pre-training help dDeep learning? J. Mach. Learn. Res. 11, 625–660 (2010)

    Google Scholar 

  82. C. Espinosa-Soto, A. Wagner, Specialization can drive the evolution of modularity. PLoS Comput. Biol. 6(3), e1000719 (2010)

    MathSciNet  Google Scholar 

  83. S.E. Fahlman, C. Lebiere, The cascade-correlation learning architecture. in ed. by D.S. Touretzky. Advances in Neural Information Processing Systems 2, (Morgan Kaufmann, 1990), pp. 524–532

    Google Scholar 

  84. D. Federici, Evolving a neurocontroller through a process of embryogeny. in Proceeding of Simulation of Adaptive Behavior (SAB) (2004), pp. 373–384

    Google Scholar 

  85. D. Federici, Evolving developing spiking neural networks. in IEEE Congress on Evolutionary Computation (2005), pp. 43–550

    Google Scholar 

  86. D. Federici, K. Downing, Evolution and development of a multicellular organism: Scalability, resilience, and neutral complexification. Artif. Life 12(3), 381–409 (2006)

    Google Scholar 

  87. J.D. Fernández, D. Lobo, G.M. Martín, R. Doursat, F.J. Vico, Emergent diversity in an open-ended evolving virtual community. Artif. Life 18(2), 199–222 (2012)

    Google Scholar 

  88. D. Floreano, J. Urzelai, Neural morphogenesis, synaptic plasticity, and evolution. Theory Biosci. 120(3–4), 225–240 (2001)

    Google Scholar 

  89. J.A. Fodor, Modularity of Mind: An Essay on Faculty Psychology (MIT Press, 1983)

    Google Scholar 

  90. R.M. French, Catastrophic forgetting in connectionist networks. Trends Cogn. Sci. 3(4), 128–135 (1999)

    MathSciNet  Google Scholar 

  91. N. García-Pedrajas, D. Ortiz-Boyer, A cooperative constructive method for neural networks for pattern recognition. Pattern Recogn. 40(1), 80–98 (2007)

    MATH  Google Scholar 

  92. S. Geman, E. Bienenstock, R. Doursat, Neural networks and the bias/variance dilemma. Neural Comput. 4(1), 1–58 (1992)

    Google Scholar 

  93. S.F. Gilbert, Developmental Biology 8 edn. (Sinauer Associates, 2008)

    Google Scholar 

  94. S.F. Gilbert, D. Epel, Ecological Developmental Biology 1 edn. (Sinauer Associates, 2008)

    Google Scholar 

  95. B. Goertzel, R. Lian, I. Arel, H. de Garis, S. Chen, A world survey of artificial brain projects, part II: biologically inspired cognitive architectures. Neurocomputing 74(1–3), 30–49 (2010)

    Google Scholar 

  96. F. Gomez, R. Miikkulainen, Solving non-markovian control tasks with neuro-evolution. in IJCAI (1999), pp. 1356–1361

    Google Scholar 

  97. F. Gomez, R. Miikkulainen, Incremental evolution of complex general behavior. Adapt. Behav. 5, 317–342 (1997)

    Google Scholar 

  98. B.C. Goodwin, How the Leopard Changed Its Spots: The Evolution of Complexity (Scribner, 1994)

    Google Scholar 

  99. S.J. Gould, The Structure of Evolutionary Theory (The Belknap Press of Harvard University Press, 2002)

    Google Scholar 

  100. S.J. Gould, R. Lewontin, The spandrels of san marco and the panglossian paradigm: a critique of the adaptationist programme. Proc. Roy. Soc. London Ser. B Biol. Sci. 205(1161), 581–598 (1979)

    Google Scholar 

  101. F. Gruau, Cellular encoding as a graph grammar. in Grammatical Inference: IEE Colloquium on Theory, Applications and Alternatives (1993), pp. 1–17

    Google Scholar 

  102. F. Gruau, Genetic synthesis of Boolean neural networks with a cell rewriting developmental process. in Proceedings of COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks (IEEE Computer Society Press, 1992), pp. 55–74

    Google Scholar 

  103. F. Gruau, Neural Network Synthesis Using Cellular Encoding And The Genetic Algorithm (PhD thesis, Université Claude Bernard-Lyon, 1994)

    Google Scholar 

  104. F. Gruau, D. Whitley, L. Pyeatt, A comparison between cellular encoding and direct encoding for genetic neural networks. in Conference on Genetic Programming (1996), pp. 81–89

    Google Scholar 

  105. H.-G. Han, J.-F. Qiao, A structure optimization algorithm for feedforward neural network construction. Neurocomputing 99, 347–357 (2012)

    Google Scholar 

  106. H.-G. Han, J.-F. Qiao, A repair algorithm for radial basis function neural network and its application to chemical oxygen demand modeling. Int. J. Neural Syst. 20(01), 63–74 (2010)

    Google Scholar 

  107. S. Harding, W. Banzhaf, Artificial development. in Organic Computing, Understanding Complex Systems (Springer, Heidelberg, 2008), pp. 201–219

    Google Scholar 

  108. S.L. Harding, J.F. Miller, The dead state: A comparison between developmental and direct encodings (updated version). in Workshop on Complexity through Development and Self-Organizing Representations (CODESOAR), Genetic and Evolutionary Computation Conference (GECCO) (2006)

    Google Scholar 

  109. S.L. Harding, J.F. Miller, W. Banzhaf, Self-modifying cartesian genetic programming, in ed. by J.F. Miller Cartesian Genetic Programming, Natural Computing Series (Springer, Berlin, 2011), pp. 101–124

    Google Scholar 

  110. C. Hartland, N. Bredeche, M. Sebag, Memory-enhanced evolutionary robotics: the echo state network approach. in IEEE Congress on Evolutionary Computation, 2009 (CEC) (2009), pp. 2788–2795

    Google Scholar 

  111. B. Hassibi, D.G. Stork, Second order derivatives for network pruning: Optimal brain surgeon. in Advances in Neural Information Processing Systems (1993), pp. 164–164

    Google Scholar 

  112. J. Hastad, Almost optimal lower bounds for small depth circuits. in Proceedings of the Eighteenth Annual ACM Aymposium on Theory of Computing, STOC ’86 (ACM, New York, 1986), pp. 6–20

    Google Scholar 

  113. S. Haykin, Neural Networks and Learning Machines 3 edn. (Pearson Inc., 2009)

    Google Scholar 

  114. D.O. Hebb, The Organization of Behavior (Wiley, New York, 1949)

    Google Scholar 

  115. J.L. Hendrikse, T.E. Parsons, B. Hallgrímsson, Evolvability as the proper focus of evolutionary developmental biology. Evol. Dev. 9(4), 393–401 (2007)

    Google Scholar 

  116. S.L. Hill, Y. Wang, I. Riachi, F. Schürman, H. Markram, Statistical connectivity provides a sufficient foundation for specific functional connectivity in neocortical neural microcircuits. vol. 18 Proceedings of the National Academy of Sciences (2012)

    Google Scholar 

  117. J. Hiller, H. Lipson, Automatic design and manufacture of soft robots. IEEE Trans. Robot. 28, 457–466 (2012)

    Google Scholar 

  118. R. Himeno, J. Savin, Largest neuronal network simulation achieved using K computer @ONLINE. http://www.riken.jp/en/pr/press/2013/20130802_1/. Accessed: 09/2013

  119. G.E. Hinton, S. Osindero, Y.W. Teh, A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)

    Google Scholar 

  120. G.E. Hinton, S.J. Nowlan, How learning can guide evolution. Complex Syst. 1, 495–502 (1987)

    MATH  Google Scholar 

  121. A. Hintze, C. Adami, Evolution of complex modular biological networks. PLoS Comput. Biol. 4(2), 1–12 (2008)

    Google Scholar 

  122. T.-H. Hoang, R.I. McKay, D. Essam, N.X. Hoai, On synergistic interactions between evolution, development and layered learning. IEEE Trans. Evol. Comput. 15(3), 287–312 (2011)

    Google Scholar 

  123. J.J. Hopfield, C.D. Brody, What is a moment? transient synchrony as a collective mechanism for spatiotemporal integration. Proc. Natl. Acad. Sci. 98(3), 1282–1287 (2001)

    Google Scholar 

  124. G.S. Hornby, Measuring, enabling and comparing modularity, regularity and hierarchy in evolutionary design. in Conference on Genetic and Evolutionary Computation (GECCO) (2007), pp. 1729–1736

    Google Scholar 

  125. G.S. Hornby, H. Lipson, J.B. Pollack, Generative representations for the automated design of modular physical robots. IEEE Trans. Robot. Autom. 19(4), 703–719 (2003)

    Google Scholar 

  126. G.S. Hornby, J.B. Pollack, Creating high-level components with a generative representation for body-brain evolution. Artif. Life 8(3), 223–246 (2002)

    Google Scholar 

  127. P.E. Hotz, Comparing direct and developmental encoding schemes in artificial evolution: a case study in evolving lens shapes. in Congress on Evolutionary Computation (CEC) (2004), pp. 752–757

    Google Scholar 

  128. C.-F. Hsu, Adaptive growing-and-pruning neural network control for a linear piezoelectric ceramic motor. Eng. Appl. Artif. Intell. 21(8), 1153–1163 (2008)

    Google Scholar 

  129. T. Hu, W. Banzhaf, Evolvability and speed of evolutionary algorithms in light of recent developments in biology. J. Artif. Evol. Appl. 1–28, 2010 (2010)

    Google Scholar 

  130. D.-S. Huang, J.-X. Du, A constructive hybrid structure optimization methodology for radial basis probabilistic neural networks. IEEE Trans. Neural Netw. 19(12), 2099–2115 (2008)

    Google Scholar 

  131. A. Huemer, M. Gongora, D. Elizondo, A robust reinforcement based self constructing neural network. in International Joint Conference on Neural Networks (IJCNN) (2010), pp. 1–7

    Google Scholar 

  132. P. Husbands, T. Smith, N. Jakobi, M. O’Shea, Better living through chemistry: evolving GasNets for robot control. Connection Sci. 10(3–4), 185–210 (1998)

    Google Scholar 

  133. A. Ilachinski, Cellular Automata: A Discrete Universe (World Scientific, 2001)

    Google Scholar 

  134. B. Inden, Neuroevolution and complexifying genetic architectures for memory and control tasks. Theory Biosci. 127(2), 187–194 (2008)

    Google Scholar 

  135. T. Ishibashi, K. Dakin, B. Stevens, P. Lee, S. Kozlov, C. Stewart, R. Fields, Astrocytes promote myelination in response to electrical impulses. Neuron 49(6), 823–832 (2006)

    Google Scholar 

  136. M.M. Islam, A. Sattar, F. Amin, Xin Yao, K. Murase, A new adaptive merging and growing algorithm for designing artificial neural networks. IEEE Trans. Syst. Man Cyber. Part B Cybern. 39(3), 705–722 (2009)

    Google Scholar 

  137. H. Jäeger, H. Haas, Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)

    Google Scholar 

  138. H. Jäeger, M. Lukoševičius, D. Popovici, U. Siewert, Optimization and applications of echo state networks with leaky- integrator neurons. Neural Netw. 20(3), 335–352 (2007)

    Google Scholar 

  139. H. Jäeger, W. Maass, J. Principe, Introduction to the special issue on echo state networks and liquid state machines. Neural Netw. 20(3), 287–289 (2007)

    Google Scholar 

  140. N. Jakobi, Harnessing morphogenesis. in International Conference on Information Processing in Cells and Tissues (1995), pp. 29–41

    Google Scholar 

  141. K. Jarrett, K. Kavukcuoglu, M. Ranzato, Y. LeCun, What is the best multi-stage architecture for object recognition? in Proceedings of International Conference on Computer Vision (ICCV) (2009), pp. 2146–2153

    Google Scholar 

  142. M. Joachimczak, T. Kowaliw, R. Doursat, B. Wróbel, Brainless bodies: controlling the development and behavior of multicellular animats by gene regulation and diffusive signals. in Conference on the Simulation and Synthesis of Living Systems (ALife), (2012), pp. 349–356

    Google Scholar 

  143. M. Joachimczak, T. Kowaliw, R. Doursat, B. Wróbel, Controlling development and chemotaxis of soft-bodied multicellular animats with the same gene regulatory network. in Advances in Artificial Life (ECAL) (MIT Press, 2013), pp. 454–461

    Google Scholar 

  144. M. Joachimczak, B. Wróbel, Processing signals with evolving artificial gene regulatory networks. in Conference on the Simulation and Synthesis of Living Systems (ALife) (MIT Press, 2010), pp. 203–210

    Google Scholar 

  145. M. Joachimczak, B. Wróbel, Evolution of robustness to damage in artificial 3-dimensional development. Biosystems 109(3), 498–505 (2012)

    Google Scholar 

  146. M. Kaiser, C.C. Hilgetag, A. von Ooyen, A simple rule for axon outgrowth and synaptic competition generates realistic connection lengths and filling fractions. Cereb. Cortex 19(12), 3001–3010 (2009)

    Google Scholar 

  147. N. Kashtan, U. Alon, Spontaneous evolution of modularity and network motifs. Proc. Natl. Acad. Sci. 102(39), 13773 (2005)

    Google Scholar 

  148. Y. Kassahun, G. Sommer, Evolution of neural networks through incremental acquisition of neural structures. Technical Report Number 0508, Christian-Albrechts-Universität zu Kiel, Institut für Informatik und Praktische Mathematik, Juni 2005

    Google Scholar 

  149. M.J. Katz, R.J. Lasek, Evolution of the nervous system: Role of ontogenetic mechanisms in the evolution of matching populations. Proc. Natl. Acad. Sci. 75(3), 1349–1352 (1978)

    Google Scholar 

  150. S.A. Kauffman, The Origins of Order: Self Organization and Selection in Evolution (Oxford University Press, Oxford, 1993)

    Google Scholar 

  151. G.M. Khan, J.F. Miller, D.M. Halliday, Evolution of cartesian genetic programs for development of learning neural architecture. Evol. Comput. 19(3), 469–523 (2011)

    Google Scholar 

  152. M.W. Kirschner, J.C. Gerhart, The Plausibility of Life: Resolving Darwin’s Dilemma (Yale University Press, 2005)

    Google Scholar 

  153. H. Kitano, Designing neural networks using genetic algorithms with graph generation system. Complex Syst. 4, 461–476 (1990)

    MATH  Google Scholar 

  154. H. Kitano, A Simple Model of Neurogenesis and Cell Differentiation based on Evolutionary Large-Scale Chaos. Artif. Life 2, 79–99 (1995)

    Google Scholar 

  155. J. Kodjabachian, J.-A. Meyer, Evolution and development of neural networks controlling locomotion, gradient-following and obstacle avoidance in artificial insects. IEEE Trans. Neural Netw. 9(5), 796–812 (1998)

    Google Scholar 

  156. M. Komosinski, The world of framsticks: simulation, evolution, interaction. in Virtual Worlds (2000), pp. 214–224

    Google Scholar 

  157. T. Kowaliw, W. Banzhaf, Augmenting artificial development with local fitness. in ed. by A. Tyrrell IEEE Congress on Evolutionary Computation (CEC) (2009), pp. 316–323

    Google Scholar 

  158. T. Kowaliw, W. Banzhaf, Mechanisms for complex systems engineering through artificial development. in ed. by Doursat et al. [67], pp. 331–351

    Google Scholar 

  159. T. Kowaliw, N. Bredeche, R. Doursat (eds.), Growing Adaptive Machines: Combining Development and Learning in Artificial Neural Networks (Springer, 2014)

    Google Scholar 

  160. T. Kowaliw, N. Bredeche, R. Doursat (eds.), Proceedings of DevLeaNN: A Workshop on Development and Learning in Artificial Neural Networks (Paris, France, 2011)

    Google Scholar 

  161. T. Kowaliw, P. Grogono, N. Kharma, Bluenome: A novel developmental model of artificial morphogenesis. in Conference on Genetic and Evolutionary Computation (GECCO) (2004), pp. 93–104

    Google Scholar 

  162. T. Kowaliw, P. Grogono, N. Kharma, Environment as a spatial constraint on the growth of structural form. in Conference on Genetic and Evolutionary Computation (GECCO) (2007), pp. 1037–1044

    Google Scholar 

  163. T. Kowaliw, P. Grogono, N. Kharma, The evolution of structural form through artificial embryogeny. in IEEE Symposium on Artificial Life (ALIFE) (2007), pp. 425–432

    Google Scholar 

  164. J.R. Koza, D. Andre, F.H Bennett III, M. Keane, Genetic Programming 3: Darwinian Invention and Problem Solving (Morgan Kaufman, 1999)

    Google Scholar 

  165. J.L. Krichmar, G.M. Edelman, Brain-based devices for the study of nervous systems and the development of intelligent machines. Artif. Life 11(1–2), 63–77 (2005)

    Google Scholar 

  166. A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks. in ed. by P. Bartlett, F.C.N. Pereira, C.J.C. Burges, L. Bottou, K.Q. Weinberger Advances in Neural Information Processing Systems 25 (2012), pp. 1106–1114

    Google Scholar 

  167. P. Lauret, E. Fock, T.A. Mara, A node pruning algorithm based on a fourier amplitude sensitivity test method. IEEE Trans. Neural Netw. 17(2), 273–293 (2006)

    Google Scholar 

  168. A. Lazar, G. Pipa, J. Triesch, SORN: a self-organizing recurrent neural network. Front. Comput. Neurosci. 3(23), 1–9 (2009)

    Google Scholar 

  169. Q. Le, A. Karpenko, J. Ngiam, A.Y. Ng, Ica with reconstruction cost for efficient overcomplete feature learning. in ed. by J. Shawe-Taylor, R.S. Zemel, P. Bartlett, F.C.N. Pereira, K.Q. Weinberger Advances in Neural Information Processing Systems 24 (2011), pp. 1017–1025

    Google Scholar 

  170. Q. Le, M. Ranzato, R. Monga, M. Devin, K. Chen, G. Corrado, J. Dean, A. Ng, Building high-level features using large scale unsupervised learning, in ed. by J. Langford, J. Pineau Proceedings of the 29th International Conference on Machine Learning (ICML-12), ICML ’12 (Omnipress, New York, 2012), pp. 81–88

    Google Scholar 

  171. Y. LeCun, Y. Bengio, Convolutional networks for images, speech, and time series. in The Handbook of Brain Theory and Neural Networks (MIT Press, 1998)

    Google Scholar 

  172. J. Lefèvre, J.-F. Mangin, A reaction-diffusion model of human brain development. PLoS Comput. Biol. 6(4) e1000749 (2010)

    Google Scholar 

  173. M. Li, P.M.B. Vitányi, An Introduction to Kolmogorov Complexity and Its Applications 3rd edn. (Springer, 2008)

    Google Scholar 

  174. H. Lipson, Principles of modularity, regularity, and hierarchy for scalable systems. J. Biol. Phys. Chem. 7, 125–128 (2007)

    Google Scholar 

  175. J. Lohn, G. Hornby, D. Linden, Evolutionary antenna design for a NASA spacecraft. in Genetic Programming Theory and Practice II Chap. 18 (Springer, Ann Arbor, 2004), pp. 301–315

    Google Scholar 

  176. R.L. Lopes, E. Costa, The regulatory network computational device. Genetic Program. Evolvable Mach. 13, 339–375 (2012)

    Google Scholar 

  177. C.J. Lowe, G.A. Wray, Radical alterations in the roles of homeobox genes during echinoderm evolution. Nature 389, 718–721 (1997)

    Google Scholar 

  178. S. Luke, L. Spector, Evolving graphs and networks with edge encoding : preliminary report. in Late Breaking Papers at the Genetic Programming 1996 Conference (1996), pp. 117–124

    Google Scholar 

  179. M. Lukoševičius, H. Jaeger, Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev. 3(3), 127–149 (2009)

    Google Scholar 

  180. W. Maass, T. Natschläger, H. Markram, Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14(11), 2531–2560 (2002)

    MATH  Google Scholar 

  181. J.M. Mandler, The Foundations of Mind: Origins of Conceptual Thought (Oxford University Press, Oxford, 2004)

    Google Scholar 

  182. H. Markram, A brain in a supercomputer. www.ted.com. Accessed: 27/12/2012

    Google Scholar 

  183. H. Markram, J. Lübke, M. Frotscher, B. Sakmann, Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275(5297), 213–215 (1997)

    Google Scholar 

  184. H. Markram, The blue brain project. Nat. Rev. Neurosci. 7(2), 153–160 (2006)

    MathSciNet  Google Scholar 

  185. A. Matos, R. Suzuki, T. Arita, Heterochrony and artificial embryogeny: A method for analyzing artificial embryogenies based on developmental dynamics. Artif. Life 15(2), 131–160 (2009)

    Google Scholar 

  186. C. Mattiussi, D. Floreano, Analog genetic encoding for the evolution of circuits and networks. IEEE Trans. Evol. Comput. 11(5), 596–607 (2007)

    Google Scholar 

  187. J. McCormack, Aesthetic evolution of L-Systems revisited. in Applications of Evolutionary Computing (Evoworkshops) (2004), pp. 477–488

    Google Scholar 

  188. J. McCormack, Impossible Nature: the Art of Jon McCormack, Australian Centre for the Moving Image (2004)

    Google Scholar 

  189. W.S. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 114–133 (1943)

    Google Scholar 

  190. N.F. McPhee, E. Crane, S.E. Lahr, R. Poli, Developmental plasticity in linear genetic programming. in conference on Genetic and Evolutionary Computation (GECCO) (2009), pp. 1019–1026

    Google Scholar 

  191. T. Menezes, E. Costa, Artificial brains as networks of computational building blocks. in European Conference on Complex Systems (2008)

    Google Scholar 

  192. T. Menezes, E. Costa, The gridbrain: an heterogeneous network for open evolution in 3d environments. in IEEE Symposium on Artificial Life (2007), pp. 155–162

    Google Scholar 

  193. Y. Meng, Y. Zhang, Y. Jin, Autonomous self-reconfiguration of modular robots by evolving a hierarchical mechanochemical model. IEEE Comput. Intell. Mag. 6(1), 43–54 (2011)

    Google Scholar 

  194. Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs 1st–3rd edn. (Springer, New-York, 1992–1996)

    Google Scholar 

  195. Y. Miche, A. Sorjamaa, P. Bas, O. Simula, C. Jutten, A. Lendasse, Op-elm: Optimally pruned extreme learning machine. IEEE Trans. Neural Netw. 21(1), 158–162 (2010)

    Google Scholar 

  196. K.D. Micheva, B. Busse, N.C. Weiler, N. O’Rourke, S.J. Smith, Single-synapse analysis of a diverse synapse population: Proteomic imaging methods and markers. Neuron 68(4), 639–653 (2004)

    Google Scholar 

  197. J.F. Miller, Evolving a self-repairing, self-regulating, french flag organism. in Conference on Genetic and Evolutionary Computation (GECCO) (Springer, 2004), pp. 129–139

    Google Scholar 

  198. J.F. Miller, Neuro-centric and holocentric approaches to the evolution of developmental neural networks. in ed. by Kowaliw et al. [160], pp. 242–268

    Google Scholar 

  199. J.F. Miller, W. Banzhaf, Evolving the program for a cell:fFrom french flags to boolean circuits. in On Growth, Form and Computers (2003), pp. 278–301

    Google Scholar 

  200. J.F. Miller, P. Thomson, A developmental method for growing graphs and circuits. in Evolvable Systems: From Biology to Hardware (2003), pp. 93–104

    Google Scholar 

  201. J.F. Miller, G.M. Khan, Where is the brain inside the brain? Memetic Comput. 3, 217–228 (2011)

    Google Scholar 

  202. R. Milo, S. Itzkovitz, N. Kashtan, R. Levitt, S. Shen-Orr, I. Ayzenshtat, M. Sheffer, U. Alon, Superfamilies of evolved and designed networks. Science 303(5663), 1538–1542 (2004)

    Google Scholar 

  203. A.A. Minai, D. Braha, Y. Bar-Yam, Complex engineered systems: Science meets technology. in ed. by D. Braha, Y. Bar-Yam, A.A. Minai Complex Engineered Systems: Science Meets Technology, Chapter Complex Engineered Systems: A New Paradigm (Springer, 2006), pp. 1–21

    Google Scholar 

  204. D.E. Moriarty, Symbiotic Evolution of Neural Networks in Sequential Decision Tasks, Ph.D. Thesis (University of Texas at Austin, USA, 1998)

    Google Scholar 

  205. J.-B. Mouret, S. Doncieux, B. Girard, Importing the computational neuroscience toolbox into neuro-evolution-application to basal ganglia. in Conference on Genetic and Evolutionary Computation (GECCO) (2010), pp. 587–595

    Google Scholar 

  206. J.-B. Mouret, P. Tonelli, Artificial evolution of plastic neural networks: a few key concepts. en ed. by Kowaliw et al. [160], pp. 269–280

    Google Scholar 

  207. J.-B. Mouret, S. Doncieux, MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars. Evol. Intell. 1(3), 187–207 (2008)

    Google Scholar 

  208. T.D. Mrsic-Flogel, S.B. Hofer, K. Ohki, R.C. Reid, T. Bonhoeffer, M. Hbener, Homeostatic regulation of eye-specific responses in visual cortex during ocular dominance plasticity. Neuron 54, 961–972 (2007)

    Google Scholar 

  209. P.L. Narasimha, W.H. Delashmit, M.T. Manry, J. Li, F. Maldonado, An integrated growing-pruning method for feedforward network training. Neurocomputing 71(13–15), 2831–2847 (2008)

    Google Scholar 

  210. T. Natschläger, W. Maass, H. Markram, The “liquid computer”: A novel strategy for real-time computing on time series. Spec Issue Found. Inf. Proc. TELEMATIK 8, 39–43 (2002)

    Google Scholar 

  211. M.E.J. Newman, Modularity and community structure in networks. Proc. Natl. Acad. Sci. 103(23), 8577–8582 (2006)

    Google Scholar 

  212. S.A. Newman, G. Forgacs, G.B. Müller, Before programs: the physical origination of multicellular forms. Int. J. Dev. Biol. 50, 289–299 (2006)

    Google Scholar 

  213. S. Nichele, G. Tufte, Genome parameters as information to forecast emergent developmental behaviors. in ed. by J. Durand-Lose, N. Jonoska Unconventional Computation and Natural Computation (UCNC) (Springer, 2012), pp. 186–197

    Google Scholar 

  214. S. Nichele, G. Tufte, Trajectories and attractors as specification for the evolution of behaviour in cellular automata. in IEEE Congress on Evolutionary Computation (CEC) (2010), pp. 1–8

    Google Scholar 

  215. M. Nicolau, M. Schoenauer, W. Banzhaf, Evolving genes to balance a pole. in ed. by A. Esparcia-Alczar, A. Ekárt, S. Silva, S. Dignum, A. Uyar Genetic Programming, Lecture Notes in Computer Science, vol. 6021 (Springer, Berlin, 2010), pp. 196–207

    Google Scholar 

  216. A.B. Nielsen, L.K. Hansen, Structure learning by pruning in independent component analysis. Neurocomputing 71(10–12), 2281–2290 (2008)

    Google Scholar 

  217. K. Nigam, A.K. Mccallum, S. Thrun, T. Mitchell, Text classification from labeled and unlabeled documents using EM. Mach. Learn. 39(2–3), 103–134 (2000)

    MATH  Google Scholar 

  218. S. Nolfi, O. Miglino, D. Parisi, Phenotypic plasticity in evolving neural networks. in From Perception to Action (PerAc) (1994), pp. 146–157

    Google Scholar 

  219. S. Nolfi, D. Floreano, Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines (MIT Press/Bradford Books, Cambridge, 2000)

    Google Scholar 

  220. D. Norton, D. Ventura, Improving liquid state machines through iterative renement of the reservoir. Neurocomputing 73, 2893–2904 (2010)

    Google Scholar 

  221. D. Norton, D. Ventura, Preparing more effective liquid state machines using hebbian learning. in International Joint Conference on Neural Networks (IJCNN) (2006), pp. 8359–8364

    Google Scholar 

  222. B.A. Olshausen, D.J. Field, Sparse coding with an overcomplete basis set: a strategy employed by V1? Vis. Res. 37(23), 3311–3325 (1997)

    Google Scholar 

  223. T.H. Oong, N.A.M.M. Isa, Adaptive evolutionary artificial neural networks for pattern classification. IEEE Trans. Neural Netw. 22, 1823–1836 (2011)

    Google Scholar 

  224. C. Öztürkeri, M.S. Capcarrere, Self-repair ability of a toroidal and non-toroidal cellular developmental model. in European conference on Advances in Artificial Life (ECAL) (Springer, 2005), pp. 138–148

    Google Scholar 

  225. M.E. Palmer, Evolved neurogenesis and synaptogenesis for robotic control: the L-brain model. in Conference on Genetic and Evolutionary Computation (GECCO) (2011), pp. 1515–1522

    Google Scholar 

  226. H. Paugam-Moisy, R. Martinez, S. Bengio, Delay learning and polychronization for reservoir computing. Neurocomputing 71(7–9), 1143–1158 (2008)

    Google Scholar 

  227. R. Perin, T.K. Berger, H. Markram, A synaptic organizing principle for cortical neuronal groups. Proc. Natl. Acad. Sci. 108, 5419–5424 (2011)

    Google Scholar 

  228. R. Pfeifer, J. Bongard, How the Body Shapes the Way We Think: A New View of Intelligence (Bradford Books, 2006)

    Google Scholar 

  229. M. Pigliucci, Is evolvability evolvable? Nat. Rev. Genet. 9, 75–82 (2008)

    Google Scholar 

  230. D.J. Price, A.P. Jarman, J.O. Mason, P.C. Kind, Building brains: an introduction to neural development. 2nd edn. (Wiley, 2009)

    Google Scholar 

  231. P. Prusinkiewicz, A. Lindenmayer, The Algorithmic Beauty of Plants (Springer, 1990)

    Google Scholar 

  232. W.J. Puma-Villanueva, E.P. dos Santos, F.J. Von Zuben, A constructive algorithm to synthesize arbitrarily connected feedforward neural networks. Neurocomputing 75(1), 14–32 (2012)

    Google Scholar 

  233. Z.W. Pylyshyn, Is vision continuous with cognition? the case for cognitive impenetrability of visual perception. Behav. Brain Sci. 22, 341–423 (1999)

    Google Scholar 

  234. S.R. Quartz, T.J. Sejnowski, H. Hughes, The neural basis of cognitive development: a constructivist manifesto. Behav. Brain Sci. 20, 537–596 (1997)

    Google Scholar 

  235. R. Raina, A. Battle, H. Lee, B. Packer, A.Y. Ng, Self-taught learning: transfer learning from unlabeled data. in ICML ’07: Proceedings of the 24th International Conference on Machine Learning (ACM, New York, 2007), pp. 759–766

    Google Scholar 

  236. S. Rebecchi, H. Paugam-Moisy, M. Sebag, Learning sparse features with an auto-associator. in ed. by Kowaliw et al. [160], pp. 144–165

    Google Scholar 

  237. T. Reil, Dynamics of gene expression in an artificial genome—implications for biological and artificial ontogeny. in Proceedings of the 5th European Conference on Artificial Life (ECAL99), Number 1674 in Lecture Notes in Artificial Intelligence (1999), pp. 457–466

    Google Scholar 

  238. J. Reisinger, R. Miikkulainen, Acquiring evolvability through adaptive representations. in 8th Conference on Genetic and Evolutionary Computation (GECCO) (2007), pp. 1045–1052

    Google Scholar 

  239. J. Reisinger, R. Miikkulainen, Selecting for evolvable representations. in 7th Conference on Genetic and Evolutionary Computation (GECCO) (2006), pp. 1297–1304

    Google Scholar 

  240. J. Rieffel, D. Knox, S. Smith, B. Trimmer, Growing and evolving soft robots. Artif. Life 1–20 (2012)

    Google Scholar 

  241. J. Rieffel, J. Pollack, The emergence of ontogenic scaffolding in a stochastic development environment. in ed. by K. Deb Conference on Genetic and Evolutionary Computation (GECCO) of Lecture Notes in Computer Science, vol. 3102 (Springer, 2004), pp. 804–815

    Google Scholar 

  242. B. Roeschies, C. Igel, Structure optimization of reservoir networks. Logic J. IGPL 18(5), 635–669 (2010)

    Google Scholar 

  243. D. Roggen, D. Federici, Multi-cellular development: is there scalability and robustness to gain? in Parallel Problem Solving from Nature (PPSN) (2004), pp. 391–400

    Google Scholar 

  244. D. Roggen, D. Federici, D. Floreano, Evolutionary morphogenesis for multi-cellular systems. Genet. Program. Evol. Mach. 8(1), 61–96 (2006)

    Google Scholar 

  245. D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by back-propagating errors. Nature 323, 533–536 (1986)

    Google Scholar 

  246. R. Salakhutdinov, A. Mnih, G. Hinton, Restricted Boltzmann machines for collaborative filtering. in Proceedings of the 24th International Conference on Machine Learning, ICML ’07 (ACM, New York, 2007), pp. 791–798

    Google Scholar 

  247. K. Sano, H. Sayama, Wriggraph: a kinetic graph model that uniformly describes ontogeny and motility of artificial creatures. in Artificial life X: proceedings of the Tenth International Conference on the Simulation and Synthesis of Living Systems, vol. 10 (MIT Press, 2006), p. 77

    Google Scholar 

  248. L. Schramm, Y. Jin, B. Sendhoff, Redundancy creates opportunity in developmental representations. in IEEE Symposium on Artificial Life (IEEE-ALIFE)(2011)

    Google Scholar 

  249. L. Schramm, B. Sendhoff, An animat’s cell doctrine. in European Conference on Artificial Life (ECAL) (MIT Press, 2011), pp. 739–746

    Google Scholar 

  250. B. Schrauwen, M. Wardermann, D. Verstraeten, J.J. Steil, D. Stroobandt, Improving reservoirs using intrinsic plasticity. Neurocomputing 71(7–9), 1159–1171 (2008)

    Google Scholar 

  251. E.K. Scott, L.L. Luo, How do dendrites take their shape? Nat. Neurosci. 4(4), 359–365 (2001)

    Google Scholar 

  252. S.I. Sen, A.M. Day, Modelling trees and their interaction with the environment: A survey. Comput. Graph. 29(5), 805–817 (2005)

    Google Scholar 

  253. B. Sendhoff, E. Körner, O. Sporns, Creating brain-like intelligence. in ed. by Sendhoff et al. [255], pp. 1–14

    Google Scholar 

  254. B. Sendhoff, E. Körner, O. Sporns, H. Ritter, K. Doya (eds.), Creating Brain-Like Intelligence vol. 5436 (Springer, 2009)

    Google Scholar 

  255. S.H. Seung, Neuroscience: towards functional connectomics. Nature 471(7337), 170–172 (2011)

    Google Scholar 

  256. C.W. Seys, R.D. Beers, Genotype reuse more important than genotype size in evolvability of embodied neural networks. in 9th European Conference on Advances in Artificial Life (ECAL) (2007), pp. 915–924

    Google Scholar 

  257. S.K. Sharma, P. Chandra, An adaptive slope sigmoidal function cascading neural networks algorithm. in 2010 3rd International Conference on Emerging Trends in Engineering and Technology (ICETET) (2010), pp. 531–536

    Google Scholar 

  258. A.A. Siddiqi, S.M. Lucas, Comparison of matrix rewriting versus direct encoding for evolving neural networks. in IEEE International Conference on Evolutionary Computation, ICEC’98 (1998), pp. 392–397

    Google Scholar 

  259. M.S.M. Siddiqui, B. Bhaumik, Reaction-diffusion based model to develop binocular simple cells in visual cortex along with cortical maps. in International Joint Conference on Neural Networks (IJCNN) (2010), pp. 1–8

    Google Scholar 

  260. J. Šíma, P. Orponen, General-purpose computation with neural networks: a survey of complexity theoretic results. Neural Comput. 15(12), 2727–2778 (2003)

    MATH  Google Scholar 

  261. K. Sims, Evolving virtual creatures. in Proceedings of SIGGRAPH (1994), pp. 15–22

    Google Scholar 

  262. A. Soltoggio, P. Durr, C. Mattiussi, D. Floreano, Evolving neuromodulatory topologies for reinforcement learning-like problems. IEEE Congress on Evolutionary Computation (CEC) (2007), pp. 2471–2478

    Google Scholar 

  263. O. Sporns, From complex networks to intelligent systems. in ed. by Sendhoff et al. [255], pp. 15–30

    Google Scholar 

  264. K.O. Stanley, R. Miikkulainen, Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)

    Google Scholar 

  265. K.O. Stanley, R. Miikkulainen, A taxonomy for artificial embryogeny. Artif. Life 9(2), 93–130 (2003)

    Google Scholar 

  266. K.O. Stanley, D.B. D’Ambrosio, J. Gauci, A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15(2), 185–212 (2009)

    Google Scholar 

  267. T. Steiner, Y. Jin, B. Sendhoff, Vector field embryogeny. PLoS ONE 4(12), e8177 (2009)

    Google Scholar 

  268. G.F. Striedter, Principles of Brain Evolution (Sinauer Associates, Sunderland, 2005)

    Google Scholar 

  269. J.L. Subirats, L. Franco, J.M. Jerez, C-mantec: a novel constructive neural network algorithm incorporating competition between neurons. Neural Netw. 26, 130–140 (2012)

    Google Scholar 

  270. M. Suchorzewski, J. Clune, A novel generative encoding for evolving modular, regular and scalable networks. in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation - GECCO ’11 (2011), pp. 1523–2531

    Google Scholar 

  271. R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction (MIT Press, 1998)

    Google Scholar 

  272. H. Tanaka, L.T. Landmesser, Cell death of lumbosacral motoneurons in chick, quail, and chick-quail chimera embryos: a test of the quantitative matching hypothesis of neuronal cell death. J. Neurosci. 6(10), 2889–2899 (1986)

    Google Scholar 

  273. M.E. Taylor, S. Whiteson, P. Stone, Temporal difference and policy search methods for reinforcement learning: an empirical comparison. in Proceedings of the Twenty-Second Conference on Artificial Intelligence (AAAI-07) (2007)

    Google Scholar 

  274. G. Tesauro, Practical issues in temporal difference learning. Mach. Learn. 8(3), 257–277 (1992)

    Google Scholar 

  275. R. Thenius, M. Dauschanand, T. Schmickl, K. Crailsheim, Regenerative abilities in modular robots using virtual embryogenesis. in International Conference on Adaptive and Intelligent Systems (ICAIS) (2011), pp. 227–237

    Google Scholar 

  276. P. Tonelli, J.-B. Mouret, On the relationships between synaptic plasticity and generative systems. in Conference on Genetic and Evolutionary Computation (GECCO) (2011)

    Google Scholar 

  277. P. Tonelli, J.B. Mouret, On the relationshipd between generative encodings, regularity, and learning abilities when encoding plastic artificial neural networks. PLoS One 8(11), e79138 (2013)

    Google Scholar 

  278. T. Trappenberg, Fundamentals of Computational Neuroscience 2nd edn. (Oxford University Press, Oxford, 2009)

    Google Scholar 

  279. T. Trappenberg. A brief introduction to probabilistic machine learning and its relation to neuroscience. in ed. by Kowaliw et al. [160], pp. 62–110

    Google Scholar 

  280. G. Tufte, P.C. Haddow, Extending artificial development: exploiting environmental information for the achievement of phenotypic plasticity. in Conference on Evolvable Systems: from Biology to Hardware (ICES) (Springer, 2007), pp. 297–308

    Google Scholar 

  281. A. Turing, The chemical basis of morphogenesis. Philosop. Trans. Roy. Soc. B 237, 37–72 (1952)

    Google Scholar 

  282. M. Ulieru, R. Doursat, Emergent engineering: a radical paradigm shift. Int. J. Auton. Adap. Commun. Syst. 4(1), 39–60 (2011)

    Google Scholar 

  283. V. Valsalam, J.A. Bednar, R. Miikkulainen, Developing complex systems using evolved pattern generators. IEEE Trans. Evol. Comput. 11(2), 181–198 (2007)

    Google Scholar 

  284. P. Verbancsics, K.O. Stanley, Constraining connectivity to encourage modularity in HyperNEAT. in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (GECCO) (ACM Press, New York, 2011), pp. 1483–1490

    Google Scholar 

  285. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. (2010)

    Google Scholar 

  286. C. von der Malsburg, Synaptic plasticity as basis of brain organization. in ed. by J.P. Changeux, M. Konishi The Neural and Molecular Bases of Learning (Wiley, 1987), pp. 411–432

    Google Scholar 

  287. C. von der Malsburg, The correlation theory of brain function. in Models of Neural Networks II: Temporal Aspects of Coding and Information Processing in Biological Systems (Springer, 1981), pp. 95–119

    Google Scholar 

  288. C. von der Malsburg, E. Bienenstock, Statistical coding and short-term synaptic plasticity. in Disordered Systems and Biological Organization (Springer, 1986), pp. 247–272

    Google Scholar 

  289. G.P. Wagner, M. Pavlicev, J.M. Cheverud, The road to modularity. Nat. Rev. Genet. 8(12), 921–931 (2007)

    Google Scholar 

  290. V.J. Wedeen, D.L. Rosene, R. Wang, G. Dai, F. Mortazavi, P. Hagmann, J.H. Kaas, W.-Y.I. Tseng, The geometric structure of the brain fiber pathways. Science 335(6076), 1628–1634 (2012)

    Google Scholar 

  291. J. Weng, J. McClelland, A. Pentland, O. Sporns, I. Stockman, M. Sur, E. Thelen, Autonomous mental development by robots and animals. Science 291(5504), 599–600 (2001)

    Google Scholar 

  292. J. Weng, A computational introduction to the biological brain-mind. Nat. Intell. INNS Mag. 1(3), 5–16 (2012)

    Google Scholar 

  293. D.J. Willshaw, C. von der Malsburg, How patterned neural connections can be set up by self-organization. Proc. Roy. Soc. London Ser. B Biol. Sci. 194(1117), 431–445 (1976)

    Google Scholar 

  294. L. Wolpert, Developmental Biology (Oxford University Press, Oxford, 2011)

    Google Scholar 

  295. L. Wolpert, Positional information and the spatial pattern of cellular differentiation. J. Theor. Biol. 1, 1–47 (1969)

    Google Scholar 

  296. B. Wróbel, A. Abdelmotaleb, M. Joachimczak, Evolving spiking neural networks in the GReaNs (gene regulatory evolving artificial networks) plaftorm. in EvoNet2012: Evolving Networks, from Systems/Synthetic Biology to Computational Neuroscience Workshop at Artificial Life XIII (2012), pp. 19–22

    Google Scholar 

  297. B. Wróbel, M. Joachimczak, Using the GReaNs (genetic regulatory evolving artificial networks) platform for signal processing, animat control, and artificial multicellular development. in ed. by Kowaliw et al. [160], pp. 198–214

    Google Scholar 

  298. H. Yamada, T. Nakagaki, R.E. Baker, P.K. Maini, Dispersion relation in oscillatory reaction-diffusion systems with self-consistent flow in true slime mold. J. Math. Biol. 54(6), 745–760 (2007)

    MATH  MathSciNet  Google Scholar 

  299. S.-H. Yang, Y.-P. Chen, An evolutionary constructive and pruning algorithm for artificial neural networks and its prediction applications. Neurocomputing 86, 140–149 (2012)

    Google Scholar 

  300. Yann LeCun, J.S. Denker, S. Solla, R.E. Howard, L.D. Jackel, Optimal brain damage. in ed. by D. Touretzky NIPS’89 (Morgan Kaufman, 1990)

    Google Scholar 

  301. X. Yao, Y. Liu, A new evolutionary system for evolving artificial neural networks. IEEE Trans. Neural Netw. 8, 694–713 (1997)

    Google Scholar 

  302. X. Yao, Evolving neural networks. Proc. IEEE 87(9), 1423–1447 (1999)

    Google Scholar 

  303. I.B. Yildiz, H. Jaeger, S.J. Kiebel, Re-visiting the echo state property. Neural Netw. 35, 1–9 (2012)

    MATH  Google Scholar 

  304. J. Yin, Y. Meng, Y. Jin, A developmental approach to structural self-organization in reservoir computing. IEEE Trans. Auton. Ment. Dev. 4(4), 273–289 (2012)

    Google Scholar 

  305. T. Yu, J. Miller, Neutrality and the evolvability of boolean function landscape. in ed. by J. Miller, M. Tomassini, P.L. Lanzi, C. Ryan, A. Tettamanz, W.B. Langdon Genetic Programming (Springer, 2001), pp. 204–217

    Google Scholar 

  306. C. Yu, M.T. Manry, J. Li, An efficient hidden layer training method for multilayer perceptron. Neurocomputing 70(1–3), 525–535 (2006)

    Google Scholar 

  307. B. Zhang, D.J. Miller, Y. Wang, Nonlinear system modelling with random matrices: echo state networks revisited. IEEE Trans. Neural Netw. Learn. Syst. 23(1), 175–182 (2012)

    MathSciNet  Google Scholar 

  308. R. Zhang, Y. Lan, G.-B. Huang, Z.-B. Xu, Universal approximation of extreme learning machine with adaptive growth of hidden nodes. IEEE Trans. Neural Netw. Learn. Syst. 23(2), 365–371 (2012)

    Google Scholar 

  309. P. Zheng, C. Dimitrakakis, J. Triesch, Network self-organization explains the distribution of synaptic efficacies in neocortex. in ed. by Kowaliw et al. [159], pp. 8–9

    Google Scholar 

  310. N.E. Ziv, C.C. Garner, Principles of glutamatergic synapse formation: seeing the forest for the trees. Current Opin. Neurobiol. 11(5), 536–543 (2001)

    Google Scholar 

  311. F. Zubler, A. Hauri, S. Pfister, A.M. Whatley, M. Cook, R. Douglas, An instruction language for self-construction in the context of neural networks. Front. Comput. Neurosci. 5(57), 1–15 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut des Systèmes Complexes - Paris Île-de-France, CNRS, Paris, France

    Taras Kowaliw

  2. Sorbonne Universités, UPMC University Paris 06, UMR 7222 ISIR, F-75005, Paris, France

    Nicolas Bredeche

  3. CNRS, UMR 7222 ISIR, F-75005, Paris, France

    Nicolas Bredeche

  4. Versailles Systems Engineering Laboratory (LISV), University of Versailles, Velizy, France

    Sylvain Chevallier

  5. School of Biomedical Engineering, Drexel University, Philadelphia, USA

    René Doursat

Authors
  1. Taras Kowaliw
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicolas Bredeche
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sylvain Chevallier
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. René Doursat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Taras Kowaliw .

Editor information

Editors and Affiliations

  1. CNRS, Institut des Systèmes Complexes - Paris Île-de-France, Paris, France

    Taras Kowaliw

  2. Institute of Intelligent Systems and Robotics, CNRS UMR 7222, Université Pierre et Marie Curie, Paris, France

    Nicolas Bredeche

  3. School of Biomedical Engineering, Drexel University, Philadelphia, Pennsylvania, USA

    René Doursat

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kowaliw, T., Bredeche, N., Chevallier, S., Doursat, R. (2014). Artificial Neurogenesis: An Introduction and Selective Review. In: Kowaliw, T., Bredeche, N., Doursat, R. (eds) Growing Adaptive Machines. Studies in Computational Intelligence, vol 557. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-55337-0_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-55337-0_1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-55336-3

  • Online ISBN: 978-3-642-55337-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation