Evolving Neural Network Structures: An Evaluation of Encoding Techniques

  • Stephen G. Roberts
  • Mike Turega


Feed-forward neural network structures, trained with back-propagation, are adapted by the use of the genetic algorithm (GA). Through this search technique problem specific topologies are found. The method used to represent network structure, in a form suitable for the GA, is investigated. Comparisons are made between three such encoding methods. Details are given of how these representational schemes can influence the performance of both the genetic algorithm and the resultant neural networks found by the GA.


Genetic Algorithm Encode Scheme Hide Node Encode Technique Neural Network Structure 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hornik, K., Stinchcombe, M., White, H.; Multi-layer feedforward networks are universal approximators.; Neural Networks, 2, pp 359–68; (1989)Google Scholar
  2. 2.
    Miller, G.F., Todd, P.M., Hegde, S.U.; Designing neural networks using genetic algorithms.; In 3rd Int. Conf. Genetic Algorithms, Morgan Kaufmann, (1989)Google Scholar
  3. 3.
    Holland, J.H.; Adaptation in natural and artificial systems.; Ann Arbor ( MI ): University of Michigan Press, (1975)Google Scholar
  4. 4.
    Goldberg, D.; Genetic algorithms in search, optimisation and machine learning; Addison-Wesley (1988)Google Scholar
  5. 5.
    Jones, AJ.; Genetic Algorithms and their Application to the Design of Neural Networks; Neural Computing & Applications, 1, pp 32–45; (1993)Google Scholar
  6. 6.
    Kitano, H.; Designing neural networks using genetic algorithms with graph generation system.; Complex Systems, (1990)Google Scholar
  7. 7.
    Whitley, D., Starkweather, T., Bogart, C.; Genetic algorithms and neural networks: Optimising connections and connectivity.; Parallel Computing, Vol. 14,pp. 347–361, (1990)Google Scholar
  8. 8.
    Maniezzo, V.; Genetic evolution of the topology and weight distribution of neural networks.; IEEE Transactions on Neural Networks, Vol. 5, No.1, January (1994)Google Scholar
  9. 9.
    Robbins, G.E., Plumbley, M.D., Hughes, J. C., Fallside, F., Prager, R.; Generation and adaptation of neural networks by evolutionary techniques; Neural Computing and Applications, Vol. 1, pp 23–31, (1993)Google Scholar
  10. 10.
    Braun, H., Weisbrod, J.; Evolving Neural Feedforward Networks; Int. Conf. on Neural Networks & Genetic Algorithms, GA-ANN93, Heidelberg: Springer-Verlag, pp 25–32; (1993)Google Scholar
  11. 11.
    Harp, S.A., Samad, T.; The Genetic Synthesis of Neural Network Architecture.; Ch.15, in Davis, L.; Genetic Algorithms and Simulated Annealing. Pitman, London, (1987)Google Scholar
  12. 12.
    Harp, S.A., Samad, T., Guha, A.; Designing application specific neural networks using the genetic algorithm.; In Neural Information Processing Systems - Neural and Synthetic, pp. 447–454. Morgan Kaufmann, (1989)Google Scholar
  13. 13.
    Harp, S.A., Samad, T., Guha, A.; Towards the Genetic Synthesis of Neural Networks; in 3rd Int. Conf. on Genetic Algorithms, Morgan Kaufmann, (1989)Google Scholar
  14. 14.
    Schaffer, J.D., Caruana, R.A., Eshelman, L.J.; Using genetic search to exploit the emergent behaviour of neural networks. Physica D, Vol. 42, pp. 244–248, (1990)CrossRefGoogle Scholar
  15. 15.
    Boers, E.J. W., Kuiper, H.; Biological metaphors and the design of modular artificial neural networks.; Masters Thesis, Dept. of Computer Science & Experimental & Theoretical Psychology, Leiden University, Holland, (1992)Google Scholar
  16. 16.
    Merrill, J. W.L., Port, R. F.; Fractally configured neural networks; Neural Networks, 4, pp 53–60; (1991)Google Scholar
  17. 17.
    Lindenmayer, A.; Mathematical models for cellular interaction in development, parts I and II.; in Journal of theoretical biology, 18,pp. 280–315, (1968)Google Scholar
  18. 18.
    Prusinkiewicz, P., Lindenmayer, A.; The algorithmic beauty of plants.; Springer-Verlag, New York, (1990)MATHCrossRefGoogle Scholar
  19. 19.
    Rumelhart, D., McClelland, J.; Parallel distributed processing: Explorations in the microstructure of cognition; Vol. 1, MIT Press, Cambridge (MA ) (1986)Google Scholar
  20. 20.
    Denker J., Schwartz, D., Wittner, B., Solla, S., Howard, R., Jackel, L., Hopfield J., Large Automatic Learning, Rule Extraction, and Generalsation. Complex Systems 1, pp 877–922 (1987)MathSciNetMATHGoogle Scholar

Copyright information

© Springer-Verlag/Wien 1995

Authors and Affiliations

  • Stephen G. Roberts
    • 1
  • Mike Turega
    • 1
  1. 1.Department of ComputationUMISTManchesterUK

Personalised recommendations