Advertisement

Evolving Neural Feedforward Networks

  • Heinrich Braun
  • Joachim Weisbrod

Abstract

For many practical problem domains the use of neural networks has led to very satisfactory results. Nevertheless the choice of an appropriate, problem specific network architecture still remains a very poorly understood task. Given an actual problem, one can choose a few different architectures, train the chosen architectures a few times and finally select the architecture with the best behaviour. But, of course, there may exist totally different and much more suited topologies. In this paper we present a genetic algorithm driven network generator that evolves neural feedforward network architectures for specific problems. Our system ENZO1 optimizes both the network topology and the connection weights at the same time, thereby saving an order of magnitude in necessary learning time. Together with our new concept to solve the crucial neural network problem of permuted internal representations this approach provides an efficient and successfull crossover operator. This makes ENZO very appropriate to manage the large networks needed in application oriented domains. In experiments with three different applications our system generated very successful networks. The generated topologies possess distinct improvements referring to network size, learning time, and generalization ability.

Keywords

Genetic Algorithm Crossover Operator Generalization Ability Training Epoch Digit Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    D. Nguyen and B. Widrow, The truck backerupper, Proc. Internat. Network Conf. (Kluwer Academic Publishers, Dordrecht, 1990)Google Scholar
  2. [2]
    H. Braun, J. Feulner, V. Ullrich, Learning strategies for solving the problem of planning using backpropagation, in: Proc. Fourth Intern. Conf. Neural Networks (Nimes, 1991)Google Scholar
  3. [3]
    R.K. Belew, J. McInerney, N.N. Schraudolph, Evolving networks: using the genetic algorithm with connectionist learning, CSE Technical Report #CS90-174 (University of California, San Diego)Google Scholar
  4. [4]
    D.J. Chalmers, The evolution of learning: an experiment in genetic connectionism, in: Proc. of the 1990 Connectionist Models Summer School (Morgan-Kaufmann, San Mateo, CA, 1990)Google Scholar
  5. [5]
    D.B. Fogel, L.J. Fogel, V.W. Porto, Evolving neural networks, in: Biological Cybernetics 63 (Springer, Berlin, 1990)Google Scholar
  6. [6]
    P.J.B. Hancock, L.S. Smith, GANNET: Genetic design of a neural net for face recognition, in: Parallel Problem Solving from Nature (Springer, Berlin, 1990)Google Scholar
  7. [7]
    S. Harp, T. Samad, A. Guha, Towards the genetic synthesis of neural networks, in: Proc. Third Internat. Conf. Genetic Algorithms (Morgan Kaufmann, San Mateo, CA, 1989)Google Scholar
  8. [8]
    K.U. Hoeffgen, H.P. Siemon, A. Ultsch, Genetic improvements of feedforward nets for approximating functions, in: Parallel Problem Solving from Nature (Springer, Berlin, 1990)Google Scholar
  9. [9]
    G. Miller, P. Todd, S. Hedge, Designing neural networks using genetic algorithms, in: Proc. Third Internat. Conf. Genetic Algorithms (Morgan Kaufmann, San Mateo, CA, 1989)Google Scholar
  10. [10]
    H. Muehlenbein, Limitations of multi-layer perceptron networks — steps towards genetic neural networks, in: Parallel Computing 14 (1990)Google Scholar
  11. [11]
    W. Schiffmann, M. Joost, R. Werner, Performance evaluation of evolutionary created neural network topologies, in: Parallel Problem Solving from Nature (Springer, Berlin, 1990)Google Scholar
  12. [12]
    M. Scholz, A learning strategy for neural networks based on a modified evolutionary strategy, in: Parallel Problem Solving from Nature (Springer, Berlin, 1990)Google Scholar
  13. [13]
    D. Whitley, T. Hanson, Optimizing neural networks using faster, more accurate genetic search, in: Proc. Third Internat. Conf. Genetic Algorithms (Morgan Kaufmann, San Mateo, CA, 1989)Google Scholar
  14. [14]
    D. Whitley, T. Starkweather, C. Bogart, Genetic algorithms and neural networks: optimizing connections and connectivity, in: Parallel Computing 14 (1990)Google Scholar
  15. [15]
    D. Whitley, S. Dominic, R. Das, Genetic reinforcement learning with multilayer neural networks, in: Proc. Fourth Internat. Conf. Genetic Algorithms (Morgan Kaufmann, San Mateo, CA, 1991)Google Scholar
  16. [16]
    J.H. Holland, Adaption in natural and artificial systems, (Ann Arbor: University of Michigan Press, 1975)MATHGoogle Scholar
  17. [17]
    H. Braun, Massiv parallele Algorithmen fuer kombinatorische Optimierungsprobleme und ihre Implementierung auf einem Parallelrechner, Dissertation TH Karlsruhe, Fakultaet fuer Informatik, 1990Google Scholar
  18. [18]
    H. Braun, On solving traveling salesman problems by genetic algor ithms, in: Parallel Problem Solving from Nature, LNCS 496 (Berlin, 1991)Google Scholar
  19. [19]
    J. Weisbrod, Einsatz Genetischer Algorithmen zur Optimierung der Topologie mehrschichtiger Feedforward-Netzwerke, Diplomarbeit TH Karlsruhe, Fakultät für Informatik (1992)Google Scholar
  20. [20]
    J. Weisbrod, Untersuchung der Einsatzmöglichkeiten Neuronaler Netze zur visuellen Mustererkennung gestanzter Ziffern, Studienarbeit TH Karlsruhe, Fakultät für Elektrotechnik (1989)Google Scholar
  21. [21]
    V. Ullrich, Erlernen von Spielstrategien für Mühle durch Neuronale Netze, Diplomarbeit TH Karlsruhe, Fakultät für Informatik (1991)Google Scholar

Copyright information

© Springer-Verlag/Wien 1993

Authors and Affiliations

  • Heinrich Braun
    • 1
  • Joachim Weisbrod
    • 2
  1. 1.Institut für Logik, Komplexität und DeduktionssystemeUniversität KarlsruheGermany
  2. 2.Institut für Programmstrukturen und DatenorganisationUniversität KarlsruheGermany

Personalised recommendations