Advertisement

Application of Genetic Algorithms to the Construction of Topologies for Multilayer Perceptrons

  • W. Schiffmann
  • M. Joost
  • R. Werner

Abstract

In this paper we present a new approach for automatic topology optimization of backpropagation networks. It is based on a genetic algorithm. In contrast to other approaches it allows that two networks with different number of units can be crossed to a new valid “child” network. We applied this algorithm to a medical classification task, which is extremely difficult to solve. The results confirm, that optimization make sence, because the generated network outperform all fixed topologies.

Keywords

Neural Network Genetic Algorithm Topology Optimization Network Architecture Genetic Operator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Minsky and Papert, Perceptrons, MIT Press, 1969Google Scholar
  2. [2]
    Rumelhart D.E., Hinton G.E. and Williams R.J., Learning internal representations by error propagation, in Parallel Distributed Processing: Explorations in the Microstructures of Cognition, Vol.I, MIT Press, pp. 318–362, 1986Google Scholar
  3. [3]
    Hornik K., Stinchcombe M. and White H., Multilayer Feedforward Networks are Universal Approximators, Neural Networks, Vol. 2, pp. 359–366, Pergamon Press, 1989CrossRefGoogle Scholar
  4. [4]
    Miller G., Todd P.M. and Hegde S.U., Designing Neural Networks Using Genetic Algorithms, Proc. of the third Intern. Conf. on Genetic Algorithms (ICGA), pp. 379-384, San Mateo (CA), 1989Google Scholar
  5. [5]
    Schiffmann W.H. and Mecklenburg K., Genetic Generation of Backpropagation Trained Neural Networks, Proc. of Parallel Processing in Neural Systems and Computers (ICNC), Eckmiller R. et al. (Eds.), pp. 205-208, Elsevier, 1990Google Scholar
  6. [6]
    Schiffmann W.H., Selbstorganisation neuronaler Netze nach den Prinzipien der Evolution, Fachbericht 7/1989, Universität KoblenzGoogle Scholar
  7. [7]
    Schiffmann W.H., Joost M. and Werner R., Performance Evaluation of Evolutionarily Created Neural Network Topologies, Proc. of Parallel Problem Solving from Nature, Schwefel H.P. and Maenner R. (Eds.), pp. 274-283, Lect. Notes in Computer Science, Springer, 1991Google Scholar
  8. [8]
    Whitley D., Applying Genetic Algorithms to Neural Net Problems, Neural Networks, Vol. 1, p. 230, 1988CrossRefGoogle Scholar
  9. [9]
    Whitley D. and Hanson T., Optimizing Neural Networks Using Faster, More Accurate Genetic Search, Proc. of the third Intern. Conf. on Genetic Algorithms (ICGA), pp. 391-396, San Mateo (CA), 1989Google Scholar
  10. [10]
    Whitley D. and Bogart C., The Evolution of Connectivity: Pruning Neural Networks Using Genetic Algorithms, Proc. of the intern. Joint Conf. on Neural Networks, Vol. I, pp. 134–137, 1990Google Scholar
  11. [11]
    Whitley D., Starkweather and Bogart C., Genetic Algorithms and Neural Networks: Optimizing Connections and Connectivity, Parallel Computing Vol. 14, pp. 347–361, Elsevier, 1990CrossRefGoogle Scholar
  12. [12]
    Whitley D., The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best, Proc. of the third Intern. Conf. on Genetic Algorithms (ICGA), pp. 116-121, San Mateo (CA), 1989Google Scholar
  13. [13]
    Heistermann J., Parallel Algorithms for Learning in Neural Networks with Evolution Strategy, Parallel Computing, Vol. 12, 1989Google Scholar
  14. [14]
    Heistermann J., Learning in Neural Nets by Genetic Algorithms, Proc. of Parallel Processing in Neural Systems and Computers (ICNC), Eckmiller R. et al. (Eds.), pp. 165-168, Elsevier, 1990Google Scholar
  15. [15]
    Montana D. and Davis C., Training Feedforward Neuronal Networks Using Genetic Algorithms, Technical Report, BBN Systems and Technologies Inc., Cambridge (MA), 1989Google Scholar
  16. [16]
    de Garis H., Genetic Programming — Modular Neural Evolution for Darwin Machines, Proc. of the Intern. Joint Conf. on Neural Networks, Vol I, pp. 194–197, 1990Google Scholar
  17. [17]
    Weiss, G., Combining neural and evolutionary learning: Aspects and approaches, Report FKI-132-90, Technische Universität München, 1990Google Scholar
  18. [18]
    Kitano H., Empirical Studies on the Speed of Convergence of Neural Network Training Using Genetic Algorithms, Proc. of the National Conf. of the American Association of Artificial Intelligence (AAAI), pp. 789-795, 1990Google Scholar
  19. [19]
    Dodd N., Optimization of Network Structure Using Genetic Techniques, Proc. of the Intern. Conf. on Neural Networks, pp. 693-696, Paris, 1990Google Scholar
  20. [20]
    Lehar S. and Weaver J., A Developmental Approach to Neural Network Design, Proc. of the IEEE Intern. Conf. on Neural Networks, Vol. I, pp. 97–104, 1987Google Scholar
  21. [21]
    Harp S.A., Samad T. and Guha A., Towards the Genetic Synthesis of Neural Networks, Proc. of the third Intern. Conf. on Genetic Algorithms (ICGA), pp. 360-369, San Mateo (CA), 1989Google Scholar
  22. [22]
    Harp S.A., Samad T. and Guha A., The Genetic Synthesis of Neural Networks, Technical Report CSDD-89-14852-2, Honeywell, Golden Valley (MN), 1989Google Scholar
  23. [23]
    Merrill J. and Port R., Fractally Configured Neural Networks, Neural Networks, Vol. 4, pp.53–60, 1991CrossRefGoogle Scholar
  24. [24]
    Mjolsness E. and Sharp D.H., A Prelimiary Analysis of Recursively Generated Networks, in Denker J. (Eds.): Neural Networks for Computing, Snowbird (Utah), 1986Google Scholar
  25. [25]
    Mjolsness E., Sharp D.H. and Alpert B.K., Recursively Generated Neural Networks, Proc. of the IEEE Intern. Conf. on Neural Networks, Vol. III, pp. 165–171, 1987Google Scholar
  26. [26]
    Mühlenbein H. and Kindermann J., The Dynamics of Evolution and Learning — Towards Genetic Neural Networks, in Pfeifer et al. (Eds.): Connectionism in Perspective, Elsevier, 1989Google Scholar
  27. [27]
    Schiffmann W.H., Joost M. and Werner R. Optimization of the Backpropagation Algorithm for Training Multilayer Perceptrons, Technical Report 15/1992, University of KoblenzGoogle Scholar
  28. [28]
    Schiffmann W.H., Joost M. and Werner R. Synthesis and Performance Analysis of Multilayer Neural Network Architectures, Technical Report 16/1992, University of KoblenzGoogle Scholar
  29. [29]
    Fahlman S.E., An Empirical Study of Learning Speed in Back-Propagation Networks, Technical Report, Carnegie-Mellon University, CMU-CS-88-162, 1988Google Scholar

Copyright information

© Springer-Verlag/Wien 1993

Authors and Affiliations

  • W. Schiffmann
    • 1
  • M. Joost
    • 1
  • R. Werner
    • 1
  1. 1.Institute of PhysicsUniversity of KoblenzKoblenzGermany

Personalised recommendations