Searching among Search Spaces: Hastening the genetic evolution of feedforward neural networks
The paper introduces a genetic paradigm for evolving feedforward neural network, where both the network topology and the weights distribution are coded in the individuals. The resulting binary coded strings are too long for efficient evolution, thus two novel techniques are employed. The first consists in a coding procedure that allows the genetic algorithm to evolve the length of the coding string along with its content. This goes by the name of granularity evolution procedure. The second technique is inspired by linear programming and yields a fast and accurate fine tuning of the solutions.
These ideas have been implemented in a running system. Computational results show how during evolution the genetic algorithm uses several coding lengths, thus it autonomously identifies the search spaces that lead to more promising solutions.
KeywordsGenetic Algorithm Search Space Feedforward Neural Network Mutation Probability Weight Code
Unable to display preview. Download preview PDF.
- Rumelhart D.E., Hinton G.E. and Williams R.J., “Learning internal representations by error propagation”, in Parallel Distributed Processing: Explorations in the microstructures of cognition, D.E. Rumelhart, J.L. McLelland Eds, Cambridge: MIT Press, 1986, 318–362.Google Scholar
- Kitano H., ‘Empirical studies on the speed of convergence of neural network training using genetic algorithms’, in Proc. of the eigth nat. conf. on AI (AAAI-90), 1990, 789-795.Google Scholar
- Whitley D. and Hanson T., ‘Optimizing neural networks using faster, more accurate genetic search’, Proc. of the Third Int. Conf. on Genetic Algorithms and their Applications, 1989, 391-396.Google Scholar
- Parisi D., Cecconi F. and Nolfi S., ‘Econets: neural networks that learn in an environment’, Network, 1, 1990, 149–168.Google Scholar
- Maniezzo V., ‘Anna Eleonora: Genetic Evolution of Feedforward Neural Networks’, Technical Report No. 93-003, Politecnico di Milano, Italy, 1993.Google Scholar
- Bersini H. and Seront G., ‘In search of a good crossover between evoltion and optimization’, in Parallel Problem Solving from Nature 2, B. Manderick, R. Männer Eds., Amsterdam: Elsevier, 1992, 479-488Google Scholar
- Back T., Hoffmeister F. and Schwefel H.P., ‘A Survey of Evolution Strategies’, Proc. of the fourth int. conf. on Genetic Algorithms, 1991.Google Scholar
- Goldberg D.E., Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley, INDOA, 1989.Google Scholar
- Hancock P.J.B., ‘Recombination operators for the design of neural nets by genetic algorithm’, in R. Manner, B. Manderick (eds.) Parallel Problem solving from Nature, 2, Amsterdam: Elsevier, 1992, 441–450.Google Scholar
- Mühlenbein H., ‘Parallel Genetic Algorithms, Population Genetics and Combinatorial Optimization’, Proc. of the third int. conf. on Genetic Algorithms, 1989, 416-421.Google Scholar
- Colorni A., Dorigo M. and Maniezzo V., ‘Algodesk: an experimental comparison of eight evolutionary heuristics applied to the QAP problem’, Technical Report No. 92-052, Politecnico di Milano, Italy, 1992.Google Scholar
- de Jong K.A., Analysis of the behavior of a class of genetic adaptive systems, Ph.D. dissertation, Univ. of Michigan, 1975.Google Scholar