The Influence of Gaussian, Uniform, and Cauchy Perturbation Functions in the Neural Network Evolution

Abstract

Majority of algorithms in the field of evolutionary artificial neural networks (EvoANN) rely on the proper choice and implementation of the perturbation function to maintain their population’s diversity from generation to generation. Maintaining diversity is an important factor in the evolution process since it helps the population of ANN (Artificial Neural Networks) to escape local minima. To determine which among the perturbation functions are ideal for ANN evolution, this paper analyzed the influence of the three commonly used functions, namely: Gaussian, Cauchy, and Uniform. Statistical comparisons were conducted to examine their influence in the generalization and training performance of EvoANN. Our simulations using the glass classification problem indicated that for mutation-with-crossover-based EvoANN, generalization performance among the three perturbation functions were not significantly different. On the other hand, mutation-based EvoANN that used Gaussian mutation performed as good as that with crossover but it performed worst when it used either Uniform or Cauchy distribution function. These observations suggest that crossover operation becomes a significant operation in systems that employ strong perturbation functions but has less significance in systems that use weak or conservative perturbation functions.