Comparing Evolutionary Artificial Neural Networks from Second and Third Generations for Solving Supervised Classification Problems

  • G. López-Vázquez
  • A. Espinal
  • Manuel Ornelas-RodríguezEmail author
  • J. A. Soria-Alcaraz
  • A. Rojas-Domínguez
  • Héctor Puga
  • J. Martín Carpio
  • H. Rostro-González
Part of the Studies in Computational Intelligence book series (SCI, volume 862)


Constituting nature-inspired computational systems, Artificial Neural Networks (ANNs) are generally classified into several generations depending on the features and capabilities of their neuron models. As generations develop, newer models of ANNs portrait more plausible properties than their predecessors, accounting for closer resemblance to biological neurons or for augmentations in their problem-solving abilities. Evolutionary Artificial Neural Networks (EANNs) is a paradigm to design ANNs involving Evolutionary Algorithms (EAs) to determine inherent aspects of the networks such as topology or parameterization, while prescinding—totally or partially—from expert proficiency. In this paper a comparison of the performance of evolutionary-designed ANNs from the second and third generations is made. An EA-based technique known as Grammatical Evolution (GE) is used to automatically design ANNs for solving supervised classification problems. Partially-connected three-layered feedforward topologies and synaptic connections for both types of considered ANNs are determined by the evolutionary process of GE; an explicit training task is not necessary. The proposed framework was tested on several well-known benchmark datasets, providing relevant and consistent results; accuracies exhibited by third-generation ANNs matched or bested those from second-generation ANNs. Furthermore, produced networks achieved a considerable reduction in the amount of existing synapses, as in comparison with equivalent fully-connected topologies, and a lower usage of traits from the input vector.


Artificial neural networks Grammatical evolution Evolutionary artificial neural networks 



Authors wish to thank National Technology of Mexico and University of Guanajuato. G. López-Vázquez and A. Rojas-Domínguez thank to the National Council of Science and Technology of Mexico (CONACYT) for the support provided by means of the Scholarship for Postgraduate Studies (701071) and Research Grant (CÁTEDRAS-2598), respectively. This work was supported by the CONACYT Project FC2016-1961 “Neurociencia Computacional: de la teoría al desarrollo de sistemas neuromórficos”.


  1. 1.
    Amaldi, E., Mayoraz, E., de Werra, D.: A review of combinatorial problems arising in feedforward neural network design. Discrete Appl. Math. 52(2), 111–138 (1994)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Belatreche, A., Maguire, L.P., Mcginnity, M., Wu, Q.X.: An evolutionary strategy for supervised training of biologically plausible neural networks. In: Proceedings of the Sixth International Conference on Computational Intelligence and Natural Computing, pp. 1524–1527 (2003)Google Scholar
  3. 3.
    Belatreche, A., Maguire, L.P., McGinnity, T.M.: Advances in design and application of spiking neural networks. Soft Comput. 11(3), 239–248 (2007)CrossRefGoogle Scholar
  4. 4.
    Dheeru, D., Karra Taniskidou, E.: UCI Machine Learning Repository (2017)Google Scholar
  5. 5.
    Ding, S., Li, H., Su, C., Yu, J., Jin, F.: Evolutionary artificial neural networks: a review. Artif. Intell. Rev. 39(3), 251–260 (2013)CrossRefGoogle Scholar
  6. 6.
    Elfwing, S., Uchibe, E., Doya, K.: Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Netw. (2018)Google Scholar
  7. 7.
    Elizondo, D., Fiesler, E.: A survey of partially connected neural networks. Int. J. Neural Syst. 8(5–6), 535–558 (1997)CrossRefGoogle Scholar
  8. 8.
    Espinal, A., Carpio, M., Ornelas, M., Puga, H., Melin, P., Sotelo-Figueroa, M.: Comparing metaheuristic algorithms on the training process of spiking neural networks. In: Recent Advances on Hybrid Approaches for Designing Intelligent Systems, pp. 391–403. Springer (2014)Google Scholar
  9. 9.
    Espinal, A., Carpio, M., Ornelas, M., Puga, H., Melín, P., Sotelo-Figueroa, M.: Developing architectures of spiking neural networks by using grammatical evolution based on evolutionary strategy. In: Mexican Conference on Pattern Recognition, pp. 71–80. Springer (2014)Google Scholar
  10. 10.
    Gerstner, W., Kistler, W.: Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press (2002)Google Scholar
  11. 11.
    Ghosh-Dastidar, S., Adeli, H.: Spiking neural networks. Int. J. Neural Syst. 19(04), 295–308 (2009)CrossRefGoogle Scholar
  12. 12.
    Gnedenko, B.V., Kolmogorov, A.N.: Limit distributions for sums of independent random variables. In: Predelnye raspredeleniia dlia summ, No. ix, 264 p. Addison-Wesley Pub. Co., Cambridge, Mass (1954)Google Scholar
  13. 13.
    Hodgkin, A.L., Huxley, A.F.: A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 117(4), 500 (1952)CrossRefGoogle Scholar
  14. 14.
    Ijspeert, A.J.: Central pattern generators for locomotion control in animals and robots: a review. Neural Netw. 21(4), 642–653 (2008)CrossRefGoogle Scholar
  15. 15.
    Judd, J.S.: On the complexity of loading shallow neural networks. J. Complex. 4(3), 177–192 (1988)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Judd, J.S.: Neural Network Design and the Complexity of Learning. Neural Network Modeling and Connectionism Series. MIT Press, Cambridge, MA (1990)Google Scholar
  17. 17.
    Lapicque, L.: Recherches quantitatives sur l’excitation electrique des nerfs traitee comme une polarization. Journal de Physiologie et de Pathologie Generalej 9, 620–635 (1907)Google Scholar
  18. 18.
    Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10(9), 1659–1671 (1997)CrossRefGoogle Scholar
  19. 19.
    Maass, W., Schmitt, M.: On the complexity of learning for spiking neurons with temporal coding. Inf. Comput. 153(1), 26–46 (1999)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Markou, M., Singh, S.: Novelty detection: a review-part 2: neural network based approaches. Sig. Process. 83(12), 2499–2521 (2003)CrossRefGoogle Scholar
  21. 21.
    McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biology 5(4), 115–133 (1943)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Montgomery, D.C.: Design and Analysis of Experiments (2013)Google Scholar
  23. 23.
    Ojha, V.K., Abraham, A., Snášel, V.: Metaheuristic design of feedforward neural networks: a review of two decades of research. Eng. Appl. Artif. Intell. 60, 97–116 (2017)CrossRefGoogle Scholar
  24. 24.
    O’Neill, M., Ryan, C.: Grammatical evolution. Trans. Evol. Comp. 5(4), 349–358 (2001)CrossRefGoogle Scholar
  25. 25.
    Quiroz-Ramírez, O., Espinal, A., Ornelas-Rodríguez, M., Rojas-Domínguez, A., Sánchez, D., Puga-Soberanes, H., Carpio, M., Espinoza, L.E.M., Ortíz-López, J.: Partially-connected artificial neural networks developed by grammatical evolution for pattern recognition problems. Stud. Comput. Intell. 749, 99–112 (2018)Google Scholar
  26. 26.
    Rosenblatt, E.: The Perceptron, A Perceiving And Recognizing Automaton (Project PARA). Cornell Aeronautical Laboratory (1957)Google Scholar
  27. 27.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533 (1986)CrossRefGoogle Scholar
  28. 28.
    Ryan, C., Collins, J., O’Neill, M.: Grammatical evolution: evolving programs for an arbitrary language. In: Proceedings of Genetic Programming: First European Workshop. EuroGP’98, Paris, France, 14–15 April 1998, pp. 83–96. Springer, Berlin, Heidelberg (1998)Google Scholar
  29. 29.
    Scarselli, F., Tsoi, A.C.: Universal approximation using feedforward neural networks: a survey of some existing methods, and some new results. Neural Netw. 11(1), 15–37 (1998)CrossRefGoogle Scholar
  30. 30.
    Shapiro, S.S., Wilk, M.B.: An analysis of variance test for normality (complete samples). Biometrika 52(3–4), 591–611 (1965)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Talbi, E.-G.: Metaheuristics: from Design to Implementation. Wiley, Hoboken, NJ, (2009). OCLC: ocn230183356Google Scholar
  32. 32.
    Yao, X.: Evolving artificial neural networks. Proc. IEEE 87(9), 1423–1447 (1999)CrossRefGoogle Scholar
  33. 33.
    Yu, J., Tan, M., Chen, J., Zhang, J.: A survey on CPG-inspired control models and system implementation. IEEE Trans. Neural Netw. Learning Syst. 3, 441–456 (2014)CrossRefGoogle Scholar
  34. 34.
    Zhang, G.P.: Neural networks for classification: a survey. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 30(4), 451–462 (2000)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • G. López-Vázquez
    • 1
  • A. Espinal
    • 2
  • Manuel Ornelas-Rodríguez
    • 1
    Email author
  • J. A. Soria-Alcaraz
    • 2
  • A. Rojas-Domínguez
    • 1
  • Héctor Puga
    • 1
  • J. Martín Carpio
    • 1
  • H. Rostro-González
    • 3
  1. 1.Division of Postgraduate Studies and ResearchNational Technology of México/León Institute of TechnologyLeón, GuanajuatoMéxico
  2. 2.Department of Organizational StudiesDCEA-University of GuanajuatoGuanajuatoMéxico
  3. 3.Department of ElectronicsDICIS-University of GuanajuatoSalamanca, GuanajuatoMéxico

Personalised recommendations