Automatic Design of Artificial Neural Networks and Associative Memories for Pattern Classification and Pattern Restoration

  • Humberto Sossa
  • Beatriz A. Garro
  • Juan Villegas
  • Carlos Avilés
  • Gustavo Olague
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7329)

Abstract

In this note we present our most recent advances in the automatic design of artificial neural networks (ANNs) and associative memories (AMs) for pattern classification and pattern recall. Particle Swarm Optimization (PSO), Differential Evolution (DE), and Artificial Bee Colony (ABC) algorithms are used for ANNs; Genetic Programming is adopted for AMs. The derived ANNs and AMs are tested with several examples of well-known databases. As we will show, results are very promising.

Keywords

Artificial neural networks Associative memories Evolutionary programming 

References

  1. 1.
    Gurney, K.: An introduction to neural networks. Taylor and Francis Group (1997)Google Scholar
  2. 2.
    McCulloch, W., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 7, 115–133 (1943)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Rosenblatt, F.: The Perceptron-a perceiving and recognizing automaton. Report 85-460-1, Cornell Aeronautical Laboratory (1957)Google Scholar
  4. 4.
    Rojas, R.: Neural networks - A systematic introduction. The back propagation algorithm, ch. 7 (1996)Google Scholar
  5. 5.
    Anderson, J.A.: A simple neural network generating an interactive memory. Mathematical Biosciences 14, 197–220 (1972)MATHCrossRefGoogle Scholar
  6. 6.
    Kohonen, T.: Correlation matrix memories. IEEE Transactions on Computers C-21(4), 353–359 (1972)CrossRefGoogle Scholar
  7. 7.
    Steinbuch, K.: Die Lernmatrix. Kybernetik 1(1), 36–45 (1961)MATHCrossRefGoogle Scholar
  8. 8.
    Ritter, G., Díaz, J.: Morphological associative memories. IEEE Transactions on Neural Networks 9(2), 281–293 (1998)CrossRefGoogle Scholar
  9. 9.
    Yao, X.: A review of evolutionary artificial neural networks. Int. J. Intell. Syst. 8(4), 539–567 (1993)CrossRefGoogle Scholar
  10. 10.
    Yao, X.: Evolutionary artificial neural networks. In: Kent, A., Williams, J.G. (eds.) Encyclopedia of Computer Science and Technology, vol. 33, pp. 137–170. Marcel Dekker, New York (1995)Google Scholar
  11. 11.
    Garro, B.A., Sossa, H., Vázquez, R.A.: Design of Artificial Neural Networks using a Modified Particle Swarm Optimization Algorithm. In: International Joint Conference on Neural Networks (IJCNN 2009), Atlanta, GE, USA, June 14-19, pp. 938–945 (2009)Google Scholar
  12. 12.
    Garro, B.A., Sossa, H., Vázquez, R.A.: Design of Artificial Neural Networks Using Differential Evolution Algorithm. In: Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds.) ICONIP 2010, Part II. LNCS, vol. 6444, pp. 201–208. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.
    Garro, B.A., Sossa, H., Vázquez, R.A.: Evolving Neural Networks: A comparison between Differential Evolution and Particle Swarm Optimization. In: Tan, Y., Shi, Y., Chai, Y., Wang, G. (eds.) ICSI 2011, Part I. LNCS, vol. 6728, pp. 447–454. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  14. 14.
    Garro, B.A., Sossa, H., Vázquez, R.A.: Artificial Neural Network Synthesis by means of Artificial Bee Colony (ABC) Algorithm. In: CEC 2011, New Orleans, June 5-8 (2011)Google Scholar
  15. 15.
    Garro, B., Sossa, H., Vazquez, R.A.: Back-Propagation vs Particle Swarm Optimization Algorithm: which Algorithm is better to adjust the Synaptic Weights of a Feed-Forward ANN? International Journal of Artificial Intelligence 7(11), 208–218 (2011)Google Scholar
  16. 16.
    Kennedy, J., Eberhart, R.: Particle Swarm Optimization. In: Proceedings of IEEE International Conference on Neural Networks, vol. IV, pp. 1942–1948 (1995)Google Scholar
  17. 17.
    Storn, R., Price, K.: Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization 11, 341–359 (1997)MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    Karaboga, D., Basturk, B.: A powerful and efficient algorithm for numerical function optimization: Artificial Bee Colony (ABC) algorithm. Journal of Global Optimization 39(3), 459–471 (2007)MathSciNetMATHCrossRefGoogle Scholar
  19. 19.
    Villegas, J., Sossa, H., Avilés, C., Olague, G.: Automatic Synthesis of Associative Memories by genetic Programming, a First Approach. Research in Computing Science 42, 91–102 (2009)Google Scholar
  20. 20.
    Villegas, J., Sossa, H., Avilés, C., Olague, G.: Automatic Synthesis of Associative Memories through Genetic Programming: a co-evolutionary approach. Revista Mexicana de Física 57(2), 110–116 (2011)Google Scholar
  21. 21.
    Murphy, P.M., Aha, D.W.: UCI Repository of machine learning databases. University of California, Department of Information and Computer Science, Irvine, CA, US., Tech. Rep. (1994)Google Scholar
  22. 22.
    Koza, J.R.: Genetic Programming: A Paradigm for Genetically Breeding Populations of Computer Programs to Solve Problems, Stanford University Computer Science Department technical report (1990)Google Scholar
  23. 23.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press (1992)Google Scholar
  24. 24.
    Koza, J.R.: Genetic Programming II: Automatic Discovery of Reusable Programs. MIT Press (1994)Google Scholar
  25. 25.
    Koza, J.R., Bennett, F.H., Andre, D., Keane, M.A.: Genetic Programming III: Darwinian Invention and Problem Solving. Morgan Kaufmann (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Humberto Sossa
    • 1
  • Beatriz A. Garro
    • 1
  • Juan Villegas
    • 2
  • Carlos Avilés
    • 2
  • Gustavo Olague
    • 3
  1. 1.CIC-IPNMéxicoMexico
  2. 2.UAM-AzcapotzalcoMéxicoMexico
  3. 3.CICESEEnsenadaMexico

Personalised recommendations