Advertisement

Enhancing Competitive Island Cooperative Neuro-Evolution Through Backpropagation for Pattern Classification

  • Gary Wong
  • Rohitash ChandraEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9489)

Abstract

Cooperative coevolution is a promising method for training neural networks which is also known as cooperative neuro-evolution. Cooperative neuro-evolution has been used for pattern classification, time series prediction and global optimisation problems. In the past, competitive island based cooperative coevolution has been proposed that employed different instances of problem decomposition methods for competition. Neuro-evolution has limitations in terms of training time although they are known as global search methods. Backpropagation algorithm employs gradient descent which helps in faster convergence which is needed for neuro-evolution. Backpropagation suffers from premature convergence and its combination with neuro-evolution can help eliminate the weakness of both the approaches. In this paper, we propose a competitive island cooperative neuro-evolutionary method that takes advantage of the strengths of gradient descent and neuro-evolution. We use feedforward neural networks on benchmark pattern classification problems to evaluate the performance of the proposed algorithm. The results show improved performance when compared to related methods.

References

  1. 1.
    Potter, M., De Jong, K.: A cooperative coevolutionary approach to function optimization. In: Davidor, Y., Schwefel, H.-P., Mnner, R. (eds.) PPSN III. LNCS, vol. 866, pp. 249–257. Springer, Heidelberg (1994)Google Scholar
  2. 2.
    Chandra, R., Frean, M., Zhang, M.: On the issue of separability for problem decomposition in cooperative neuro-evolution. Neurocomputing 87, 33–40 (2012)CrossRefGoogle Scholar
  3. 3.
    Omidvar, M., Li, X., Yao, X.: Cooperative co-evolution with delta grouping for large scale non-separable function optimization. IEEE Congr. Evol. Comput. (CEC) 2010, 1762–1779 (2010)Google Scholar
  4. 4.
    Chandra, R.: Competitive two-island cooperative coevolution for training Elman recurrent networks for time series prediction. In: International Joint Conference on Neural Networks (IJCNN), Beijing, China, pp. 565–572, July 2014Google Scholar
  5. 5.
    Chandra, R., Zhang, M.: Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction. Neurocomputing 186, 116–123 (2012)CrossRefGoogle Scholar
  6. 6.
    Garcia-Pedrajas, N., Hervas-Martinez, C., Munoz-Perez, J.: COVNET: a cooperative coevolutionary model for evolving artificial neural networks. IEEE Trans. Neural Netw. 14(3), 575–596 (2003)CrossRefGoogle Scholar
  7. 7.
    Potter, M.A., De Jong, K.A.: Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evol. Comput. 8, 1–29 (2000)CrossRefGoogle Scholar
  8. 8.
    Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. J. Mach. Learn. Res. 9, 937–965 (2008)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Chandra, R., Frean, M., Zhang, M.: An encoding scheme for cooperative coevolutionary feedforward neural networks. In: Li, J. (ed.) AI 2010. LNCS, vol. 6464, pp. 253–262. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  10. 10.
    Chandra, R.: Adaptive problem decomposition in cooperative coevolution of recurrent networks for time series prediction, In: International Joint Conference on Neural Networks (IJCNN), Dallas, TX, USA, pp. 1–8, August 2013Google Scholar
  11. 11.
    Chandra, R., Frean, M., Zhang, M.: Adapting modularity during learning in cooperative co-evolutionary recurrent neural networks. Soft Comput. A Fusion Found. Methodol. Appl. 16(6), 1009–1020 (2012)Google Scholar
  12. 12.
    Chandra, R., Frean, M.R., Zhang, M.: Crossover-based local search in cooperative co-evolutionary feedforward neural networks. Appl. Soft Comput. 12(9), 2924–2932 (2012)CrossRefGoogle Scholar
  13. 13.
    Chandra, R.: Competition and collaboration in cooperative coevolution of Elman recurrent neural networks for time-series prediction. IEEE Trans. Neural Netw. Learn. Syst. p. (2015, in press)Google Scholar
  14. 14.
    Chandra, R., Wong, G.: Competitive two-island cooperative coevolution for pattern classification problems. In: International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland, pp. 393–400, July 2015Google Scholar
  15. 15.
    Bella, G.: A bug’s life: competition among species towards the environment. ser. Fondazione Eni Enrico Mattei Working Papers. Fondazione Eni Enrico Mattei (2007)Google Scholar
  16. 16.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Rumelhart, D.E., McClelland, J.L., CORPORATE PDP Research Group (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, pp. 318–362. MIT Press, Cambridge (1986). http://dl.acm.org/citation.cfm?id=104293
  17. 17.
    Asuncion, A., Newman, D.: UCI machine learning repository (2007). http://archive.ics.uci.edu/ml/datasets.html

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.School of Computing Information and Mathematical SciencesUniversity of South PacificSuvaFiji
  2. 2.Artificial Intelligence and Cybernetics Research GroupSoftware FoundationNausoriFiji

Personalised recommendations