Advertisement

An Effective Hybrid Approach for Optimising the Learning Process of Multi-layer Neural Networks

  • Seyed Jalaleddin Mousavirad
  • Azam Asilian Bidgoli
  • Hossein Ebrahimpour-Komleh
  • Gerald SchaeferEmail author
  • Iakov Korovin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11554)

Abstract

Finding the optimal connection weights in a neural network is one of the most challenging tasks in machine learning and pattern recognition. The main disadvantage of conventionally used algorithms such as back-propagation is that they show a tendency of getting trapped in local rather than global optima. To address this, population-based metaheuristic algorithms can be employed. In this paper, we propose a novel approach to optimise the weights of a neural network. Our method integrates an imperialist competitive algorithm, a powerful metaheuristic algorithm, with chaos theory and back-propagation for neural network learning. Experimental results on the three-bit parity problem and several function approximation tasks confirm that our proposed algorithm significantly outperforms several state-of-the-art methods for neural network weight optimisation.

Keywords

Machine learning Neural networks Weight optimisation Imperialist competitive algorithm Chaos theory Back-propagation 

Notes

Acknowledgements

The paper is published due to the financial support of the Ministry of Science and Higher Education of Russia, contract 14.575.21.0152, signed 26/09/2017, unique identifier RFMEFI57517X0152.

References

  1. 1.
    Alatas, B.: Chaotic bee colony algorithms for global numerical optimization. Expert Syst. Appl. 37(8), 5682–5687 (2010)Google Scholar
  2. 2.
    Alba, E., Chicano, J.F.: Training neural networks with GA hybrid algorithms. In: Deb, K. (ed.) GECCO 2004. LNCS, vol. 3102, pp. 852–863. Springer, Heidelberg (2004).  https://doi.org/10.1007/978-3-540-24854-5_87Google Scholar
  3. 3.
    Amirsadri, S., Mousavirad, S.J., Ebrahimpour-Komleh, H.: A levy flight-based grey wolf optimizer combined with back-propagation algorithm for neural network training. Neural Comput. Appl. 30, 1–14 (2017)Google Scholar
  4. 4.
    Atashpaz-Gargari, E., Lucas, C.: Imperialist competitive algorithm: an algorithm for optimization inspired by imperialistic competition. In: IEEE Congress on Evolutionary Computation, pp. 4661–4667 (2007)Google Scholar
  5. 5.
    Chan, W., Jaitly, N., Le, Q., Vinyals, O.: Listen, attend and spell: a neural network for large vocabulary conversational speech recognition. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4960–4964 (2016)Google Scholar
  6. 6.
    Egmont-Petersen, M., de Ridder, D., Handels, H.: Image processing with neural networks - a review. Pattern Recogn. 35(10), 2279–2301 (2002)Google Scholar
  7. 7.
    Heidari, A.A., Faris, H., Aljarah, I., Mirjalili, S.: An efficient hybrid multilayer perceptron neural network with grasshopper optimization. Soft Comput. 1–18 (2018)Google Scholar
  8. 8.
    Liu, D., Hohil, M.E., Smith, S.H.: N-bit parity neural networks: new solutions based on linear programming. Neurocomputing 48(1–4), 477–488 (2002)Google Scholar
  9. 9.
    Mahmoudi, M.T., Taghiyareh, F., Forouzideh, N., Lucas, C.: Evolving artificial neural network structure using grammar encoding and colonial competitive algorithm. Neural Comput. Appl. 22(1), 1–16 (2013)Google Scholar
  10. 10.
    Mirjalili, S.: How effective is the grey wolf optimizer in training multi-layer perceptrons. Appl. Intell. 43(1), 150–161 (2015)Google Scholar
  11. 11.
    Mirjalili, S., Mirjalili, S.M., Lewis, A.: Let a biogeography-based optimizer train your multi-layer perceptron. Inf. Sci. 269, 188–209 (2014)Google Scholar
  12. 12.
    Mousavirad, S.J., Rezaee, K., Nasri, K.: A new method for identification of Iranian rice kernel varieties using optimal morphological features and an ensemble classifier by image processing. Majlesi J. Multimedia Process. 1, 1–8 (2012)Google Scholar
  13. 13.
    Saremi, S., Mirjalili, S., Lewis, A.: Biogeography-based optimisation with chaos. Neural Comput. Appl. 25(5), 1077–1097 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Seyed Jalaleddin Mousavirad
    • 1
  • Azam Asilian Bidgoli
    • 1
  • Hossein Ebrahimpour-Komleh
    • 1
  • Gerald Schaefer
    • 2
    Email author
  • Iakov Korovin
    • 3
  1. 1.Department of Computer and Electrical EngineeringUniversity of KashanKashanIran
  2. 2.Department of Computer ScienceLoughborough UniversityLoughboroughUK
  3. 3.Southern Federal UniversityTaganrogRussia

Personalised recommendations