An Effective Hybrid Approach for Optimising the Learning Process of Multi-layer Neural Networks
Finding the optimal connection weights in a neural network is one of the most challenging tasks in machine learning and pattern recognition. The main disadvantage of conventionally used algorithms such as back-propagation is that they show a tendency of getting trapped in local rather than global optima. To address this, population-based metaheuristic algorithms can be employed. In this paper, we propose a novel approach to optimise the weights of a neural network. Our method integrates an imperialist competitive algorithm, a powerful metaheuristic algorithm, with chaos theory and back-propagation for neural network learning. Experimental results on the three-bit parity problem and several function approximation tasks confirm that our proposed algorithm significantly outperforms several state-of-the-art methods for neural network weight optimisation.
KeywordsMachine learning Neural networks Weight optimisation Imperialist competitive algorithm Chaos theory Back-propagation
The paper is published due to the financial support of the Ministry of Science and Higher Education of Russia, contract 14.575.21.0152, signed 26/09/2017, unique identifier RFMEFI57517X0152.
- 1.Alatas, B.: Chaotic bee colony algorithms for global numerical optimization. Expert Syst. Appl. 37(8), 5682–5687 (2010)Google Scholar
- 3.Amirsadri, S., Mousavirad, S.J., Ebrahimpour-Komleh, H.: A levy flight-based grey wolf optimizer combined with back-propagation algorithm for neural network training. Neural Comput. Appl. 30, 1–14 (2017)Google Scholar
- 4.Atashpaz-Gargari, E., Lucas, C.: Imperialist competitive algorithm: an algorithm for optimization inspired by imperialistic competition. In: IEEE Congress on Evolutionary Computation, pp. 4661–4667 (2007)Google Scholar
- 5.Chan, W., Jaitly, N., Le, Q., Vinyals, O.: Listen, attend and spell: a neural network for large vocabulary conversational speech recognition. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4960–4964 (2016)Google Scholar
- 6.Egmont-Petersen, M., de Ridder, D., Handels, H.: Image processing with neural networks - a review. Pattern Recogn. 35(10), 2279–2301 (2002)Google Scholar
- 7.Heidari, A.A., Faris, H., Aljarah, I., Mirjalili, S.: An efficient hybrid multilayer perceptron neural network with grasshopper optimization. Soft Comput. 1–18 (2018)Google Scholar
- 8.Liu, D., Hohil, M.E., Smith, S.H.: N-bit parity neural networks: new solutions based on linear programming. Neurocomputing 48(1–4), 477–488 (2002)Google Scholar
- 9.Mahmoudi, M.T., Taghiyareh, F., Forouzideh, N., Lucas, C.: Evolving artificial neural network structure using grammar encoding and colonial competitive algorithm. Neural Comput. Appl. 22(1), 1–16 (2013)Google Scholar
- 10.Mirjalili, S.: How effective is the grey wolf optimizer in training multi-layer perceptrons. Appl. Intell. 43(1), 150–161 (2015)Google Scholar
- 11.Mirjalili, S., Mirjalili, S.M., Lewis, A.: Let a biogeography-based optimizer train your multi-layer perceptron. Inf. Sci. 269, 188–209 (2014)Google Scholar
- 12.Mousavirad, S.J., Rezaee, K., Nasri, K.: A new method for identification of Iranian rice kernel varieties using optimal morphological features and an ensemble classifier by image processing. Majlesi J. Multimedia Process. 1, 1–8 (2012)Google Scholar
- 13.Saremi, S., Mirjalili, S., Lewis, A.: Biogeography-based optimisation with chaos. Neural Comput. Appl. 25(5), 1077–1097 (2014)Google Scholar