Advertisement

Adaptive memetic method of multi-objective genetic evolutionary algorithm for backpropagation neural network

  • Ashraf Osman IbrahimEmail author
  • Siti Mariyam Shamsuddin
  • Ajith Abraham
  • Sultan Noman Qasem
Original Article
  • 43 Downloads

Abstract

In recent years, multi-objective evolutionary optimization algorithms have shown success in different areas of research. Due to their efficiency and power, many researchers have concentrated on adapting evolutionary algorithms to generate Pareto solutions. This paper proposes a new memetic adaptive multi-objective evolutionary algorithm that is based on a three-term backpropagation network (MAMOT). This algorithm is an automatic search method for optimizing the parameters and performance of neural networks, and it relies on the use of the adaptive non-dominated sorting genetic algorithm-II integrated with the backpropagation algorithm, being used as a local search method. The presented MAMOT employs a self-adaptive mechanism toward improving the performance of the proposed algorithm and a local optimizer improving all the individuals in a population in order to obtain better accuracy and connection weights. In addition, it selects an appropriate number of hidden nodes simultaneously. The proposed method was applied to 11 datasets representing pattern classification problems, including two-class, multi-class and complex data reflecting real problems. Experiments were performed, and the results indicated that the proposed method is viable in pattern classification tasks compared to a multi-objective genetic algorithm based on a three-term backpropagation network (MOGAT) and some of the methods mentioned in the literature. The statistical analysis results of the t test and Wilcoxon signed-ranks test also show that the performance of the proposed method is significantly better than MOGAT.

Keywords

Self-adaptive Artificial neural networks Three-term backpropagation Multi-objective evolutionary Non-dominated sorting genetic algorithm-II 

Notes

Acknowledgements

The authors would like to thank the anonymous reviewers and the editor for their useful advice and constructive comments.

Compliance with ethical standards

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this paper.

References

  1. 1.
    Salari N, Shohaimi S, Najafi F, Nallappan M, Karishnarajah I (2014) A novel hybrid classification model of genetic algorithms, modified k-nearest neighbor and developed backpropagation neural network. PLoS One 9:e112987CrossRefGoogle Scholar
  2. 2.
    Prieto A, Bellas F, Duro RJ, Lopez-Peña F (2007) Auto adjustable ANN-based classification system for optimal high dimensional data analysis. Computational and ambient intelligence. Springer, Berlin, pp 588–596Google Scholar
  3. 3.
    Garro BA, Sossa H, Vázquez RA (2011) Evolving neural networks: a comparison between differential evolution and particle swarm optimization. Springer, Berlin, pp 447–454Google Scholar
  4. 4.
    Qasem SN, Shamsuddin SM, Zain AM (2011) Multi-objective hybrid evolutionary algorithms for radial basis function neural network design. Knowl Based Syst 27:475–497CrossRefGoogle Scholar
  5. 5.
    Sagar G, Chalam SV, Singh MK (2011) Evolutionary algorithm for optimal connection weights in artificial neural networks. Int J Eng (IJE) 5:333Google Scholar
  6. 6.
    Yu Q, Peng J (2011) Music category based on adaptive mutation particle swarm optimization BP neural network. In: Wu Y (ed) Advances in computer, communication, control and automation, vol 121. Lecture notes in electrical engineering. Berlin, Heidelberg, pp 657–663CrossRefGoogle Scholar
  7. 7.
    Abraham A, Nath B (2000) Optimal design of neural nets using hybrid algorithms. In: PRICAI 2000 topics in artificial intelligence, Springer, Berlin, pp 510–520Google Scholar
  8. 8.
    Zhang R, Tao J (2017) Data-driven modeling using improved multi-objective optimization based neural network for coke furnace system. IEEE Trans Ind Electron 64:3147–3155CrossRefGoogle Scholar
  9. 9.
    Ahmad F, Isa NAM, Hussain Z, Osman MK, Sulaiman SN (2015) A GA-based feature selection and parameter optimization of an ANN in diagnosing breast cancer. Pattern Anal Appl 18:861–870MathSciNetCrossRefGoogle Scholar
  10. 10.
    Yao X (1999) Evolving artificial neural networks. Proc IEEE 87:1423–1447CrossRefGoogle Scholar
  11. 11.
    Hernández C, Schütze O, Sun J-Q (2017) Global multi-objective optimization by means of cell mapping techniques. In: EVOLVE–a bridge between probability, set oriented numerics and evolutionary computation VII, Springer, Cham, pp 25–56Google Scholar
  12. 12.
    Zhou A, Qu B-Y, Li H, Zhao S-Z, Suganthan PN et al (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evolut Comput 1:32–49CrossRefGoogle Scholar
  13. 13.
    Cruz-Ramírez M, Hervás-Martínez C, Jurado-Expósito M, López-Granados F (2012) A multi-objective neural network based method for cover crop identification from remote sensed data. Expert Syst Appl 39:10038–10048CrossRefGoogle Scholar
  14. 14.
    Ibrahim AO, Shamsuddin SM, Ahmad NB, Qasem SN (2013) Three-term backpropagation network based on elitist multiobjective genetic algorithm for medical diseases diagnosis classification. Life Sci J 10(4):1815–1822Google Scholar
  15. 15.
    Ak R, Li Y, Vitelli V, Zio E, Droguett EL et al (2013) NSGA-II-trained neural network approach to the estimation of prediction intervals of scale deposition rate in oil & gas equipment. Exp Syst Appl 40:1205–1212CrossRefGoogle Scholar
  16. 16.
    Fernández JC, Cruz-Ramírez M, Hervás-Martínez C (2018) Sensitivity versus accuracy in ensemble models of artificial neural networks from multi-objective evolutionary algorithms. Neural Comput Appl 30(1):289–305CrossRefGoogle Scholar
  17. 17.
    Abbass HA (2003) Speeding up backpropagation using multiobjective evolutionary algorithms. Neural Comput 15:2705–2726CrossRefzbMATHGoogle Scholar
  18. 18.
    Fernandez Caballero JC, Martínez FJ, Hervás C, Gutiérrez PA (2010) Sensitivity versus accuracy in multiclass problems using memetic Pareto evolutionary neural networks. IEEE Trans Neural Netw 21:750–770CrossRefGoogle Scholar
  19. 19.
    Silva VV, Fleming PJ, Sugimoto J, Yokoyama R (2008) Multiobjective optimization using variable complexity modelling for control system design. Appl Soft Comput 8:392–401CrossRefGoogle Scholar
  20. 20.
    Gorzałczany MB, Rudziński F (2017) Interpretable and accurate medical data classification—a multi-objective genetic-fuzzy optimization approach. Exp Syst Appl 71:26–39CrossRefGoogle Scholar
  21. 21.
    Pettersson F, Chakraborti N, Saxén H (2007) A genetic algorithms based multi-objective neural net applied to noisy blast furnace data. Appl Soft Comput 7:387–397CrossRefGoogle Scholar
  22. 22.
    Jin Y, Sendhoff B, Körner E (2005) Evolutionary multi-objective optimization for simultaneous generation of signal-type and symbol-type representations. Springer, Berlin, pp 752–766zbMATHGoogle Scholar
  23. 23.
    Jin Y, Sendhoff B (2008) Pareto-based multiobjective machine learning: an overview and case studies. IEEE Trans Syst Man Cybern Part C Appl Rev 38:397–415Google Scholar
  24. 24.
    Garcıa-Pedrajas N, Ortiz-Boyer D, Hervás-Martınez C (2004) Cooperative coevolution of generalized multi-layer perceptrons. Neurocomputing 56:257–283CrossRefGoogle Scholar
  25. 25.
    Delgado M, Cuéllar MP, Pegalajar MC (2008) Multiobjective hybrid optimization and training of recurrent neural networks. IEEE Trans Syst Man Cybern Part B Cybern 38:381–403CrossRefGoogle Scholar
  26. 26.
    Jin Y, Sendhoff B, Körner E (2006) Simultaneous generation of accurate and interpretable neural network classifiers. Multi-objective machine learning. Springer, Berlin, pp 291–312Google Scholar
  27. 27.
    Wiegand S, Igel C, Handmann U (2004) Evolutionary multi-objective optimisation of neural networks for face detection. Int J Comput Intell Appl 4:237–253CrossRefGoogle Scholar
  28. 28.
    Abbass HA (2002) An evolutionary artificial neural networks approach for breast cancer diagnosis. Artif Intell Med 25:265–281CrossRefGoogle Scholar
  29. 29.
    Almeida LM, Ludermir TB (2010) A multi-objective memetic and hybrid methodology for optimizing the parameters and performance of artificial neural networks. Neurocomputing 73:1438–1450CrossRefGoogle Scholar
  30. 30.
    Cruz-Ramírez M, Sánchez-Monedero J, Fernández-Navarro F, Fernández J, Hervás-Martínez C (2010) Memetic pareto differential evolutionary artificial neural networks to determine growth multi-classes in predictive microbiology. Evol Intel 3:187–199CrossRefGoogle Scholar
  31. 31.
    Qasem SN, Shamsuddin SM (2011) Memetic elitist pareto differential evolution algorithm based radial basis function networks for classification problems. Appl Soft Comput 11:5565–5581CrossRefGoogle Scholar
  32. 32.
    Cruz-Ramírez M, Hervás-Martínez C, Fernández JC, Briceño J, de la Mata M (2012) Multi-objective evolutionary algorithm for donor-recipient decision system in liver transplants. Eur J Oper Res 222(2):317–327CrossRefGoogle Scholar
  33. 33.
    Abbass HA (2002) The self-adaptive pareto differential evolution algorithm. In: Proceedings of the 2002 congress on evolutionary computation. pp 831–836Google Scholar
  34. 34.
    Huang VL, Zhao SZ, Mallipeddi R, Suganthan PN (2009) Multi-objective optimization using self-adaptive differential evolution algorithm. In: IEEE congress on evolutionary computation pp 190–194Google Scholar
  35. 35.
    Zeng F, Low MYH, Decraene J, Zhou S, Cai W (2010) Self-adaptive mechanism for multi-objective evolutionary algorithms. In Proceedings of the international multiconference of engineers and computer scientists, pp 7–12Google Scholar
  36. 36.
    Shim VA, Tan KC, Tang H (2014) Adaptive memetic computing for evolutionary multiobjective optimization. IEEE Trans Cybern 45(4):610–621CrossRefGoogle Scholar
  37. 37.
    Caruana R, Niculescu-Mizil A (2004) Data mining in metric space: an empirical analysis of supervised learning performance criteria. In: ACM. pp 69–78Google Scholar
  38. 38.
    Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27:861–874CrossRefGoogle Scholar
  39. 39.
    Thangaraj R, Pant M, Abraham A, Badr Y (2009) Hybrid evolutionary algorithm for solving global optimization problems. Hybrid artificial intelligence systems. Springer, Berlin, pp 310–318CrossRefGoogle Scholar
  40. 40.
    Krasnogor N, Smith J (2005) A tutorial for competent memetic algorithms: model, taxonomy, and design issues. IEEE Trans Evolut Comput 9:474–488CrossRefGoogle Scholar
  41. 41.
    Fernández J, Hervás C, Martínez F, Gutiérrez P, Cruz M (2009) Memetic Pareto differential evolution for designing artificial neural networks in multiclassification problems using cross-entropy versus sensitivity. In: Hybrid artificial intelligence systems. pp 433–441Google Scholar
  42. 42.
    Schaffer JD (1985) Multiple objective optimization with vector evaluated genetic algorithms. L. Erlbaum Associates Inc, Mahwah, pp 93–100Google Scholar
  43. 43.
    Srinivas N, Deb K (1994) Muiltiobjective optimization using nondominated sorting in genetic algorithms. Evol Comput 2:221–248CrossRefGoogle Scholar
  44. 44.
    Fonseca CM, Fleming PJ (1998) Multiobjective optimization and multiple constraint handling with evolutionary algorithms. II. Application example. IEEE Trans Syst Man Cybern Part A Syst Hum 28:38–47CrossRefGoogle Scholar
  45. 45.
    Zweiri Y, Whidborne J, Seneviratne L (2003) A three-term backpropagation algorithm. Neurocomputing 50:305–318CrossRefzbMATHGoogle Scholar
  46. 46.
    Zweiri YH (2007) Optimization of a three-term backpropagation algorithm used for neural network learning. Int J Comput Intell 3:322–327Google Scholar
  47. 47.
    Deb K (1999) An introduction to genetic algorithms. Indian Acad Sci 24:293–315MathSciNetzbMATHGoogle Scholar
  48. 48.
    Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evolut Comput 6:182–197CrossRefGoogle Scholar
  49. 49.
    Ak R, Li Y, Vitelli V, Zio E, Droguett EL et al (2012) NSGA-II-trained neural network approach to the estimation of prediction intervals of scale deposition rate in oil & gas equipment. Exp Syst Appl 40(4):1205–1212CrossRefGoogle Scholar
  50. 50.
    Ramesh S, Kannan S, Baskar S (2012) Application of modified NSGA-II algorithm to multi-objective reactive power planning. Appl Soft Comput 12(2):741–753CrossRefGoogle Scholar
  51. 51.
    Deb K, Sindhya K, Okabe T. Self-adaptive simulated binary crossover for real-parameter optimization; 2007. In: Proceedings of the genetic and evolutionary computation conference (GECCO-2007), UCL London. pp 1187–1194Google Scholar
  52. 52.
    Zhang C, Ren M, Zhang B (2013) A self-adaptive multi-objective genetic algorithm for the QoS based routing and wavelength allocation problem in WDM network. Opt Int J Light Electron Opt 124(20):4571–4575CrossRefGoogle Scholar
  53. 53.
    Lara A, Sanchez G, Coello Coello CA, Schutze O (2010) HCS: a new local search strategy for memetic multiobjective evolutionary algorithms. IEEE Trans Evolut Comput 14:112–132CrossRefGoogle Scholar
  54. 54.
    Qasem SN, Shamsuddin SM, Hashi SZM, Darus M, Al-Shammari E (2013) Memetic multiobjective particle swarm optimization-based radial basis function network for classification problems. Inf Sci 239:165–190MathSciNetCrossRefGoogle Scholar
  55. 55.
    Asuncion A, Newman DJ (2007) UCI machine learning repository. http://www.icsuciedu/$\sim$mlearn/{MLR}epositoryhtml. Accessed 18 July 2017
  56. 56.
    Hervás C, Silva M, Gutiérrez PA, Serrano A (2008) Multilogistic regression by evolutionary neural network as a classification tool to discriminate highly overlapping signals: qualitative investigation of volatile organic compounds in polluted waters by using headspace-mass spectrometric analysis. Chemom Intell Lab Syst 92(2):179–185CrossRefGoogle Scholar
  57. 57.
    Goh C-K, Teoh E-J, Tan KC (2008) Hybrid multiobjective evolutionary design for artificial neural networks. IEEE Trans Neural Netw 19:1531–1548CrossRefGoogle Scholar
  58. 58.
    Bhuvana J, Aravindan C (2016) Memetic algorithm with Preferential Local Search using adaptive weights for multi-objective optimization problems. Soft Comput 20(4):1365–1388CrossRefGoogle Scholar
  59. 59.
    Stehlík M, Střelec L, Thulin M (2014) On robust testing for normality in chemometrics. Chemom Intell Lab Syst 130:98–108CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Ashraf Osman Ibrahim
    • 1
    • 2
    • 3
    Email author
  • Siti Mariyam Shamsuddin
    • 4
  • Ajith Abraham
    • 5
  • Sultan Noman Qasem
    • 6
    • 7
  1. 1.Faculty of Computer Science and Information TechnologyAlzaiem Alazhari UniversityKhartoum NorthSudan
  2. 2.Arab Open UniversityKhartoumSudan
  3. 3.Faculty of computer scienceFuture UniversityKhartoumSudan
  4. 4.UTM Big Data CentreUniversiti Teknologi MalaysiaSkudaiMalaysia
  5. 5.Machine Intelligence Research Labs (MIR Labs)Scientific Network for Innovation and Research ExcellenceAuburnUSA
  6. 6.Computer Science Department, College of Computer and Information SciencesAl Imam Mohammad Ibn Saud Islamic University (IMSIU)RiyadhSaudi Arabia
  7. 7.Computer Science Department, Faculty of Applied SciencesTaiz UniversityTaizYemen

Personalised recommendations