Evolutionary Intelligence

, Volume 10, Issue 3–4, pp 77–93 | Cite as

Automatic estimation of differential evolution parameters using Hidden Markov Models

  • Marwa Keshk
  • Hemant Singh
  • Hussein Abbass
Research Paper


Differential evolution (DE) has been successful in solving practical optimization problem. However, similar to other optimization algorithms, the search performance of DE depends on the efficacy of the adopted search operators. The ability to adapt these operators within an evolutionary run enhances their ability to find better quality solutions. This adaptation process requires learning algorithms capable of compressing the information embedded within a population into meaningful estimates to adapt the search operators. Hidden Markov Models (HMMs) are learning algorithms designed to estimate parameters by compressing information collected from on a state space. In this paper, we use HMMs to compress the information within a population and use the model for adapting the DE parameters. The resultant DE-HMM algorithm dynamically adjusts the two basic parameters of DE. After a thorough testing of this method and conducting an extensive comparison of its performance on the CEC2005 and CEC2014 dataset, it is shown that the proposed DE-HMM algorithm is able to achieve better results compared with the classical DE and other state-of-the-art methods. On average, the algorithm can achieve this performance faster than other methods in the literature.


Differential evolution (DE) Hidden Markov Models Self-adaptive parameter control 

Supplementary material

12065_2018_153_MOESM1_ESM.pdf (1.4 mb)
Supplementary material 1 (PDF 1386 KB)


  1. 1.
    The SPSS statistical tool (Jan 2016).
  2. 2.
    Abbass HA (2002) The self-adaptive pareto differential evolution algorithm. In: Evolutionary computation, 2002. CEC’02. proceedings of the 2002 congress on Bd. 1 IEEE, pp 831–836Google Scholar
  3. 3.
    Abraham A, Das S, Konar A (2006) Document clustering using differential evolution. In: 2006 IEEE international conference on evolutionary computation IEEE, pp 1784–1791Google Scholar
  4. 4.
    Al-Dabbagh RD, Kinsheel Azeddien, Mekhilef Saad, Baba Mohd S, Shamshirband Shahaboddin (2014) System identification and control of robot manipulator based on fuzzy adaptive differential evolution algorithm. Adv Eng Softw 78:60–66CrossRefGoogle Scholar
  5. 5.
    Auger A, Hansen N (2005) A restart CMA evolution strategy with increasing population size. In: 2005 IEEE congress on evolutionary computation Bd. 2 IEEE, pp 1769–1776Google Scholar
  6. 6.
    Baum LE, Petrie T (1966) Statistical inference for probabilistic functions of finite state Markov chains. Ann Math Stat 37(6):1554–1563MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Braak CJFT (2006) A Markov Chain Monte Carlo version of the genetic algorithm differential evolution: easy bayesian computing for real parameter spaces. Stat Comput 16(3):239–249MathSciNetCrossRefGoogle Scholar
  8. 8.
    Brest J, Greiner S, Boskovic Borko, Mernik Marjan, Zumer Viljem (2006) Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans Evol Comput 10(6):646–657CrossRefGoogle Scholar
  9. 9.
    Cao Y, Li Y, Coleman S, Belatreche A, McGinnity TM (2015) Adaptive Hidden Markov Model with anomaly states for price manipulation detection. IEEE Trans Neural Netw Learn Syst 26(2):318–330. MathSciNetCrossRefGoogle Scholar
  10. 10.
    Corriveau G, Guilbault R, Tahan A, Sabourin R (2016) Bayesian network as an adaptive parameter setting approach for genetic algorithms. Complex Intell Syst 2(1):1–22CrossRefGoogle Scholar
  11. 11.
    Črepinšek M, Liu SH, Mernik M (2013) Exploration and exploitation in evolutionary algorithms: a survey. ACM Comput Surv 45(3):35zbMATHGoogle Scholar
  12. 12.
    Das S, Mandal A, Mukherjee R (2014) An adaptive differential evolution algorithm for global optimization in dynamic environments. IEEE Trans Cybern 44(6):966–978CrossRefGoogle Scholar
  13. 13.
    Das S, Mullick SS, Suganthan PN (2016) Recent advances in differential evolution—an updated survey. Swarm Evol Comput 27:1–30CrossRefGoogle Scholar
  14. 14.
    Davis TE, Principe JC (1993) A Markov chain framework for the simple genetic algorithm. Evol Comput 1(3):269–288CrossRefGoogle Scholar
  15. 15.
    Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18CrossRefGoogle Scholar
  16. 16.
    Diao R, Shen Q: Deterministic parameter control in harmony search. In: 2010 UK workshop on computational intelligence (UKCI), 2010. ISSN: 2162–7657, pp 1–7Google Scholar
  17. 17.
    Elsayed SM, Sarker RA, Essam DL, Hamza NM (2014) Testing united multi-operator evolutionary algorithms on the CEC2014 real-parameter numerical optimization. In: 2014 IEEE congress on evolutionary computation (CEC). IEEE, pp 1650–1657Google Scholar
  18. 18.
    Fan Q, Yan X (2016) Self-adaptive differential evolution algorithm with zoning evolution of control parameters and adaptive mutation strategies. IEEE Trans Cybern 46(1):219–232. CrossRefGoogle Scholar
  19. 19.
    Gämperle R, Müller SD, Koumoutsakos P (2002) A parameter study for differential evolution. Adv Intell Syst Fuzzy Syst Evol Comput 10:293–298Google Scholar
  20. 20.
    García S, Molina D, Lozano M, Herrera F (2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms behaviour a case study on the CEC2005 special session on real parameter optimization. J Heurist 15(6):617–644CrossRefzbMATHGoogle Scholar
  21. 21.
    Goldberg DE, Segrest P (1987) Finite Markov chain analysis of genetic algorithms. In: Proceedings of the second international conference on genetic algorithms Bd. 1, p 1Google Scholar
  22. 22.
    Halder U, Das S, Maity D (2013) A cluster-based differential evolution algorithm with external archive for optimization in dynamic environments. IEEE Trans Cybern 43(3):881–897CrossRefGoogle Scholar
  23. 23.
    Hansen N, Müller SD, Koumoutsakos P (2003) Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evol Comput 11(1):1–18CrossRefGoogle Scholar
  24. 24.
    Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evol Comput 9(2):159–195CrossRefGoogle Scholar
  25. 25.
    Harl F, Chatelain F, Gouy-Pailler C, Achard S (2016) Bayesian Model for multiple change-points detection in multivariate time series. IEEE Trans Signal Process 64(16):4351–4362. MathSciNetCrossRefGoogle Scholar
  26. 26.
    He J, Yao X (2001) Drift analysis and average time complexity of evolutionary algorithms. Artif Intell 127(1):57–85MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    He J, Yao X (2017) Average drift analysis and population scalability. IEEE Trans Evol Comput 21(3):426–439MathSciNetGoogle Scholar
  28. 28.
    Hsu CH, Juang CF (2013) Evolutionary robot wall-following control using type-2 fuzzy controller with species-DE-activated continuous ACO. IEEE Trans Fuzzy Syst 21(1):100–112. CrossRefGoogle Scholar
  29. 29.
    Hu ZB, Su QH, Xiong SW, Hu FG (2008) Self-adaptive hybrid differential evolution with simulated annealing algorithm for numerical optimization. In: 2008 IEEE congress on evolutionary computation (IEEE world congress on computational intelligence) IEEE, pp 1189–1194.Google Scholar
  30. 30.
    Kamal A, MOhd MN, Elshaikh M, Badlishah R, (2016) Differential evolution (DE) algorithm to optimize Berkeley-MAC protocol for wireless sensor network (WSN). J Theoret Appl Inf Technol 89:2Google Scholar
  31. 31.
    Karafotias G, Hoogendoorn M, Eiben AE (2015) Parameter control in evolutionary algorithms: trends and challenges. IEEE Trans Evol Comput 19(2):167–187CrossRefGoogle Scholar
  32. 32.
    Kheawhom S (2010) Efficient constraint handling scheme for differential evolutionary algorithm in solving chemical engineering optimization problem. J Ind Eng Chem 16(4):620–628CrossRefGoogle Scholar
  33. 33.
    Ku ML, Chen Y, Liu KJR (2015) Data-driven stochastic models and policies for energy harvesting sensor communications. IEEE J Sel Areas Commun 33(8):1505–1520. Google Scholar
  34. 34.
    Liang JJ, Qin AK, Suganthan PN, Baskar S (2006) Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Trans Evol Comput 10(3):281–295CrossRefGoogle Scholar
  35. 35.
    Liang JJ, Qu BY, Suganthan PN (2013) Problem definitions and evaluation criteria for the CEC 2014 special session and competition on single objective real-parameter numerical optimization. In: Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou China and Technical Report, Nanyang Technological University, SingaporeGoogle Scholar
  36. 36.
    Liu J, Lampinen J (2005) A fuzzy adaptive differential evolution algorithm. Soft Comput 9(6):448–462CrossRefzbMATHGoogle Scholar
  37. 37.
    Liu ZZ, Wang Y, Yang S, Cai Z (2016) Differential evolution with a two-stage optimization mechanism for numerical optimization. In: 2016 IEEE congress on evolutionary computation (CEC) IEEEGoogle Scholar
  38. 38.
    Lou H-L (1995) Implementing the Viterbi algorithm. IEEE Signal Process Mag 12(5):42–52CrossRefGoogle Scholar
  39. 39.
    Mahfoud SW (1993) Finite Markov chain models of an alternative selection strategy for the genetic algorithm. Complex Syst 7(2):155zbMATHGoogle Scholar
  40. 40.
    Mallipeddi R, Suganthan PN, Pan QK, Tasgetiren MF (2011) Differential evolution algorithm with ensemble of parameters and mutation strategies. Appl Soft Comput 11(2):1679–1696CrossRefGoogle Scholar
  41. 41.
    Mohamed MA, Gader P (2000) Generalized hidden Markov models. I. Theoretical frameworks. IEEE Trans Fuzzy Syst 8(1):67–81CrossRefGoogle Scholar
  42. 42.
    Morimoto H (2016) Hidden Markov models and self-organizing maps applied to stroke incidence. Open J Appl Sci 6(03):158CrossRefGoogle Scholar
  43. 43.
    Onwubolu G, Davendra D (2006) Scheduling flow shops using differential evolution algorithm. Eur J Oper Res 171(2):674–692CrossRefzbMATHGoogle Scholar
  44. 44.
    Qin AK, Huang VL, Suganthan PN (2009) Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans Evol Comput 13(2):398–417CrossRefGoogle Scholar
  45. 45.
    Rabiner L, Juang B (1986) An introduction to hidden Markov models. IEEE ASSP Mag 3(1):4–16CrossRefGoogle Scholar
  46. 46.
    Jackie Rees, Koehler GJ (2006) Learning genetic algorithm parameters using hidden Markov models. Eur J Oper Res 175(2):806–820CrossRefzbMATHGoogle Scholar
  47. 47.
    Regulwar DG, Choudhari SA, Raj PA (2010) Differential evolution algorithm with application to optimal operation of multipurpose reservoir. J Water Resour Protect 2(06):560CrossRefGoogle Scholar
  48. 48.
    Ronkkonen J, Kukkonen S, Price KV (2005) Real-parameter optimization with differential evolution. Proc IEEE CEC 1:506–513Google Scholar
  49. 49.
    Rudolph G (1998) Finite Markov chain results in evolutionary computation: a tour d’horizon. Fundam Inf 35(1–4):67–89MathSciNetzbMATHGoogle Scholar
  50. 50.
    Storn R (1996) On the usage of differential evolution for function optimization. In: Fuzzy Information Processing Society. NAFIPS. 1996 Biennial Conference of the North American IEEE, pp 519–523Google Scholar
  51. 51.
    Storn R, Price K (1995) Differential evolution—a simple and efficient adaptive scheme for global optimization over continuous spaces. Bd. 3. ICSI BerkeleyGoogle Scholar
  52. 52.
    Suganthan PN, Hansen N, Liang JJ, Deb K, Chen YP, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Nanyang Technological University, Singapore; IIT Kanpur Kanpur, India, ForschungsberichtGoogle Scholar
  53. 53.
    Tanabe R, Fukunaga AS (2014) Improving the search performance of SHADE using linear population size reduction. In: 2014 IEEE congress on evolutionary computation (CEC) IEEE, pp 1658–1665Google Scholar
  54. 54.
    Tang L, Dong Y, Liu J (2015) Differential evolution with an individual-dependent mechanism. IEEE Trans Evol Comput 19(4):560–574CrossRefGoogle Scholar
  55. 55.
    Veček N, Črepinšek M, Mernik M (2017) On the influence of the number of algorithms, problems, and independent runs in the comparison of evolutionary algorithms. Appl Soft Comput 54:23–45CrossRefGoogle Scholar
  56. 56.
    Veček N, Mernik M, Črepinšek M (2014) A chess rating system for evolutionary algorithms: a new method for the comparison and ranking of evolutionary algorithms. Inf Sci 277:656–679MathSciNetCrossRefGoogle Scholar
  57. 57.
    Wang Y, Cai Z, Zhang Q (2011) Differential evolution with composite trial vector generation strategies and control parameters. IEEE Trans Evol Comput 15(1):55–66CrossRefGoogle Scholar
  58. 58.
    Wang Y, Li HX, Huang T, Li L (2014) Differential evolution based on covariance matrix learning and bimodal distribution parameter setting. Appl Soft Comput 18:232–247CrossRefGoogle Scholar
  59. 59.
    Wang Y, Liu ZZ, Li J, Li HX, Yen GG (2016) Utilizing cumulative population distribution information in differential evolution. Appl Soft Comput 48:329–346CrossRefGoogle Scholar
  60. 60.
    Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82CrossRefGoogle Scholar
  61. 61.
    Yu WJ, Shen M, Chen WN, Zhan ZH, Gong YJ, Lin Y, Liu O, Zhang J (2014) Differential evolution with two-level parameter adaptation. IEEE Trans Cybern 44(7):1080–1099CrossRefGoogle Scholar
  62. 62.
    Zaman MF, Elsayed SM, Ray T, Sarker RA (2016) Evolutionary algorithms for dynamic economic dispatch problems. IEEE Trans Power Syst 31(2):1486–1495. CrossRefGoogle Scholar
  63. 63.
    Zhang J, Sanderson AC (2009) JADE: adaptive differential evolution with optional external archive. IEEE Trans Evol Comput 13(5):945–958CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of New South Wales-CanberraCanberraAustralia

Personalised recommendations