Advertisement

Basics of Differential Evolution

  • Anyong Qing
Chapter
  • 1.2k Downloads
Part of the Evolutionary Learning and Optimization book series (ALO, volume 4)

A Short History

Inception

Differential evolution was proposed by K.V. Price and R. Storn in 1995 [1]. At that time, Price was asked to solve the Chebyshev polynomial fitting problem [1]-[5] by Storn [2], [5]. Initially, he tried to solve it by using genetic annealing algorithm [6]. However, although he eventually found the solution to the 5-dimensional Chebyshev polynomial fitting problem by using genetic annealing algorithm, he was frustrated to notice that genetic annealing algorithm fails to fulfill the three requirements for a practical optimization technique: strong global search capability, fast convergence, and user friendliness.

A breakthrough happened when Price came up with an innovative scheme for generating trial parameter vectors. In this scheme, a new parameter vector is generated by adding the weighted difference vector between two population members to a third member. Such a scheme was named as differential mutation and has been well known to be the crucial idea behind the success of differential evolution. The cornerstone for differential evolution was therefore laid.

Price wrapped up his invention with other critical ideas: natural real code, arithmetic operations, mother-child competition and selection, and execution of evolutionary operations in the order of mutation-crossover-selection. Consequently, differential evolution, a very reliable, efficient, robust, and simple evolutionary algorithm was developed.

Keywords

Differential Evolution Differential Evolution Algorithm General Regression Neural Network Arithmetic Crossover Hybrid Differential Evolution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Storn, R., Price, K.V.: Differential Evolution - A Simple and Efficient Adaptive Scheme for Global Optimization over Continuous Spaces, Technical Report TR-95-012, International Computer Science Insitute (March 1995)Google Scholar
  2. 2.
    Price, K.V.: Differential evolution vs. the functions of the 2nd ICEO. In: 1997 IEEE Int. Conf. Evolutionary Computation, Indianapolis, IN, April 13-16, pp. 153–157 (1997)Google Scholar
  3. 3.
    Price, K.V., Storn, R.M., Lampinen, J.A.: Differential Evolution: a Practical Approach to Global Optimization. Springer, Berlin (2005)zbMATHGoogle Scholar
  4. 4.
    Qing, A.: Differential Evolution: Fundamentals and Applications in Electrical Engineering. John Wiley, New York (2009)Google Scholar
  5. 5.
    Storn, R.: Differential evolution (DE) for continuous function optimization (an algorithm by Kenneth Price and Rainer Storn) (2009), http://www.icsi.berkeley.edu/~storn/code.html (last accessed on October 23, 2009)
  6. 6.
    Price, K.V.: Genetic annealing. Dr. Bobb’s J. 19(10), 127–132 (1994)Google Scholar
  7. 7.
    Price, K.V.: Differential evolution: a fast and simple numerical optimizer. In: 1996 Biennial Conf. North American Fuzzy Information Processing Society, Berkeley, CA, June 19-22, pp. 524–527 (1996)Google Scholar
  8. 8.
    Storn, R., Price, K.V.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optimization 11(4), 341–359 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Brutovský, B., Ulicný, J., Miškovský, P.: Application of genetic algorithms based techniques in the theoretical analysis of molecular vibrations. In: 1st Int. Conf. Genetic Algorithms Occasion 130th Anniversary Mendel’s Laws in Brno, Brno, Czech Republic, September 26-28, pp. 29–33 (1995)Google Scholar
  10. 10.
    Storn, R., Price, K.V.: Minimizing the real functions of the ICEC 1996 contest by differential evolution. In: 1996 IEEE Int. Conf. Evolutionary Computation, Nagoya, May 20-22, pp. 842–844 (1996)Google Scholar
  11. 11.
    Storn, R.: Modeling and Optimization of PET-Redundancy Assignment for MPEG-Sequences, Technical Report TR-95-018, International Computer Science Institute (May 1995)Google Scholar
  12. 12.
    Storn, R.: Differential Evolution Design of an IIR-Filter with Requirements for Magnitude and Group Delay, Technical Report TR-95-026, International Computer Science Institute (June 1995)Google Scholar
  13. 13.
    Storn, R.: Differential evolution design of an IIR-filter. In: 1996 IEEE Int. Conf. Evolutionary Computation, Nagoya, May 20-22, pp. 268–273 (1996)Google Scholar
  14. 14.
    Storn, R.: On the usage of differential evolution for function optimization. In: 1996 Biennial Conf. North American Fuzzy Information Processing Society, Berkeley, CA, June 19-22, pp. 519–523 (1996)Google Scholar
  15. 15.
    Storn, R.: System Design by Constraint Adaptation and Differential Evolution, Technical Report TR-96-039, International Computer Science Institute (November 1996)Google Scholar
  16. 16.
    Joshi, R., Sanderson, A.C.: Multisensor fusion and model selection us-ing a minimal representation size framework. In: 1996 IEEE/SICE/RSJ Int. Conf. Multisensor Fusion Integration Intelligent Systems, Washington, DC, Deemeber 8-11, pp. 25–32 (1996)Google Scholar
  17. 17.
    Chiou, J.P., Wang, F.S.: Hybrid differential evolution for parameter estimation of a batch bioprocess. In: IEEE Int. Symp. Control Theory Applications, Singapore, July 29-30, pp. 171–174 (1997)Google Scholar
  18. 18.
    Fleiner, C.: Parallel Optimizations: Advanced Constructs and Compiler Optimizations for a Parallel, Object Oriented, Shared Memory Language Running on a Distributed System, Ph. D. Thesis, University of Fribourg (April 11, 1997)Google Scholar
  19. 19.
    Joshi, R., Sanderson, A.C.: Experimental studies on minimal representation multisensor fusion. In: 8th Int. Conf. Advanced Robotics, Monterey, CA, July 7-9, pp. 603–610 (1997a)Google Scholar
  20. 20.
    Joshi, R., Sanderson, A.C.: Minimal representation multisensor fusion using differential evolution. In: 1997 IEEE Int. Symp. Computational Intelligence Robotics Automation, Monterey, CA, July 10-11, pp. 266–273 (1997b)Google Scholar
  21. 21.
    Joshi, R., Sanderson, A.C.: Multisensor fusion of touch and vision using minimal representation size. In: 1997 IEEE/RSJ Int. Conf. Intelligent Robots Systems, Grenoble, September 7-11, vol. 3, pp. v4–v5 (1997c)Google Scholar
  22. 22.
    Masters, T., Land, W.: A new training algorithm for the general regression neural network. In: 1997 IEEE Int. Conf. Systems Man Cybernetics, Orlando, FL, October 12-15, vol. 3, pp. 1990–1994 (1997)Google Scholar
  23. 23.
    Michael, C., McGraw, G.: Opportunism and Diversity in Automated Software Test Data Generation, Technical Report RSTR-003-97-13, ver-sion 1.3, RST Corporation, Sterling, VA, USA (December 8, 1997)Google Scholar
  24. 24.
    Rüttgers, M.: Differential Evolution: A Method for Optimization of Real Scheduling Problems, Technical Report TR-97-013, International Computer Science Institute (March 1997)Google Scholar
  25. 25.
    Rüttgers, M.: Design of a method for machine scheduling for core blowers in foundries. In: Reusch, B. (ed.) Fuzzy Days 1997. LNCS, vol. 1226, p. 572. Springer, Heidelberg (1997)Google Scholar
  26. 26.
    Rüttgers, M.: Design of a new algorithm for scheduling in parallel machine shops. In: 1997 5th European Congress Intelligent Techniques Soft Computing, Aachen, Germany, September 8-11, vol. 3, pp. 2182–2187 (1997)Google Scholar
  27. 27.
    Thomas, P., Vernon, D.: Image registration by differential evolution. In: 1st Irish Machine Vision Image Processing Conf., Magee College, University of Ulster, pp. 221–225 (1997)Google Scholar
  28. 28.
    Wang, F.S., Chiou, J.P.: Differential evolution for dynamic optimization of differential algebraic systems. In: 1997 IEEE Int. Conf. Evolutionary Computation, Indianapolis, IN, April 13-16, pp. 531–536 (1997)Google Scholar
  29. 29.
    Wang, F.S., Chiou, J.P.: Optimal control and optimal time location problems of differential-algebraic systems by differential evolution. Industrial Engineering Chemistry Research 36(12), 5348–5357 (1997)CrossRefGoogle Scholar
  30. 30.
    Price, K., Storn, R.: Differential evolution: a simple evolution strategy for fast optimization. Dr. Dobb’s J. 22(4), 18–24, 78 (1997)MathSciNetGoogle Scholar
  31. 31.
    Chang, T.T., Chang, H.C.: Application of differential evolution to passive shunt harmonic filter planning. In: 8th Int. Conf. Harmonics Quality Power, Athens, Greece, October 14-16, vol. 1, pp. 149–153 (1998)Google Scholar
  32. 32.
    Meyer, M.: Construction of a multi-purpose X-ray CCD detector and its implementation on a 4-circle kappa goniometer, Ph. D. Thesis, l’Université de Lausanne (1998)Google Scholar
  33. 33.
    Mastorakis, N.E. (ed.): Recent Advances in Circuits and Systems. World Scientific, Singapore (1998)Google Scholar
  34. 34.
    Corn, D., Dorigo, M., Glover, F. (eds.): New Ideas in Optimization. McGraw-Hill, London (1999)Google Scholar
  35. 35.
    Qing, A.: Dynamic differential evolution strategy and applications in electromagnetic inverse scattering problems. IEEE Trans. Geosci. Remote Sens. 44(1), 116–125 (2006)CrossRefGoogle Scholar
  36. 36.
    Bergey, P.K.: An agent enhanced intelligent spreadsheet solver for multi-criteria decision making. In: 1999 Americas Conf. Information Systems, Milwaukee, August 13-15, pp. 966–968 (1999)Google Scholar
  37. 37.
    Chang, C.S., Xu, D.Y., Quek, H.B.: Pareto-optimal set based multiobjective tuning of fuzzy automatic train operation for mass transit system. IEE Proc. B-Electric Power Applications 146(5), 577–583 (1999)CrossRefGoogle Scholar
  38. 38.
    Rigling, B.D., Moore, F.W.: Exploitation of sub-populations in evolution strategies for improved numerical optimization. In: 10th Midwest Artificial Intelligence Cognitive Science Conf., Bloomington, Indiana, April 23-25, pp. 80–88 (1999)Google Scholar
  39. 39.
    Lee, M.H., Han, C., Chang, K.S.: Dynamic optimization of a continuous polymer reactor using a modified differential evolution algorithm. Industrial Engineering Chemistry Research 38(12), 4825–4831 (1999)CrossRefGoogle Scholar
  40. 40.
    Michalski, K.A.: Electromagnetic imaging of circular-cylindrical conductors and tunnels using a differential evolution algorithm. Microwave Optical Technology Letters 27(5), 330–334 (2000)CrossRefGoogle Scholar
  41. 41.
    Babu, B.V., Chaturvedi, G.: Evolutionary computation strategy for optimization of an alkylation reaction. In: Int. Symp. 53rd Annual Session IIChE, Science City, Calcutta, December 18-21 (2000)Google Scholar
  42. 42.
    Babu, B.V., Munawar, S.A.: Differential evolution for the optimal design of heat exchangers. In: All India Seminar Chemical Engineering Progress Resource Development: A Vision 2010 Beyond, Orissa State Center, Bhuvaneshwar (March 13, 2000)Google Scholar
  43. 43.
    Pahner, U., Hameyer, K.: Adaptive coupling of differential evolution and multiquadrics approximation for the tuning of the optimization process. IEEE Trans. Magnetics 36(4), 1047–1051 (2000)CrossRefGoogle Scholar
  44. 44.
    Lampinen, J.: A bibliography on differential evolution algorithm, Technical Report, Lappeenranta University of Technology, Department of Information Technology, Laboratory of Information Processing (2001) (last updated on October 14, 2002), available via internet, http://www2.lut.fi/~jlampine/debiblio.htm (accessed on October 12, 2009)
  45. 45.
    Lampinen, J.: Solving problems subject to multiple nonlinear constraints by the differential evolution. In: 7th Int. Conf. Soft Computing, Brno, Czech Republic, June 6-8, pp. 50–57 (2001)Google Scholar
  46. 46.
    Angira, R., Babu, B.V.: Non-dominated sorting differential evolution (NSDE): an extension of differential evolution for multi-objective optimization. In: 2nd Indian Int. Conf. Artificial Intelligence, Pune, India, December 20-22, pp. 1428–1443 (2005)Google Scholar
  47. 47.
    Qin, A.K., Suganthan, P.N.: Self-adaptive differential evolution algorithm for numerical optimization. In: 2005 IEEE Congress Evolutionary Computation, Edinburgh, UK, September 2-5, vol. 2, pp. 1785–1791 (2005)Google Scholar
  48. 48.
    Rahnamayan, S., Tizhoosh, H.R., Salama, M.M.A.: Opposition-based differential evolution for optimization of noisy problems. In: 2006 IEEE Congress Evolutionary Computation, Vancouver, Canada, July 16-21, pp. 1865–1872 (2006)Google Scholar
  49. 49.
    Rahnamayan, S., Tizhoosh, H.R., Salama, M.M.A.: Opposition-based differential evolution algorithms. In: 2006 IEEE Congress Evolutionary Computation, Vancouver, Canada, July 16-21, pp. 2010–2017 (2006)Google Scholar
  50. 50.
    Zaharie, D.: A comparative analysis of crossover variants in differential evolution. In: Int. Multiconference Computer Science Information Technology, pp. 171–181 (2007)Google Scholar
  51. 51.
    Lawson, K.: Darwin and Evolution for Kids: His Life and Ideas with 21 Activities. Chicago Review Press, Chicago (2003)Google Scholar
  52. 52.
    Chen, C.W., Chen, D.Z., Cao, G.Z.: An improved differential evolution algorithm in training and encoding prior knowledge into feedforward networks with application in chemistry. Chemometrics Intelligent Laboratory Systems 64(1), 27–43 (2002)CrossRefGoogle Scholar
  53. 53.
    Chakraborty, U.K. (ed.): Advances in Differential Evolution. Springer, Berlin (March 2008)zbMATHGoogle Scholar
  54. 54.
    Fan, H.Y., Lampinen, J.: A trigonometric mutation operation to differential evolution. J. Global Optimization 27, 105–129 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  55. 55.
    Fan, H.Y., Lampinen, J.: A directed mutation operation for the differential evolution algorithm. Int. J. Industrial Engineering-Theory Applications Practice 10(1), 6–15 (2003)MathSciNetGoogle Scholar
  56. 56.
    Fischer, M.M., Hlavackova-Schindler, K., Reismann, M.: An evolutionary mutation-based algorithm for weight training in neural networks for telecommunication flow modelling, Computational Intelligence Modelling, Control Automation. In: Evolutionary Computation and Fuzzy Logic for Intelligent Control, Knowledge Acquisition and Information Retrieval, Vienna, Austria, Febuary 17-19. Concurrent Systems Engineering Series, vol. 55, pp. 54–59 (1999)Google Scholar
  57. 57.
    Qing, A.: A parametric study on differential evolution based on benchmark electromagnetic inverse scattering problem. In: 2007 IEEE Congress Evolutionary Computation, Singapore, September 25-28, pp. 1904–1909 (2007)Google Scholar
  58. 58.
    Qing, A.: A study on base vector for differential evolution. In: 2008 IEEE World Congress Computational Intelligence/2008 IEEE Congress Evolutionary Computation, Hong Kong, June 1-6, pp. 550–556 (2008)Google Scholar
  59. 59.
    Lampinen, J., Zelinka, I.: Mixed variable non-linear optimization by differential evolution. In: 2nd Int. Prediction Conf., Zlin, Czech Republic, October 7-8, pp. 45–55 (1999)Google Scholar
  60. 60.
    Krink, T., Filipič, B., Fogel, G.B., Thomsen, R.: Noisy optimization problems - a particular challenge for differential evolution? In: 2004 IEEE Congress Evolutionary Computation, Portland, OR, June 19-23, vol. 1, pp. 332–339 (2004)Google Scholar
  61. 61.
    Bindal, A., Ierapetritou, M.G., Balakrishnan, S., Armaou, A., Makeev, A.G., Kevrekidis, I.G.: Equation-free, coarse-grained computational optimization using timesteppers. Chemical Engineering Science 61(2), 779–793 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Anyong Qing
    • 1
  1. 1.Temasek LaboratoriesNational University of SingaporeSingapore

Personalised recommendations

Cite chapter

Buy options