Journal of Computer Science and Technology

, Volume 27, Issue 5, pp 1024–1034 | Cite as

Classification- and Regression-Assisted Differential Evolution for Computationally Expensive Problems

  • Xiao-Fen LuEmail author
  • Ke Tang
Regular Paper


Differential Evolution (DE) has been well accepted as an effective evolutionary optimization technique. However, it usually involves a large number of fitness evaluations to obtain a satisfactory solution. This disadvantage severely restricts its application to computationally expensive problems, for which a single fitness evaluation can be highly time-consuming. In the past decade, a lot of investigations have been conducted to incorporate a surrogate model into an evolutionary algorithm (EA) to alleviate its computational burden in this scenario. However, only limited work was devoted to DE. More importantly, although various types of surrogate models, such as regression, ranking, and classification models, have been investigated separately, none of them consistently outperforms others. In this paper, we propose to construct a surrogate model by combining both regression and classification techniques. It is shown that due to the specific selection strategy of DE, a synergy can be established between these two types of models, and leads to a surrogate model that is more appropriate for DE. A novel surrogate model-assisted DE, named Classification- and Regression-Assisted DE (CRADE) is proposed on this basis. Experimental studies are carried out on a set of 16 benchmark functions, and CRADE has shown significant superiority over DE-assisted with only regression or classification models. Further comparison to three state-of-the-art DE variants, i.e., DE with global and local neighborhoods (DEGL), JADE, and composite DE (CoDE), also demonstrates the superiority of CRADE.


surrogate model differential evolution computationally expensive problem 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

11390_2012_1282_MOESM1_ESM.docx (16 kb)
(DOC 15.8 kb)


  1. [1]
    Storn R, Price K. Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 1997, 11(4): 341–359.MathSciNetzbMATHCrossRefGoogle Scholar
  2. [2]
    Price K, Storn R M, Lampinen J A. Differential Evolution: A Practical Approach to Global Optimization. New York, USA: Springer-Verlag, 2005.Google Scholar
  3. [3]
    Jin Y, Olhofer M, Sendhoff B. Managing approximate models in evolutionary aerodynamic design optimization. In Proc. the 2001 IEEE Congress on Evolutionary Computation (CEC2001), Seoul, Korea, May 2001, pp.592–599.Google Scholar
  4. [4]
    Ong Y S, Nair P B, Keane A J. Evolutionary optimization of computationally expensive problems via surrogate modeling. AIAA Journal, 2003, 41(4): 687–696.CrossRefGoogle Scholar
  5. [5]
    Zhang P, Yao X, Jia L, Sendhoff B, Schnier T. Target shape design optimization by evolving splines. In Proc. CEC2007, Singapore, Sept. 2007, pp.2009–2016.Google Scholar
  6. [6]
    Farina M, Sykulski J. Comparative study of evolution strategies combined with approximation techniques for practical electromagnetic optimization problems. IEEE Transactions on Magnetics, 2002, 37(5): 3216–3220.CrossRefGoogle Scholar
  7. [7]
    Hajela P, Lee J. Genetic algorithms in multidisciplinary rotor blade design. In Proc. the 36th Conference on Structures, Structural Dynamics, and Materials, New Orleans, USA, April 1995, pp.2187–2197.Google Scholar
  8. [8]
    Shi L, Rasheed K. A survey of fitness approximation methods applied in evolutionary algorithms. Computational Intelligence in Expensive Optimization Problems, 2010, 2(1): 3–28.CrossRefGoogle Scholar
  9. [9]
    Jin Y, Olhofer M, Sendhoff B. A framework for evolutionary optimization with approximate fitness functions. IEEE Transactions on Evolutionary Computation, 2002, 6(5): 481–494.CrossRefGoogle Scholar
  10. [10]
    Jin Y. A comprehensive survey of fitness approximation in evolutionary computation. Soft Computing, 2005, 9(1): 3–12.CrossRefGoogle Scholar
  11. [11]
    Jin Y, Olhofer M, Sendhoff B. On evolutionary optimization with approximate fitness functions. In Proc. Genetic and Evolutionary Computation Conference, Las Vegas, Nevada, USA, July 2000, pp.786–793.Google Scholar
  12. [12]
    Buche D, Schraudolph N N, Koumoutsakos P. Accelerating evolutionary algorithms with Gaussian process fitness function models. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2005, 35(2): 183–194.CrossRefGoogle Scholar
  13. [13]
    Runarsson T. Ordinal regression in evolutionary computation. In Proc. the 9th Int. Conf. Parallel Problem Solving from Nature-PPSN IX, Reykjavik, Iceland, Sept. 2006, pp.1048–1057.Google Scholar
  14. [14]
    Loshchilov I, Schoenauer M, Sebag M. Comparison-based optimizers need comparison-based surrogates. In Proc. the 11th Int. Conf. Parallel Problem Solving from Nature-PPSN XI, Krakov, Poland, September 2010, pp.364–373.Google Scholar
  15. [15]
    Handoko S D, Kwoh C K, Ong Y S. Using classification for constrained memetic algorithm: A new paradigm. In Proc. IEEE International Conference on Systems, Man and Cybernetics. Suntec, Singapore, Oct. 2008, pp.547–552.Google Scholar
  16. [16]
    Handoko S D, Kwoh C K, Ong Y S. Classification-assisted memetic algorithms for equality-constrained optimization problems. In Proc. the 22nd AI, Melbourne, Australia, May 2009, pp.391–400.Google Scholar
  17. [17]
    Lim D, Ong Y S, Setiawan R, Idris M. Classifier-assisted constrained evolutionary optimization for automated geometry selection of orthodontic retraction spring. In Proc. the 2010 IEEE Congress on Evolutionary Computation (CEC2010), Barcelona, Spain, July 2010, pp.1–8.Google Scholar
  18. [18]
    Handoko S D, Kwoh C K, Ong Y S. Feasibility structure modeling: An effective chaperone for constrained memetic algorithms. IEEE Transactions on Evolutionary Computation, 2010, 14(5): 740–758.CrossRefGoogle Scholar
  19. [19]
    Yang Z, Tang K, Yao X. Self-adaptive differential evolution with neighborhood search. In Proc. the 2008 IEEE Congress on Evolutionary Computation (CEC2008), Hong kong, China, June 2008, pp. 1110–1116.Google Scholar
  20. [20]
    Das S, Abraham A, Chakraborty U K, Konar A. Differential evolution using a neighborhood-based mutation operator. IEEE Transactions on Evolutionary Computation, 2009, 13(3): 526–553.CrossRefGoogle Scholar
  21. [21]
    Zhang J, Sanderson A C. JADE: Adaptive differential evolution with optional external archive. IEEE Transactions on Evolutionary Computation, 2009, 13(5): 945–958.CrossRefGoogle Scholar
  22. [22]
    Yang Z, Tang K, Yao X. Scalability of generalized adaptive differential evolution for large-scale continuous optimization. Soft Computing, 2011, 15(11): 2141–2155.CrossRefGoogle Scholar
  23. [23]
    Wang Y, Cai Z, Zhang Q. Differential evolution with composite trial vector generation strategies and control parameters. IEEE Transactions on Evolutionary Computation, 2011, 15(1): 55–66.MathSciNetCrossRefGoogle Scholar
  24. [24]
    Wang Y, Cai Z, Zhang Q. Enhancing the search ability of differential evolution through orthogonal crossover. Information Sciences, 2012, 185(1): 153–177.MathSciNetCrossRefGoogle Scholar
  25. [25]
    Zhang J, Sanderson A. DE-AEC: A differential evolution algorithm based on adaptive evolution control. In Proc. the 2007 IEEE Congress on Evolutionary Computation (CEC2007), Singapore, Sept. 2007, pp.3824–3830.Google Scholar
  26. [26]
    Wang Y, Shi Y, Yue B, Teng H. An efficient differential evolution algorithm with approximate fitness functions using neural networks. In Proc. the 2010 Int. Conf. Artificial Intelligence and Computational Intelligence, Part 2, Oct. 2010, pp.334–341.Google Scholar
  27. [27]
    Lu X, Tang K, Yao, X. Classification-assisted differential evolution for computationally expensive problems. In Proc. the 2011 IEEE Congress on Evolutionary Computation (CEC2011), New Orleans, USA, June 2011, pp.1986–1993.Google Scholar
  28. [28]
    Lim D, Jin Y, Ong Y S, Sendhoff B. Generalizing surrogateassisted evolutionary computation. IEEE Transactions on Evolutionary Computation, 2010, 14(3): 329–355.CrossRefGoogle Scholar
  29. [29]
    Suganthan P N, Hansen N, Liang J J, Deb K, Chen Y P, Auger A, Tiwari S. Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Technical Report, Nanyang Technological University, Singapore, and KanGAL Report, Kanpur Genetic Algorithms Laboratory, IITkanpur, 2005.Google Scholar
  30. [30]
    Tang K, Yao X, Suganthan P N, MacNish C, Chen Y P, Chen C M, Yang Z. Benchmark functions for the CEC2008 special session and competition on large scale global optimization. Technical Report, Nature Inspired Comput. Applicat. Lab., USTC, China, 2007.
  31. [31]
    Duin R, Juszczak P, Paclik P, Pekalska E, Ridder D de, Tax D M J, Verzakov S. PRTools 4.1, a matlab toolbox for pattern recognition. Delft University of Technology, 2007.Google Scholar
  32. [32]
    Yu H, Kim S. SVM tutorial: Classification, regression, and ranking. In Handbook of Natural Computing, Rozenderg G, Bäck T, Kok J (eds.), Springer 2009.Google Scholar
  33. [33]
    Gunn S. Support vector machines for classification and regression. Technical Report, University of Southampton, 1998.Google Scholar

Copyright information

© Springer Science+Business Media New York & Science Press, China 2012

Authors and Affiliations

  1. 1.Nature Inspired Computation and Applications LaboratorySchool of Computer Science and Technology University of Science and Technology of ChinaHefeiChina

Personalised recommendations