Structural and Multidisciplinary Optimization

, Volume 57, Issue 4, pp 1553–1577 | Cite as

Surrogate-based optimization with clustering-based space exploration for expensive multimodal problems

  • Huachao Dong
  • Baowei Song
  • Peng Wang
  • Zuomin Dong


This paper presents a surrogate-based global optimization algorithm to solve multimodal expensive black-box optimization problems (EBOPs) with or without expensive nonlinear constraints. Two approximation methods (kriging and quadratic response surfaces, QRS) are used to construct surrogate models, among which kriging can predict multiple promising local optima and QRS can reflect the overall trend of a true model. According to their characteristics, two different optimizers are employed to capture the promising samples on kriging and QRS, respectively. One is the nature-inspired algorithm “Grey wolf optimization (GWO)”, which can efficiently find the global optimum of a QRS model. The other one is a multi-start optimization algorithm that can find several different local optimal locations from a kriging model. In addition, the complete optimization flow is presented and its detailed pseudo code is given. In the presented optimization flow, if a proposed local convergence criterion is satisfied, sparsely sampled regions will be explored. Such a space exploration strategy is developed based on the k-means clustering algorithm, which can make search jump out of a local optimal location and focus on unexplored regions. Furthermore, two penalty functions are proposed to make this algorithm applicable for constrained optimization. With tests on 15 bound constrained and 7 nonlinear constrained benchmark examples, the presented algorithm shows remarkable capacity in dealing with multimodal EBOPs and constrained EBOPs.


Kriging model Quadratic response surface Clustering-based space exploration Multimodal problems Nonlinear constrained optimization 



Supports from Natural Sciences and Engineering Research Council of Canada, and National Natural Science Foundation of China (Grant No. 51375389) are gratefully acknowledged. The authors are also grateful to members of the research group for the implementation of some existing global optimization algorithms and benchmark test cases.

Supplementary material


  1. Alexandrov NM, Dennis JEJ, Lewis RM, Torczon V (1998) A trust-region framework for managing the use of approximation modelsin optimization. Struct Optim 15(1):16–23CrossRefGoogle Scholar
  2. Beasley JE, Chu PC (1996) A genetic algorithm for the set covering problem. Eur J Oper Res 94(2):392–404CrossRefzbMATHGoogle Scholar
  3. Boggs PT, Tolle JW (1995) Sequential quadratic programming. Acta Numer 4:1–51MathSciNetCrossRefzbMATHGoogle Scholar
  4. Coello CAC (2002) Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art. Comput Methods Appl Mech Eng 191(11):1245–1287MathSciNetCrossRefzbMATHGoogle Scholar
  5. Cutbill A, Wang GG (2016) Mining constraint relationships and redundancies with association analysis for optimization problem formulation. Eng Optim 48(1):115–134MathSciNetCrossRefGoogle Scholar
  6. Deshmukh AP, Allison JT (2016) Multidisciplinary dynamic optimization of horizontal axis wind turbine design. Struct Multidiscip Optim 53(1):15–27MathSciNetCrossRefGoogle Scholar
  7. Diaz-Manriquez A, Toscano-Pulido G, Gomez-Flores W (2011) On the selection of surrogate models in evolutionary optimization algorithms. IEEE Congr. Evol. Comput. 2155–2162Google Scholar
  8. Gu J, Li GY, Dong Z (2012) Hybrid and adaptive meta-model-based global optimization. Eng Optim 44(1):87–104CrossRefGoogle Scholar
  9. Gutmann HM (2001) A radial basis function method for global optimization. J Glob Optim 19(3):201–227MathSciNetCrossRefzbMATHGoogle Scholar
  10. Haftka RT, Villanueva D, Chaudhuri A (2016) Parallel surrogate-assisted global optimization with expensive functions–a survey. Struct Multidiscip Optim 54(1):3–13MathSciNetCrossRefGoogle Scholar
  11. Hartigan JA, Wong MA (1979) Algorithm AS 136: A k-means clustering algorithm. J R Stat Soc: Ser C Appl Stat 28(1):100–108zbMATHGoogle Scholar
  12. Jamil M, Yang XS (2013) A literature survey of benchmark functions for global optimisation problems. International Journal of Mathematical Modelling and Numerical Optimisation 4(2):150–194CrossRefzbMATHGoogle Scholar
  13. Jie H, Wu Y, Ding J (2015) An adaptive metamodel-based global optimization algorithm for black-box type problems. Eng Optim 47(11):1459–1480MathSciNetCrossRefGoogle Scholar
  14. Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492MathSciNetCrossRefzbMATHGoogle Scholar
  15. Kenny QY, Li W, Sudjianto A (2000) Algorithmic construction of optimal symmetric Latin hypercube designs. J Stat Plan Inference 90(1):145–159MathSciNetCrossRefzbMATHGoogle Scholar
  16. Krityakierne T, Akhtar T, Shoemaker CA (2016) SOP: parallel surrogate global optimization with Pareto center selection for computationally expensive single objective problems. Journal of Global Optimization, pp: 1–21Google Scholar
  17. Leifsson L, Koziel S (2016) Surrogate modelling and optimization using shape-preserving response prediction: A review. Eng Optim 48(3):476–496CrossRefGoogle Scholar
  18. Long T, Wu D, Guo X et al (2015) Efficient adaptive response surface method using intelligent space exploration strategy. Struct Multidiscip Optim 51(6):1335–1362CrossRefGoogle Scholar
  19. Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61CrossRefGoogle Scholar
  20. Montemayor-garcía G, Toscano-pulido G (2011) A Study of Surrogate models for their use in multiobjective evolutionary algorithms. In: 8th International Conference on Electrical Engineering Computing Science and Automatic Control (CCE)Google Scholar
  21. Myers RH, Montgomery DC (1995) Response Surface Methodology: Process and product in optimization using designed experiments. Wiley, New YorkzbMATHGoogle Scholar
  22. Regis RG (2014) Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points. Eng Optim 46(2):218–243MathSciNetCrossRefGoogle Scholar
  23. Regis RG, Shoemaker CA (2013a) Combining radial basis function surrogates dynamic coordinate search in high dimensional expensive black-box optimization. Eng Optim 45(5):529–555MathSciNetCrossRefGoogle Scholar
  24. Regis RG, Shoemaker CA (2013b) A quasi-multistart framework for global optimization of expensive functions using response surface models. J Glob Optim 56(4):1719–1753MathSciNetCrossRefzbMATHGoogle Scholar
  25. Sacks J, Welch WJ, Mitchell TJ, et al (1989) Design and analysis of computer experiments. Statistical science pp: 409–423Google Scholar
  26. Sadollah A, Eskandar H, Kim JH (2015) Water cycle algorithm for solving constrained multi-objective optimization problems. Appl Soft Comput 27:279–298CrossRefGoogle Scholar
  27. Shi Y, Eberhart RC (1998) Parameter selection in particle swarm optimization. International Conference on Evolutionary Programming. Springer, Berlin Heidelberg, pp 591–600Google Scholar
  28. Toropov VV, Filatov AA, Polynkin AA (1993) Multiparameter structural optimization using FEM and multipoint explicit approximations[J]. Structural optimization 6(1):7–14CrossRefGoogle Scholar
  29. Viana FAC, Haftka R, Watson L (2013) Efficient global optimization algorithm assisted by multiple surrogate techniques. J Glob Optim 56(2):669–689CrossRefzbMATHGoogle Scholar
  30. Wang GG, Simpson T (2004) Fuzzy clustering based hierarchical metamodeling for design space reduction and optimization. Eng Optim 36(3):313–335CrossRefGoogle Scholar
  31. Weise T, Wu Y, Chiong R, et al (2016) Global versus local search: the impact of population sizes on evolutionary algorithm performance. Journal of Global Optimization, pp: 1–24Google Scholar
  32. Xie S, Liang X, Zhou H et al (2016) Crashworthiness optimisation of the front-end structure of the lead car of a high-speed train. Struct Multidiscip Optim 53(2):339–347CrossRefGoogle Scholar
  33. Yang XS (2009) Harmony search as a metaheuristic algorithm. Music-inspired Harmony Search Algorithm. Springer, Berlin Heidelberg, pp 1–14CrossRefGoogle Scholar
  34. Yang XS (2010) A new metaheuristic bat-inspired algorithm. Nature inspired cooperative strategies for optimization (NICSO 2010). Springer, Berlin Heidelberg, pp 65–74CrossRefGoogle Scholar
  35. Yin H, Fang H, Wen G et al (2016) An adaptive RBF-based multi-objective optimization method for crashworthiness design of functionally graded multi-cell tube. Struct Multidiscip Optim 53(1):129–144MathSciNetCrossRefGoogle Scholar
  36. Zadeh PM, Toropov VV, Wood AS (2009) Metamodel-based collaborative optimization framework[J]. Struct Multidiscip Optim 38(2):103–115CrossRefGoogle Scholar
  37. Zeng F, Xie H, Liu Q et al (2016) Design and optimization of a new composite bumper beam in high-speed frontal crashes. Struct Multidiscip Optim 53(1):115–122CrossRefGoogle Scholar
  38. Zhang M, Luo W, Wang X (2008) Differential evolution with dynamic stochastic selection for constrained optimization. Inf Sci 178(15):3043–3074CrossRefGoogle Scholar
  39. Zhou Y, Haftka RT, Cheng G (2016) Balancing diversity and performance in global optimization. Struct Multidisc Optim 54(4): 1093–1105Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Huachao Dong
    • 1
    • 2
  • Baowei Song
    • 1
  • Peng Wang
    • 1
  • Zuomin Dong
    • 2
  1. 1.School of Marine Science and TechnologyNorthwestern Polytechnical UniversityXi’anChina
  2. 2.Department of Mechanical EngineeringUniversity of VictoriaVictoriaCanada

Personalised recommendations