# Ensemble of surrogates with hybrid method using global and local measures for engineering design

- 159 Downloads

## Abstract

Surrogate models are usually used as a time-saving approach to reduce the computational burden of expensive computer simulations for engineering design. However, it is difficult to choose an appropriate model for an unknown design space. To tackle this problem, an effective method is forming an ensemble model that combines several surrogate models. Many efforts were made to determine the weight factors of ensemble, which include global and local measures. This article investigates the characteristics of global and local measures, and presents a new ensemble model which combines the advantages of these two measures. In the proposed method, the design space is divided into two parts, and different strategies are introduced to evaluate the weight factors in these two parts respectively. The results from numerical and engineering design cases show that the proposed ensemble model has satisfactory robustness and accuracy (it performs best for most cases tested in this article), while spending almost the equivalent modeling time (the additional cost is not more than 6.7% for any case tested in this article) compared with the combined global and local ensemble models.

## Keywords

Ensemble model Global measure Local measure Surrogate models## Nomenclature

*d*Number of design variables.

*E*_{i}Root generalized mean square cross-validation error of the

*i*^{ th }surrogate.*e*_{ik}Cross-validation error of the

*i*^{ th }surrogate at the*k*^{ th }sample point.- \( {\widehat{f}}^{ens} \)
Predictor of the ensemble.

- \( {\widehat{f}}_i \)
Predictor of the

*i*^{ th }surrogate.*N*Number of test points.

*N*_{s}Number of surrogates used in the ensemble.

*n*Number of sample points.

*P*_{k}Ratio of the global cross-validation error to the local cross-validation error at the

*k*^{ th }sample point.*R*^{o}Outer region.

*R*^{i}Inner region.

*r*_{k}Radius of the

*k*^{ th }point’s inner region.- \( {r}_k^{\mathrm{max}} \)
Euclidean distance between the

*k*^{ th }sample point and the closest sample point.*S*Sample points set.

*WCVE*Weighted cross-validation error.

*w*_{i}Normalized weight of the

*i*^{ th }surrogate.- \( {w}_i^{\ast } \)
Unnormalized weight of the

*i*^{ th }surrogate.*w*_{ik}Pointwise weight of the

*i*^{ th }surrogate at the*k*^{ th }sample point.*x*^{nearest}Sample point which is nearest to the prediction point.

- \( {\widehat{y}}_{ik} \)
Response predicted by the

*i*^{ th }surrogate at the*k*^{ th }point, the surrogate is constructed by using leave-one-out cross-validation.*y*_{k}True response at the

*k*^{ th }sample/test point.- \( {\widehat{y}}_k \)
Prediction response at the

*k*^{ th }sample/test point.*ρ*Impact metric of local measure.

## Notes

### Acknowledgments

Financial support from the National Natural Science Foundation of China under Grant No. 51675198, 973 National Basic Research Program of China under Grant No. 2014CB046705 and National Natural Science Foundation of China under Grant No. 51421062 are gratefully acknowledged.

## References

- Acar E, Rais-Rohani M (2009) Ensemble of metamodels with optimized weight factors. Struct Multidiscip Optim 37(3):279–294CrossRefGoogle Scholar
- Acar E (2010) Various approaches for constructing an ensemble of metamodels using local measures. Struct Multidiscip Optim 42(6):879–896CrossRefGoogle Scholar
- Acar E (2015) Effect of error metrics on optimum weight factor selection for ensemble of metamodels. Expert Syst Appl 42(5):2703–2709CrossRefGoogle Scholar
- Bishop CM (1995) Neural networks for pattern recognition. Oxford university press, OxfordMATHGoogle Scholar
- Box GE, Draper NR (1987) Empirical model-building and response surfaces, vol 424. Wiley, New YorkMATHGoogle Scholar
- Buckland ST, Burnham KP, Augustin NH (1997) Model selection: an integral part of inference. Biometrics 53:603–618CrossRefMATHGoogle Scholar
- Cherkassky V, Shao X, Mulier FM, Vapnik VN (1999) Model complexity control for regression using VC generalization bounds. IEEE Trans Neural Netw 10(5):1075–1089CrossRefGoogle Scholar
- Dixon LCW, Szegö GP (eds) (1978) Towards global optimisation. North-Holland, AmsterdamGoogle Scholar
- Forrester AI, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1):50–79CrossRefGoogle Scholar
- Forrester A, Sobester A, Keane A (2008) Engineering design via surrogate modelling: a practical guide. John Wiley & Sons, ChichesterCrossRefGoogle Scholar
- Goel T, Haftka RT, Shyy W, Queipo NV (2007) Ensemble of surrogates. Struct Multidiscip Optim 33(3):199–216CrossRefGoogle Scholar
- Hardy RL (1971) Multiquadric equations of topography and other irregular surfaces. J Geophys Res 76(8):1905–1915CrossRefGoogle Scholar
- Hoeting JA, Madigan D, Raftery AE, Volinsky CT (1999) Bayesian model averaging: a tutorial. Stat Sci:382–401Google Scholar
- Jones DR (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21(4):345–383MathSciNetCrossRefMATHGoogle Scholar
- Kass RE, Raftery AE (1995) Bayes factors. J Am Stat Assoc 90(430):773–795MathSciNetCrossRefMATHGoogle Scholar
- Madigan D, Raftery AE (1994) Model selection and accounting for model uncertainty in graphical models using Occam's window. J Am Stat Assoc 89(428):1535–1546CrossRefMATHGoogle Scholar
- McKay MD, Beckman RJ, Conover WJ (1979) Comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245MathSciNetMATHGoogle Scholar
- Penrose R (1955) A generalized inverse for matrices. Math Proc Camb Philos Soc 51(03):406–413 Cambridge University PressCrossRefMATHGoogle Scholar
- Powell MJD (1987) Radial basis functions for multivariable interpolation: a review. In: Mason JC, Cox MG (eds) Proceedings of the IMA conference on algorithms for the approximation of functions and data, Oxford University Press, London, pp 143-167Google Scholar
- Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41(1):1–28CrossRefGoogle Scholar
- Rennie JDM (2005) Volume of the n-sphere. Retrieved in April 2017, from http://people.csail.mit.edu/jrennie/writing/sphereVolume.pdf
- Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4:409–423MathSciNetCrossRefMATHGoogle Scholar
- Sanchez E, Pintos S, Queipo NV (2008) Toward an optimal ensemble of kernel-based approximations with engineering applications. Struct Multidiscip Optim 36(3):247–261CrossRefGoogle Scholar
- Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222MathSciNetCrossRefGoogle Scholar
- Viana FA, Haftka RT, Steffen V (2009) Multiple surrogates: how cross-validation errors can help us to obtain the best predictor. Struct Multidiscip Optim 39(4):439–457CrossRefGoogle Scholar
- Zerpa LE, Queipo NV, Pintos S, Salager JL (2005) An optimization methodology of alkaline–surfactant–polymer flooding processes using field scale numerical simulation and multiple surrogates. J Pet Sci Eng 47(3):197–208CrossRefGoogle Scholar
- Zhou XJ, Ma YZ, Li XF (2011) Ensemble of surrogates with recursive arithmetic average. Struct Multidiscip Optim 44(5):651–671CrossRefGoogle Scholar