# Monte Carlo integration with adaptive variance selection for improved stochastic efficient global optimization

- 27 Downloads

## Abstract

In this paper, the minimization of computational cost on evaluating multidimensional integrals is explored. More specifically, a method based on an adaptive scheme for error variance selection in Monte Carlo integration (MCI) is presented. It uses a stochastic efficient global optimization (sEGO) framework to guide the optimization search. The MCI is employed to approximate the integrals, because it provides the variance of the error in the integration. In the proposed approach, the variance of the integration error is included into a stochastic kriging framework by setting a target variance in the MCI. We show that the variance of the error of the MCI may be controlled by the designer and that its value strongly influences the computational cost and the exploration ability of the optimization process. Hence, we propose an adaptive scheme for automatic selection of the target variance during the sEGO search. The robustness and efficiency of the proposed adaptive approach were evaluated on global optimization stochastic benchmark functions as well as on a tuned mass damper design problem. The results showed that the proposed adaptive approach consistently outperformed the constant approach and a multi-start optimization method. Moreover, the use of MCI enabled the method application in problems with high number of stochastic dimensions. On the other hand, the main limitation of the method is inherited from sEGO coupled with the kriging metamodel: the efficiency of the approach is reduced when the number of design variables increases.

## Keywords

Stochastic kriging Efficient global optimization Integral minimization Adaptive target variance Robust optimization## Notes

### Funding information

This study received financial support from the Brazilian research funding agencies CNPq and CAPES.

## References

- Ankenman B, Nelson BL, Staum J (2010) Stochastic kriging for simulation metamodeling. Oper Res 58 (2):371–382MathSciNetzbMATHGoogle Scholar
- Bae HG, Park SH, Kwon JH (2012) Ego method for diffusing s-duct shape design. In: International conference on computation fluid dynamics, vol 7Google Scholar
- Beck AT, Kougioumtzoglou IA, dos Santos KRM (2014) Optimal performance-based design of non-linear stochastic dynamical rc structures subject to stationary wind excitation. Eng Struct 78:145–153Google Scholar
- Beck J, Dia BM, Espath LFR, Long Q, Tempone R (2018) Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain. Comput Methods Appl Mech Eng 334:523–553. https://doi.org/10.1016/j.cma.2018.01.053 https://doi.org/10.1016/j.cma.2018.01.053 MathSciNetGoogle Scholar
- Bobby S, Spence SMJ, Kareem A (2016) Data-driven performance-based topology optimization of uncertain wind-excited tall buildings. Struct Multidiscip Optim 54(6):1379–1402. https://doi.org/10.1007/s00158-016-1474-6 https://doi.org/10.1007/s00158-016-1474-6 MathSciNetGoogle Scholar
- Capiez-Lernout E, Soize C (2008) Robust design optimization in computational mechanics. Journal of Applied Mechanics Transactions ASME 75(2):021001–1–021001–11Google Scholar
- Chakraborty S, Roy BK (2011) Reliability based optimum design of tuned mass damper in seismic vibration control of structures with bounded uncertain parameters. Probab Eng Mech 26(2):215– 221Google Scholar
- Chaudhuri A, Haftka RT, Ifju P, Chang K, Tyler C, Schmitz T (2015) Experimental flapping wing optimization and uncertainty quantification using limited samples. Struct Multidiscip Optim 51(4):957–970. ISSN 1615-1488. https://doi.org/10.1007/s00158-014-1184-x Google Scholar
- Chen WF, Lui EM (2005) Handbook of structural engineering, 2nd edn. CRC Press, Boca Raton. ISBN 9781420039931Google Scholar
- Chen X, Kim K-K (2014) Stochastic kriging with biased sample estimates. ACM Trans Model Comput Simul (TOMACS) 24(2):8MathSciNetzbMATHGoogle Scholar
- Chen X, Ankenman BE, Nelson BL (2012) The effects of common random numbers on stochastic kriging metamodels. ACM Trans Model Comput Simul (TOMACS) 22(2):7MathSciNetzbMATHGoogle Scholar
- Chen X, Ankenman BE, Nelson BL (2013) Enhancing stochastic kriging metamodels with gradient estimators. Oper Res 61(2):512–528MathSciNetzbMATHGoogle Scholar
- Couckuyt I, Declercq F, Dhaene T, Rogier H, Knockaert L (2010) Surrogate-based infill optimization applied to electromagnetic problems. Int J RF Microwave Comput Aided Eng 20(5):492–501Google Scholar
- Cressie N (1993) Statistics for spatial data, vol 15. Wiley, New YorkGoogle Scholar
- Duda RO, Hart PE, Stork DG (2012) Pattern classification. Wiley, New YorkzbMATHGoogle Scholar
- Duvigneau R, Chandrashekar P (2012) Kriging-based optimization applied to flow control. Int J Numer Methods Fluids 69(11):1701–1714Google Scholar
- Fadel Miguel Letícia F, Fadel MLF, Holdorf LR (2016) Failure probability minimization of buildings through passive friction dampers. Struct Design Tall Spec Build 25(17):869–885Google Scholar
- Forrester A, Sobester A, Keane A (2008) Engineering design via surrogate modelling: a practical guide. Wiley, ChichesterGoogle Scholar
- Gengembre E, Ladevie B, Fudym O, Thuillier A (2012) A kriging constrained efficient global optimization approach applied to low-energy building design problems. Inverse Prob Sci Eng 20(7):1101–1114Google Scholar
- Michael BG (2008) Multilevel monte carlo path simulation. Oper Res 56(3):607–617MathSciNetzbMATHGoogle Scholar
- Goel T, Hafkta RT, Shyy W (2008) Comparing error estimation measures for polynomial and kriging approximation of noise-free functions. Struct Multidiscip Optim 38(5):429. ISSN 1615-1488. https://doi.org/10.1007/s00158-008-0290-z Google Scholar
- Gomes WJS, Beck AT, Lopez RH, Miguel LFF (2018) A probabilistic metric for comparing metaheuristic optimization algorithms. Struct Saf 70:59–70. ISSN 0167-4730Google Scholar
- Gramacy RB, Lee HKH (2010) Optimization under unknown constraints. In: Proceedings of the sixth Valencia international meeting. Oxford University Press, USAGoogle Scholar
- Haftka RT, Villanueva D, Chaudhuri A (2016) Parallel surrogate-assisted global optimization with expensive functions – a survey. Struct Multidiscip Optim 54(1):3–13. ISSN 1615-1488. https://doi.org/10.1007/s00158-016-1432-3 https://doi.org/10.1007/s00158-016-1432-3 MathSciNetGoogle Scholar
- Haji-Ali A-L, Nobile F, Tempone R (2016) Multi-index monte carlo: when sparsity meets sampling. Numer Math 132(4):767–806MathSciNetzbMATHGoogle Scholar
- Hammersley JM, Handscomb DC (1964) General principles of the monte carlo method. In: Monte carlo methods. Springer , pp 50–75Google Scholar
- Hao P, Feng S, Zhang K, Li Z, Wang B, Li G (2018) Adaptive gradient-enhanced kriging model for variable-stiffness composite panels using isogeometric analysis. Struct Multidiscip Optim 58(1):1–16Google Scholar
- Huan X, Marzouk YM (2013) Simulation-based optimal bayesian experimental design for nonlinear systems. J Comput Phys 232(1):288–317MathSciNetGoogle Scholar
- Huang D, Allen TT, Notz WI, Zeng N (2006) Global optimization of stochastic black-box systems via sequential kriging meta-models. J Glob Optim 34(3):441–466MathSciNetzbMATHGoogle Scholar
- Huang Z, Wang C, Chen J, Tian H (2011) Optimal design of aeroengine turbine disc based on kriging surrogate models. Comput Struct 89(1):27–37Google Scholar
- Forrester AIJ, Keane AJ, Bressloff NW (2006) Design and analysis of “noisy” computer experiments. AIAA J 44(10):2331–2339Google Scholar
- Jalali H, Nieuwenhuyse IV, Picheny V (2017) Comparison of kriging-based algorithms for simulation optimization with heterogeneous noise. Eur J Oper Res 261(1):279–301. ISSN 0377-2217. https://doi.org/10.1016/j.ejor.2017.01.035 MathSciNetzbMATHGoogle Scholar
- Jones D (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21 (4):345–383. https://doi.org/10.1023/a:1012771025575 MathSciNetzbMATHGoogle Scholar
- Jones DR, Schonlau M, William J (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492. ISSN 09255001. https://doi.org/10.1023/a:1008306431147 MathSciNetzbMATHGoogle Scholar
- Kalos MH, Whitlock PA (1986) Monte Carlo methods, volume 1: Basics Monte Carlo methods. Wiley, New York. ISBN 9780471898399zbMATHGoogle Scholar
- Kamiński B (2015) A method for the updating of stochastic kriging metamodels. Eur J Oper Res 247 (3):859–866MathSciNetzbMATHGoogle Scholar
- Kanai K (1957) Semi-empirical formula for the seismic characteristics of the ground. 35, 07Google Scholar
- Kanazaki M, Matsumo T, Maeda K, Kawazoe M (2015) Efficient global optimization applied to multi-objective design optimization of lift creating cylinder using plasma actuators. In: Proceedings of the 18th Asia Pacific symposium on intelligent and evolutionary systems, vol 1, pp 663–677Google Scholar
- Kleijnen JPC, Mehdad E (2016) Estimating the variance of the predictor in stochastic kriging. Simul Model Pract Theory 66:166–173Google Scholar
- Krige DG (1951) A statistical approach to some basic mine valuation problems on the Witwatersrand. J Chem Metall Min Soc S Afr 52(6):119–139. https://doi.org/10.2307/3006914 Google Scholar
- Le Riche R, Haftka RT (2012) On global optimization articles in SMO. Struct Multidiscip Optim 46 (5):627–629. ISSN 1615-1488. https://doi.org/10.1007/s00158-012-0785-5 https://doi.org/10.1007/s00158-012-0785-5 Google Scholar
- Li J, Heap AD (2011) A review of comparative studies of spatial interpolation methods in environmental sciences: performance and impact factors. Eco Inform 6(3–4):228–241Google Scholar
- Locatelli M (1997) Bayesian algorithms for one-dimensional global optimization. J Glob Optim 10(1):57–76. ISSN 1573-2916. https://doi.org/10.1023/A:1008294716304 MathSciNetzbMATHGoogle Scholar
- Lopez RH, Miguel LFF, Beck AT (2015) Tuned mass dampers for passive control of structures under earthquake excitations. Springer, Berlin, pp 3814–3823. ISBN 978-3-642-35344-4. https://doi.org/10.1007/978-3-642-35344-4_215 Google Scholar
- Lopez RH, Ritto TG, Sampaio R, Souza de Cursi JE (2014) A new algorithm for the robust optimization of rotor-bearing systems. Eng Optim 46(8):1123–1138. https://doi.org/10.1080/0305215X.2013.819095 https://doi.org/10.1080/0305215X.2013.819095 Google Scholar
- Luersen MA, Le Riche R, Guyon F (2004) A constrained, globalized, and bounded Nelder–Mead method for engineering optimization. Struct Multidiscip Optim 27(1):43–54Google Scholar
- Luersen MA, Riche RL (2004) Globalized Nelder–Mead method for engineering optimization. Comput Struct 82(23):2251–2260Google Scholar
- Mantovani GZ, Miguel LFF, Lopez RH, Miguel LFF, Torii AJ (2017) Optimum design of multiple friction tuned mass dampers under seismic excitationsGoogle Scholar
- MATLAB (2015) MATLAB version 8.6.0.267246 (R2015b). The Mathworks, Inc., Natick, MassachusettsGoogle Scholar
- Melchers RE, Beck AT (2018) Structural reliability analysis and prediction. Wiley, New YorkGoogle Scholar
- Miguel LFF, Lopez RH, Torii AJ, Miguel LFF, Beck AT (2016) Robust design optimization of TMDs in vehicle–bridge coupled vibration problems. Eng Struct 126:703–711. ISSN 0141-0296Google Scholar
- Miguel LFF, Lopez RH , Miguel LFF (2013) A hybrid approach for damage detection of structures under operational conditions. J Sound Vib 332(18):4241–4260. ISSN 0022-460XGoogle Scholar
- Mohebbi M, Shakeri K, Ghanbarpour Y, Majzoub H (2013) Designing optimal multiple tuned mass dampers using genetic algorithms (GAs) for mitigating the seismic response of structures. J Vib Control 19(4):605–625Google Scholar
- Montgomery DC, Runger GC (2010) Applied statistics and probability for engineers. Wiley, New YorkzbMATHGoogle Scholar
- Nelder JA, Mead R (1965) A simplex method for function minimization. Comput J 7(4):308–313MathSciNetzbMATHGoogle Scholar
- Nhamage IA, Lopez RH, Miguel LFF (2016) An improved hybrid optimization algorithm for vibration based-damage detection. Adv Eng Softw 93:47–64. ISSN 0965-9978. https://doi.org/10.1016/j.advengsoft.2015.12.003 https://doi.org/10.1016/j.advengsoft.2015.12.003 Google Scholar
- Osborne MA, Garnett R, Roberts SJ (2009) Gaussian processes for global optimization. In: 3rd international conference on learning and intelligent optimization (LION3), pp 1–15Google Scholar
- Picheny V, Ginsbourger D (2014) Noisy kriging-based optimization methods: a unified implementation within the diceoptim package. Comput Stat Data Anal 71:1035–1053MathSciNetzbMATHGoogle Scholar
- Picheny V, Ginsbourger D, Richet Y, Caplin G (2013a) Quantile-based optimization of noisy computer experiments with tunable precision. Technometrics 55(1):2–13MathSciNetGoogle Scholar
- Picheny V, Wagner T, Ginsbourger D (2013b) A benchmark of kriging-based infill criteria for noisy optimization. Struct Multidiscip Optim 48(3):607–626Google Scholar
- Plumlee M, Tuo R (2014) Building accurate emulators for stochastic simulations via quantile kriging. Technometrics 56(4):466– 473MathSciNetGoogle Scholar
- Qu H, Fu MC (2014) Gradient extrapolated stochastic kriging. ACM Trans Model Comput Simul (TOMACS) 24(4):23MathSciNetzbMATHGoogle Scholar
- Ritto TG, Lopez RH, Sampaio R, Souza De Cursi JE (2011) Robust optimization of a flexible rotor-bearing system using the campbell diagram. Eng Optim 43(1):77–96MathSciNetGoogle Scholar
- Rubinstein RY (2007) Simulation and the Monte Carlo method. Wiley, New YorkGoogle Scholar
- Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Statist Sci 4(4):409–423, 11. https://doi.org/10.1214/ss/1177012413 MathSciNetzbMATHGoogle Scholar
- Sakata S, Ashida F, Tanaka H (2011) Kriging-based convex subspace single linkage method with path-based clustering technique for approximation-based global optimization. Struct. Multidiscip Optim 44(3):393–408. ISSN 1615-1488. https://doi.org/10.1007/s00158-011-0643-x Google Scholar
- Scott W, Frazier P, Powell W (2011) The correlated knowledge gradient for simulation optimization of continuous parameters using gaussian process regression. SIAM J Optim 21(3):996–1026MathSciNetzbMATHGoogle Scholar
- Shen H, Hong LJ, Zhang X (2018) Enhancing stochastic kriging for queueing simulation with stylized models. IISE Transactions (just-accepted)Google Scholar
- Soize C, Capiez-Lernout E, Ohayon R (2008) Robust updating of uncertain computational models using experimental modal analysis. Mech Syst Signal Process 22(8):1774–1792Google Scholar
- Seymour MJS, Kareem A (2014) Performance-based design and optimization of uncertain wind-excited dynamic building systems. Eng Struct 78:133–144Google Scholar
- Staum J (2009) Better simulation metamodeling: The why, what, and how of stochastic kriging. IEEE, Austin, pp 119–133Google Scholar
- Tajimi H (1960) A statistical method of determining the maximum response of a building structure during an earthquake. In: Proceedings of the 2nd world conference on earthquake engineering. Tokyo, Japan, pp 781–797Google Scholar
- Theodossiou N, Latinopoulos P (2006) Evaluation and optimisation of groundwater observation networks using the kriging methodology. Environ Modell Softw 21(7):991–1000Google Scholar
- Torii AJ, Lopez RH, Luersen MA (2011) A local-restart coupled strategy for simultaneous sizing and geometry truss optimization. Latin American Journal of Solids and Structures 8:335–349. ISSN 1679-7825. https://doi.org/10.1590/S1679-78252011000300008 Google Scholar
- Ulaganathan S, Couckuyt I, Dhaene T, Laermans E, Degroote J (2014) On the use of gradients in kriging surrogate models. In: 2014 winter simulation conference (WSC). IEEE, pp 2692–2701Google Scholar
- Ur Rehman S, Langelaar M (2017) Adaptive efficient global optimization of systems with independent components. Struct Multidiscip Optim 55(4):1143–1157. ISSN 1615-1488. https://doi.org/10.1007/s00158-017-1663-y zbMATHGoogle Scholar
- Vazquez E, Villemonteix J, Sidorkiewicz M, Walter E (2008) Global optimization based on noisy evaluations: an empirical study of two statistical approaches. In: Journal of physics: conference series, vol 135. IOP Publishing, p 012100Google Scholar
- Villemonteix J, Vazquez E, Walter E (2009) An informational approach to the global optimization of expensive-to-evaluate functions. J Glob Optim 44(4):509MathSciNetzbMATHGoogle Scholar
- Wang K, Xi C, Yang F, Porter DW, Wu N (2014) A new stochastic kriging method for modeling multi-source exposure–response data in toxicology studies. ACS Sustain Chem Eng 2(7):1581–1591Google Scholar
- Zhang Q, Xie W (2017) Asymmetric kriging emulator for stochastic simulation. In: Proceedings of the 2017 winter simulation conference. IEEE Press, p 137Google Scholar
- Zhu P, Zhang LW, Liew KM (2014) Geometrically nonlinear thermomechanical analysis of moderately thick functionally graded plates using a local petrov–galerkin approach with moving kriging interpolation. Compos Struct 107:298–314Google Scholar
- Zou L, Zhang X (2018) Stochastic kriging for inadequate simulation models. arXiv:1802.00677