Skip to main content
SpringerLink
Log in

Kriging Metamodels and Their Designs

  • Chapter
Design and Analysis of Simulation Experiments

Part of the book series: International Series in Operations Research & Management Science ((ISOR,volume 230))

Abstract

This chapter is organized as follows. Section 5.1 introduces Kriging, which is also called Gaussian process (GP) or spatial correlation modeling. Section 5.2 details so-called ordinary Kriging (OK), including the basic Kriging assumptions and formulas assuming deterministic simulation. Section 5.3 discusses parametric bootstrapping and conditional simulation for estimating the variance of the OK predictor. Section 5.4 discusses universal Kriging (UK) in deterministic simulation. Section 5.5 surveys designs for selecting the input combinations that gives input/output data to which Kriging metamodels can be fitted; this section focuses on Latin hypercube sampling (LHS) and customized sequential designs. Section 5.6 presents stochastic Kriging (SK) for random simulations. Section 5.7 discusses bootstrapping with acceptance/rejection for obtaining Kriging predictors that are monotonic functions of their inputs. Section 5.8 discusses sensitivity analysis of Kriging models through functional analysis of variance (FANOVA) using Sobol’s indexes. Section 5.9 discusses risk analysis (RA) or uncertainty analysis (UA). Section 5.10 discusses several remaining issues. Section 5.11 summarizes the major conclusions of this chapter, and suggests topics for future research. The chapter ends with Solutions of exercises, and a long list of references.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Anderson B, Borgonovo E, Galeotti M, Roson R (2014) Uncertainty in climate change modeling: can global sensitivity analysis be of help? Risk Anal 34(2):271–293

    Article  Google Scholar 

  • Ankenman B, Nelson B, Staum J (2010) Stochastic kriging for simulation metamodeling. Oper Res 58(2):371–382

    Article  Google Scholar 

  • Antognini B, Zagoraiou M (2010) Exact optimal designs for computer experiments via kriging metamodelling. J Stat Plan Inference 140(9):2607–2617

    Article  Google Scholar 

  • Archer GEB, Saltelli A, Sobol IM (1997) Sensitivity measures, ANOVA-like techniques and the use of bootstrap. J Stat Comput Simul 58:99–120

    Article  Google Scholar 

  • Ba S, Brenneman WA, Myers WR (2014) Optimal sliced Latin hypercube designs. Technometrics (in press)

    Google Scholar 

  • Bachoc F (2013) Cross validation and maximum likelihood estimation of hyper-parameters of Gaussian processes with model misspecification. Comput Stat Data Anal 66:55–69

    Article  Google Scholar 

  • Barton RR, Nelson BL, Xie W (2014) Quantifying input uncertainty via simulation confidence intervals. INFORMS J Comput 26(1):74–87

    Article  Google Scholar 

  • Bassamboo A, Randhawa RS, Zeevi A (2010) Capacity sizing under parameter uncertainty: safety staffing principles revisited. Manag Sci 56(10):1668–1686

    Article  Google Scholar 

  • Bastos LS, O’Hagan A (2009) Diagnostics for Gaussian process emulators. Technometrics 51(4):425–438

    Article  Google Scholar 

  • Batarseh OG, Wang Y (2008) Reliable simulation with input uncertainties using an interval-based approach. In: Mason SJ, Hill RR, Mönch L, Rose O, Jefferson T, Fowler JW (eds) Proceedings of the 2008 winter simulation conference, Miami, pp 344–352

    Google Scholar 

  • Bekki J, Chen X, Batur D (2014) Steady-state quantile parameter estimation: an empirical comparison of stochastic kriging and quantile regression. In: Tolk A, Diallo SY, Ryzhov IO, Yilmaz L, Buckley S, Miller JA (eds) Proceedings of the 2014 Winter Simulation Conference, Savannah, pp 3880–3891

    Google Scholar 

  • Borgonovo E, Plischke E (2015) Sensitivity analysis: a review of recent advances. Eur J Oper Res (in press)

    Google Scholar 

  • Borgonovo E, Tarantola S, Plischke E, Morris MD (2014) Transformations and invariance in the sensitivity analysis of computer experiments. J R Stat Soc, Ser B 76:925–947

    Article  Google Scholar 

  • Boukouvalas A, Cornford D, Stehlík M (2014) Optimal design for correlated processes with input-dependent noise. Comput Stat Data Anal 71:1088–1102

    Article  Google Scholar 

  • Bowman VE, Woods DC (2013) Weighted space-filling designs. J Simul 7:249–263

    Article  Google Scholar 

  • Busby D, Farmer CL, Iske A (2007) Hierarchical nonlinear approximation for experimental designs and statistical data fitting. SIAM J Sci Comput 29(1):49–69

    Article  Google Scholar 

  • Butler A, Haynes RD, Humphriesa TD, Ranjan P (2014) Efficient optimization of the likelihood function in Gaussian process modelling. Comput Stat Data Anal 73:40–52

    Article  Google Scholar 

  • Callahan BG (ed) (1996) Special issue: commemoration of the 50th anniversary of Monte Carlo. Hum Ecol Risk Assess 2(4):627–1037

    Google Scholar 

  • Challenor P (2013) Experimental design for the validation of Kriging metamodels in computer experiments. J Simul (7):290–296

    Article  Google Scholar 

  • Chen EJ, Li M (2014) Design of experiments for interpolation-based metamodels. Simul Model Pract Theory 44:14–25

    Article  Google Scholar 

  • Chen VCP, Tsui K-L, Barton RR, Meckesheimer M (2006) A review on design, modeling, applications of computer experiments. IIE Trans 38:273–291

    Article  Google Scholar 

  • Chen X, Ankenman B, Nelson BL (2012) The effects of common random numbers on stochastic Kriging metamodels. ACM Trans Model Comput Simul 22(2):7:1–7:20

    Google Scholar 

  • Chen X, Kim K-K (2013) Building metamodels for quantile-based measures using sectioning. In: Pasupathy R, Kim S-H, Tolk A, Hill R, Kuhl ME (eds) Proceedings of the 2013 winter simulation conference, Washington, DC, pp 521–532

    Chapter  Google Scholar 

  • Chen X, Wang K, Yang F (2013) Stochastic kriging with qualitative factors. In: Pasupathy R, Kim S-H, Tolk A, Hill R, Kuhl ME (eds) Proceedings of the 2013 winter simulation conference, Washington, DC, pp 790–801

    Chapter  Google Scholar 

  • Chen X, Zhou Q (2014) Sequential experimental designs for stochastic kriging. In: Tolk A, Diallo SD, Ryzhov IO, Yilmaz L, Buckley S, Miller JA (eds) Proceedings of the 2014 winter simulation conference, Savannah, pp 3821–3832

    Google Scholar 

  • Chevalier C, Ginsbourger D (2012) Corrected Kriging update formulae for batch-sequential data assimilation. arXiv, 1203.6452v1

    Google Scholar 

  • Chevalier C, Ginsbourger D, Bect J, Molchanov I (2013) Estimating and quantifying uncertainties on level sets using the Vorob’ev expectation and deviation with Gaussian process models. In: Ucinski D, Atkinson AC, Patan M (eds) mODa 10 – advances in model-oriented design and analysis; proceedings of the 10th international workshop in model-oriented design and analysis. Springer, New York, pp 35–43

    Chapter  Google Scholar 

  • Chevalier C, Ginsbourger D, Bect J, Vazquez E, Picheny V, Richet Y (2014) Fast parallel Kriging-based stepwise uncertainty reduction with application to the identification of an excursion set. Technometrics 56(4): 455–465

    Article  Google Scholar 

  • Chilès J-P, Delfiner P (2012) Geostatistics: modeling spatial uncertainty, 2nd edn. Wiley, New York

    Book  Google Scholar 

  • Clark I (2010) Statistics or geostatistics? Sampling error or nugget effect? J S Afr Inst Min Metall 110:307–312

    Google Scholar 

  • Couckuyt I, Dhaene T, Demeester P (2014) ooDACE toolbox: a flexible object-oriented Kriging implementation. J Mach Learn Res 15:3183–3186

    Google Scholar 

  • Couckuyt I, Forrester A, Gorissen D, Dhaene T (2012) Blind kriging; implementation and performance analysis. Adv Eng Softw 49:1–13

    Article  Google Scholar 

  • Cressie NAC (1993) Statistics for spatial data, rev edn. Wiley, New York

    Google Scholar 

  • Crombecq K, Laermans E, Dhaene T (2011) Efficient space-filling and non-collapsing sequential design strategies for simulation-based modeling. Eur J Oper Res 214:683–696

    Article  Google Scholar 

  • Damblin G, Couplet M, Iooss B (2013) Numerical studies of space-filling designs: optimization of Latin hypercube samples and subprojection properties.J Simul 7:276–289

    Google Scholar 

  • Da Veiga S, Marrel A (2012) Gaussian process modeling with inequality constraints. Annales de la faculté des sciences de Toulouse Sér. 6 21(3):529–555

    Google Scholar 

  • De Rocquigny E, Devictor N, Tarantola S (2008) Uncertainty settings and natures of uncertainty. In: de Rocquigny E, Devictor N, Tarantola S (eds) Uncertainty in industrial practice. Wiley, Chichester

    Chapter  Google Scholar 

  • Den Hertog D, Kleijnen JPC, Siem AYD (2006) The correct Kriging variance estimated by bootstrapping. J Oper Res Soc 57(4):400–409

    Article  Google Scholar 

  • Deng H, Shao W, Ma Y, Wei Z (2012) Bayesian metamodeling for computer experiments using the Gaussian Kriging models. Qual Reliab Eng 28(4):455–466

    Article  Google Scholar 

  • Dette H, Pepelyshev A (2010) Generalized Latin hypercube design for computer experiments. Technometrics 25:421–429

    Article  Google Scholar 

  • Deutsch CV (1996) Correcting for negative weights in ordinary Kriging. Comput Geosci 22(7):765–773

    Article  Google Scholar 

  • Deutsch JL, Deutsch CV (2012) Latin hypercube sampling with multidimensional uniformity. J Stat Plan Inference 142(3):763–772

    Article  Google Scholar 

  • Evans JR, Olson DL (1998) Introduction to simulation and risk analysis. Prentice-Hall, Upper Saddle River

    Google Scholar 

  • Fang K-T, Li R, Sudjianto A (2006) Design and modeling for computer experiments. Chapman & Hall/CRC, London

    Google Scholar 

  • Farah M, Kottas A (2014) Bayesian inference for sensitivity analysis of computer simulators, with an application to radiative transfer models. Technometrics 56(2):159–173

    Article  Google Scholar 

  • Forrester AIJ (2013) Comment: properties and practicalities of the expected quantile improvement. Technometrics 55(1):13–18

    Article  Google Scholar 

  • Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79

    Article  Google Scholar 

  • Forrester A, Sóbester A, Keane A (2008) Engineering design via surrogate modelling: a practical guide. Wiley, Chichester

    Book  Google Scholar 

  • Frazier PI (2011) Learning with dynamic programming. In: Cochran JJ, Cox LA, Keskinocak P, Kharoufeh JP, Smith JC (eds) Encyclopedia of operations research and management science. Wiley, New York

    Google Scholar 

  • Gano SE, Renaud JE, Martin JD, Simpson TW (2006) Update strategies for Kriging models for using in variable fidelity optimization. Struct Multidiscip Optim 32(4):287–298

    Article  Google Scholar 

  • Georgiou SD, Stylianou S (2011) Block-circulant matrices for constructing optimal Latin hypercube designs. J Stat Plan Inference 141:1933–1943

    Article  Google Scholar 

  • Ghosh BK, Sen PK (eds) (1991) Handbook of sequential analysis. Marcel Dekker, New York

    Google Scholar 

  • Ginsbourger D, Dupuy D, Badea A, Carraro L, Roustant O (2009) A note on the choice and the estimation of Kriging models for the analysis of deterministic computer experiments. Appl Stoch Models Bus Ind 25: 115–131

    Article  Google Scholar 

  • Ginsbourger D, Iooss B, Pronzato L (2015) Editorial. J Stat Comput Simul 85(7):1281–1282

    Article  Google Scholar 

  • Giunta AA, McFarland JM, Swiler LP, Eldred MS (2006) The promise and peril of uncertainty quantification using response surface approximations. Struct Infrastruct Eng 2(3–4):175–189

    Article  Google Scholar 

  • Goel T, Haftka R, Queipo N, Shyy W (2006) Performance estimate and simultaneous application of multiple surrogates. In: 11th AIAA/ISSMO multidisciplinary analysis and optimization conference, multidisciplinary analysis optimization conferences. American Institute of Aeronautics and Astronautics, Reston, VA 20191–4344, pp 1–26

    Google Scholar 

  • Goh J, Bingham D, Holloway JP, Grosskopf MJ, Kuranz CC, Rutter E (2013) Prediction and computer model calibration using outputs from multi-fidelity simulators. Technometrics 55(4):501–512

    Article  Google Scholar 

  • Goldberg PW, Williams CKI, Bishop CM (1998) Regression with input-dependent noise: a Gaussian process treatment. In: Jordan MI, Kearns MJ, Solla SA (eds) Advances in neural information processing systems, vol 10. MIT, Cambridge, pp 493–499

    Google Scholar 

  • Golzari A, Sefat MH, Jamshidi S (2015) Development of an adaptive surrogate model for production optimization. J Petrol Sci Eng (in press)

    Google Scholar 

  • Gramacy RB and Haaland B (2015) Speeding up neighborhood search in local Gaussian process prediction. Technometrics (in press)

    Google Scholar 

  • Gramacy RB, Lee HKH (2008) Bayesian treed Gaussian process models with an application to computer modeling. J Am Stat Assoc 103(483):1119–1130

    Article  Google Scholar 

  • Gramacy RB, Lee HKH (2012) Cases for the nugget in modeling computer experiments. Stat Comput 22:713–722

    Article  Google Scholar 

  • Grosso A, Jamali ARMJU, Locatelli M (2009) Finding maximin Latin hypercube designs by iterated local search heuristics. Eur J Oper Res 197(2):541–54

    Article  Google Scholar 

  • Hankin RKS (2005) Introducing BACCO, an R bundle for Bayesian analysis of computer code output. J Stat Softw 14(16):1–21

    Article  Google Scholar 

  • Harari O, Steinberg DM (2014a) Optimal designs for Gaussian process models via spectral decomposition. J Stat Plan Inference (in press)

    Google Scholar 

  • Harari O, Steinberg DM (2014b) Convex combination of Gaussian processes for Bayesian analysis of deterministic computer experiments. Technometrics 56(4):443–454

    Article  Google Scholar 

  • Helton JC, Davis FJ, Johnson JD (2005) A comparison of uncertainty and sensitivity results obtained with random and Latin hypercube sampling. Reliab Eng Syst Saf 89:305–330

    Article  Google Scholar 

  • Helton JC, Johnson JD, Oberkampf WD, Sallaberry CJ (2006a) Sensitivity analysis in conjunction with evidence theory representations of epistemic uncertainty. Reliab Eng Syst Saf 91:1414–1434

    Article  Google Scholar 

  • Helton JC, Johnson JD, Oberkampf WD, Sallaberry CJ (2010) Representation of analysis results involving aleatory and epistemic uncertainty. Int J Gen Syst 39(6):605–646

    Article  Google Scholar 

  • Helton JC, Johnson JD, Sallaberry CJ, Storlie CB (2006b) Survey of sampling-based methods for uncertainty and sensitivity analysis. Reliab Eng Syst Saf 91:1175–1209

    Article  Google Scholar 

  • Helton JC, Pilch M (2011) Guest editorial: quantification of margins and uncertainty. Reliab Eng Syst Saf 96:959–964

    Article  Google Scholar 

  • Helton JC, Hansen CW, Sallaberry CJ (2014) Conceptual structure and computational organization of the 2008 performance assessment for the proposed high-level radioactive waste repository at Yucca Mountain, Nevada. Reliab Eng Syst Saf 122:223–248

    Article  Google Scholar 

  • Henkel T, Wilson H, Krug W (2012) Global sensitivity analysis of nonlinear mathematical models – an implementation of two complementing variance-based algorithms. In: Laroque C, Himmelspach J, Pasupathy R, Rose O, Uhrmacher AM (eds) Proceedings of the 2012 winter simulation conference, Washington, DC, pp 1737–1748

    Google Scholar 

  • Hernandez AF, Grover MA (2010) Stochastic dynamic predictions using Gaussian process models for nanoparticle synthesis. Comput Chem Eng 34(12):1953–1961

    Article  Google Scholar 

  • Hernandez AS, Lucas TW, Sanchez PJ (2012) Selecting random Latin hypercube dimensions and designs through estimation of maximum absolute pairwise correlation. In: Laroque C, Himmelspach J, Pasupathy R, Rose O, Uhrmacher AM (eds) Proceedings of the 2012 winter simulation conference, Berlin, pp 280–291

    Google Scholar 

  • Hubert M, Engelen S (2007) Fast cross-validation of high-breakdown resampling methods for PCA. Comput Stat Data Anal 51(10):5013–5024

    Article  Google Scholar 

  • Iooss B, Boussouf L, Feuillard V, Marrel A (2010) Numerical studies of the metamodel fitting and validation processes. Int J Adv Syst Meas 3:11–21

    Google Scholar 

  • Jala M, Lévy-Leduc C, Moulines É, Conil E, Wiart J (2014) Sequential design of computer experiments for the assessment of fetus exposure to electromagnetic fields. Technometrics (in press)

    Google Scholar 

  • Janssen H (2013) Monte-Carlo based uncertainty analysis: sampling efficiency and sampling convergence. Reliab Eng Syst Saf 109:123–132

    Article  Google Scholar 

  • Jeon JS, Lee SR, Pasquinelli L, Fabricius IL (2015) Sensitivity analysis of recovery efficiency in high-temperature aquifer thermal energy storage with single well. Energy (in press)

    Google Scholar 

  • Jian N, Henderson S, Hunter SR (2014) Sequential detection of convexity from noisy function evaluations. In: Tolk A, Diallo SY, Ryzhov IO, Yilmaz L, Buckley S, Miller JA (eds) Proceedings of the 2014 winter simulation conference, Savannah, pp 3892–3903

    Google Scholar 

  • Jin, R, Chen W, Sudjianto A (2002) On sequential sampling for global metamodeling in engineering design. In: Proceedings of DET’02, ASME 2002 design engineering technical conferences and computers and information in engineering conference, DETC2002/DAC-34092, Montreal, 29 Sept–2 Oct 2002

    Google Scholar 

  • Jones B, Silvestrini RT, Montgomery DC, Steinberg DM (2015) Bridge designs for modeling systems with low noise. Technometrics 57(2): 155–163

    Article  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492

    Article  Google Scholar 

  • Joseph VR, Hung Y, Sudjianto A (2008) Blind Kriging: a new method for developing metamodels. J Mech Des 130(3):31–102

    Article  Google Scholar 

  • Jourdan A, Franco J (2010) Optimal Latin hypercube designs for the Kullback-Leibler criterion. AStA Adv Stat Anal 94:341–351

    Article  Google Scholar 

  • Kamiński B (2015) A method for updating of stochastic Kriging meta- models. Eur J Oper Res (accepted)

    Google Scholar 

  • Kersting K, Plagemann C, Pfaff P, Burgard W (2007) Most-likely heteroscedastic Gaussian process regression. In: Ghahramani Z (ed) Proceedings of the 24th annual international conference on machine learning (ICML-07), Corvalis, pp 393–400

    Google Scholar 

  • Kleijnen JPC (1983). Risk analysis and sensitivity analysis: antithesis or synthesis?. Simuletter, 14(1–4):64–72

    Google Scholar 

  • Kleijnen JPC (1990) Statistics and deterministic simulation models: why not? In: Balci O, Sadowski RP, Nance RE (eds) Proceedings of the 1990 winter simulation conference, Washington, DC, pp 344–346

    Chapter  Google Scholar 

  • Kleijnen JPC (1994) Sensitivity analysis versus uncertainty analysis: when to use what? In: Grasman J, van Straten G (eds) Predictability and nonlinear modelling in natural sciences and economics. Kluwer, Dordrecht, pp 322–333

    Chapter  Google Scholar 

  • Kleijnen JPC (1997) Sensitivity analysis and related analyses: a review of some statistical techniques. J Stat Comput Simul 57(1–4):111–142

    Article  Google Scholar 

  • Kleijnen JPC (2008) Design and analysis of simulation experiments. Springer, New York

    Google Scholar 

  • Kleijnen JPC (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192(3):707–716

    Article  Google Scholar 

  • Kleijnen JPC (2014) Simulation-optimization via Kriging and bootstrapping: a survey. J Simul 8(4):241–250

    Article  Google Scholar 

  • Kleijnen JPC, Mehdad E (2013) Conditional simulation for efficient global optimization. In: Pasupathy R, Kim S-H, Tolk A, Hill R, Kuhl ME (eds) Proceedings of the 2013 winter simulation conference, Washington, DC, pp 969–979

    Chapter  Google Scholar 

  • Kleijnen JPC, Mehdad E (2014) Multivariate versus univariate Kriging metamodels for multi-response simulation models. Eur J Oper Res 236:573–582

    Article  Google Scholar 

  • Kleijnen JPC, Mehdad E (2015) Estimating the correct predictor variance in stochastic Kriging. CentER Discussion Paper, 2015, Tilburg

    Google Scholar 

  • Kleijnen JPC, Mehdad E, Van Beers WCM (2012) Convex and monotonic bootstrapped Kriging. In: Laroque C, Himmelspach J, Pasupathy R, Rose O, Uhrmacher AM (eds) Proceedings of the 2012 winter simulation conference, Washington, DC, pp 543–554

    Google Scholar 

  • Kleijnen JPC, Pierreval H, Zhang J (2011) Methodology for determining the acceptability of system designs in uncertain environments. Eur J Oper Res 209:176–183

    Article  Google Scholar 

  • Kleijnen JPC, Ridder AAN, Rubinstein RY (2013) Variance reduction techniques in Monte Carlo methods. In: Gass SI, Fu MC (eds) Encyclopedia of operations research and management science, 3rd edn. Springer, New York, pp 1598–1610

    Chapter  Google Scholar 

  • Kleijnen JPC, Van Beers WCM (2004) Application-driven sequential designs for simulation experiments: Kriging metamodeling. J Oper Res Soc 55(9):876–883

    Article  Google Scholar 

  • Kleijnen JPC, Van Beers WCM (2013) Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations. J Oper Res Soc 64:708–717

    Article  Google Scholar 

  • Koch P, Wagner T, Emmerich MTM, Bäck T, Konen W (2015) Efficient multi-criteria optimization on noisy machine learning problems. Appl Soft Comput (in press)

    Google Scholar 

  • Koziel S, Bekasiewicz A, Couckuyt I, Dhaene T (2014) Efficient multi-objective simulation-driven antenna design using co-Kriging. IEEE Trans Antennas Propag 62(11):5901–5915

    Article  Google Scholar 

  • Krige DG (1951) A statistical approach to some basic mine valuation problems on the Witwatersrand. J Chem, Metall Min Soc S Afr 52(6):119–139

    Google Scholar 

  • Lamboni M, Iooss B, Popelin A-L, Gamboa F (2013) Derivative-based global sensitivity measures: general links with Sobol indices and numerical tests. Math Comput Simul 87:45–54

    Article  Google Scholar 

  • Lancaster P, Salkauskas K (1986) Curve and surface fitting: an introduction. Academic, London

    Google Scholar 

  • Law AM (2015) Simulation modeling and analysis, 5th edn. McGraw-Hill, Boston

    Google Scholar 

  • Le Gratiet L, Cannamela C (2015) Cokriging-based sequential design strategies using fast cross-validation techniques for multi-fidelity computer codes. Technometrics 57(3):418–427

    Article  Google Scholar 

  • Lemaître P, Sergienko E, Arnaud A, Bousquet N, Gamboa F, Iooss B (2014) Density modification based reliability sensitivity analysis. J Stat Comput Simul (in press)

    Google Scholar 

  • Lemieux C (2009) Monte Carlo and quasi-Monte Carlo sampling. Springer, New York

    Google Scholar 

  • Li K, Jiang B, Ai M (2015) Sliced space-filling designs with different levels of two-dimensional uniformity. J Stat Plan Inference 157–158:90–99

    Article  Google Scholar 

  • Li R, Sudjianto A (2005) Analysis of computer experiments using penalized likelihood in Gaussian Kriging models. Technometrics 47(2):111–120

    Article  Google Scholar 

  • Li Y, Zhou Q (2015) Pairwise meta-modeling of multivariate output computer models using nonseparable covariance function. Technometrics (in press)

    Google Scholar 

  • Lin Y, Mistree F, Allen JK, Tsui K-L, Chen VCP (2004) Sequential metamodeling in engineering design. In: 10th AIAA/ISSMO symposium on multidisciplinary analysis and optimization, Albany, 30 Aug–1 Sept, 2004. Paper number AIAA-2004-4304

    Google Scholar 

  • Lin Y, Mistree F, Tsui K-L, Allen JK (2002) Metamodel validation with deterministic computer experiments. In: 9th AIAA/ISSMO symposium on multidisciplinary analysis and optimization, Atlanta, 4–6 Sept 2002. Paper number AIAA-2002-5425

    Google Scholar 

  • Lloyd-Smith B, Kist AA, Harris RJ, Shrestha N (2004) Shortest paths in stochastic networks. In: Proceedings 12th IEEE international conference on networks 2004, Wakefield, MA, vol 2, pp 492–496

    Google Scholar 

  • Loeppky JL, Sacks J, Welch W (2009) Choosing the sample size of a computer experiment: a practical guide. Technometrics 51(4):366–376

    Article  Google Scholar 

  • Lophaven SN, Nielsen HB, Sondergaard J (2002) DACE: a Matlab Kriging toolbox, version 2.0. IMM Technical University of Denmark, Kongens Lyngby

    Google Scholar 

  • MacCalman AD, Vieira H, Lucas T (2013) Second order nearly orthogonal Latin hypercubes for exploring stochastic simulations. Naval Postgraduate School, Monterey

    Google Scholar 

  • McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245 (reprinted in Technometrics, 42(1,2000):55–61)

    Google Scholar 

  • Marrel A, Iooss B, Da Veiga S, Ribatet M (2012) Global sensitivity analysis of stochastic computer models with joint metamodels. Stat Comput 22:833–847

    Article  Google Scholar 

  • Marrel A, Iooss B, Van Dorpe F, Volkova E (2008) An efficient methodology for modeling complex computer codes with Gaussian processes. Comput Stat Data Anal 52:4731–4744

    Article  Google Scholar 

  • Martin JD, Simpson TW (2005) Use of Kriging models to approximate deterministic computer models. AIAA J 43(4):853–863

    Article  Google Scholar 

  • Martin JD, Simpson TW (2006) A methodology to manage system-level uncertainty during conceptual design. ASME J Mech Des 128(4): 959–968

    Article  Google Scholar 

  • Matheron G (1963) Principles of geostatistics. Econ Geol 58(8):1246–1266

    Article  Google Scholar 

  • Mehdad E, Kleijnen JPC (2015a) Classic Kriging versus Kriging with bootstrapping or conditional simulation: classic Kriging’s robust confidence intervals and optimization. J Oper Res Soc (in press)

    Google Scholar 

  • Mehdad E, Kleijnen JPC (2015b) Stochastic intrinsic Kriging for simulation metamodelling. CentER Discussion Paper, Tilburg

    Google Scholar 

  • Meng Q, Ng SH (2015, in press) An additive global and local Gaussian process model for large datasets. In: Yilmaz L, Chan WKV, Moon I, Roeder TMK, Macal C, Rossetti MD (eds) Proceedings of the 2015 winter simulation conference. [Will be made available on the WSC website in January 2016, after the conference in Dec. 2015]

    Google Scholar 

  • Miller GA (1956) The magical number seven plus or minus two: some limits on our capacity for processing information. The Psychol Rev 63:81–97

    Article  Google Scholar 

  • Mitchell TJ, Morris MD (1992) The spatial correlation function approach to response surface estimation. In: Swain JJ, Goldsman D, Crain RC, Wilson JR (eds) Proceedings of the 1992 winter simulation conference, Arlington

    Google Scholar 

  • Moutoussamy V, Nanty S, Pauwels B (2014) Emulators for stochastic simulation codes. In: ESAIM: Proceedings, Azores, pp 1–10

    Google Scholar 

  • Muehlenstaedt T, Roustant O, Carraro L, Kuhnt S (2012) Data-driven Kriging models based on FANOVA-decomposition. Stat Comput 22:723–738

    Article  Google Scholar 

  • Ng SH, Yin J (2012), Bayesian Kriging analysis and design for stochastic simulations. ACM Trans Model Comput Simul 22(3):1–26

    Article  Google Scholar 

  • Norton J (2015) An introduction to sensitivity assessment of simulation models. Environ Model Softw 69:166–174

    Article  Google Scholar 

  • Oakley J, O’Hagan A (2004) Probabilistic sensitivity analysis of complex models: a Bayesian approach. J R Stat Soc, Ser B, 66(3):751–769

    Article  Google Scholar 

  • Opsomer JD, Ruppert D, Wand MP, Holst U, Hossjer O (1999) Kriging with nonparametric variance function estimation. Biometrics 55(3): 704–710

    Article  Google Scholar 

  • Owen AB, Dick J, Chen S (2013) Higher order Sobol’ indices. http://arxiv.org/abs/1306.4068

  • Plumlee M, Tuo R (2014) Building accurate emulators for stochastic simulations via quantile Kriging, Technometrics 56(4):466–473

    Article  Google Scholar 

  • Qian PZG, Hwang Y, Ai M, Su H (2014) Asymmetric nested lattice samples. Technometrics 56(1):46–54

    Article  Google Scholar 

  • Qu H, Fu MC (2014) Gradient extrapolated stochastic kriging. ACM Trans Model Comput Simul 24(4):23:1–23:25

    Google Scholar 

  • Quadrianto N, Kersting K, Reid MD, Caetano TS, Buntine WL (2009) Kernel conditional quantile estimation via reduction revisited. In: IEEE 13th international conference on data mining (ICDM), Miami, pp 938–943

    Google Scholar 

  • Quaglietta E (2013) Supporting the design of railway systems by means of a Sobol variance-based sensitivity analysis. Transp Res Part C 34:38–54

    Article  Google Scholar 

  • Ranjan P, Spencer N (2014) Space-filling Latin hypercube designs based on randomization restrictions in factorial experiments. Stat Probab Lett (in press)

    Google Scholar 

  • Rasmussen CE, Nickisch H (2010) Gaussian processes for machine learning (GPML) toolbox. J Mach Learn Res 11:3011–3015

    Google Scholar 

  • Rasmussen CE, Williams C (2006) Gaussian processes for machine learning. MIT, Cambridge

    Google Scholar 

  • Razavi S, Tolson BA, Burn DH (2012) Review of surrogate modeling in water resources. Water Resour Res 48, W07401:1–322

    Google Scholar 

  • Razavi S, Gupta HV (2015) What do we mean by sensitivity analysis? The need for comprehensive characterization of “global” sensitivity in earth and environmental systems models. Water Resour Res 51 (in press)

    Google Scholar 

  • Risk J, Ludkovski M (2015) Statistical emulators for pricing and hedging longevity risk products. Preprint arXiv:1508.00310

    Google Scholar 

  • Roustant O, Ginsbourger D, Deville Y (2012) DiceKriging, DiceOptim: two R packages for the analysis of computer experiments by Kriging-based metamodeling and optimization. J Stat Softw 51(1):1–55

    Article  Google Scholar 

  • Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments (includes comments and rejoinder). Stat Sci 4(4):409–435

    Article  Google Scholar 

  • Salemi P, Staum J, Nelson BL (2013) Generalized integrated Brownian fields for simulation metamodeling. In: Pasupathy R, Kim S-H, Tolk A, Hill R, Kuhl ME (eds) Proceedings of the 2013 winter simulation conference, Washington, DC, pp 543–554

    Chapter  Google Scholar 

  • Saltelli A, Annoni P, Azzini I, Campolongo F, Ratto M, Tarantola S (2010) Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Comput Phys Commun 181:259–270

    Article  Google Scholar 

  • Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D, Saisana M, Tarantola S (2008) Global sensitivity analysis: the primer. Wiley, Chichester

    Google Scholar 

  • Santner TJ, Williams BJ, Notz WI (2003) The design and analysis of computer experiments. Springer, New York

    Book  Google Scholar 

  • Shahraki AF, Noorossana R (2014) Reliability-based robust design optimization: a general methodology using genetic algorithm. Comput Ind Eng 74:199–207

    Article  Google Scholar 

  • Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang R-J (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidiscip Optim 27(5):302–313

    Google Scholar 

  • Simpson TW, Mauery TM, Korte JJ, Mistree F (2001) Kriging metamodels for global approximation in simulation-based multidisciplinary design. AIAA J 39(12):853–863

    Article  Google Scholar 

  • Sobol IM (1990) Sensitivity estimates for non-linear mathematical models. Matematicheskoe Modelirovanie 2:112–118

    Google Scholar 

  • Song E, Nelson BL, Pegden D (2014) Advanced tutorial: input uncertainty quantification. In: Tolk A, Diallo SY, Ryzhov IO, Yilmaz L, Buckley S, Miller JA (eds) Proceedings of the 2014 winter simulation conference, Savannah, pp 162–176

    Google Scholar 

  • Spöck G, Pilz J (2015) Incorporating covariance estimation uncertainty in spatial sampling design for prediction with trans-Gaussian random fields. Front Environ Sci 3(39):1–22

    Google Scholar 

  • Stein ML (1999) Statistical interpolation of spatial data: some theory for Kriging. Springer, New York

    Book  Google Scholar 

  • Storlie CB, Swiler LP, Helton JC, Sallaberry CJ (2009) Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models. Reliab Eng Syst Saf 94(11): 1735–1763

    Article  Google Scholar 

  • Stripling HF, Adams ML, McClarren RG, Mallick BK (2011) The method of manufactured universes for validating uncertainty quantification methods. Reliab Eng Syst Saf 96(9):1242–1256

    Article  Google Scholar 

  • Sun L, Hong LJ, Hu Z (2014) Balancing exploitation and exploration in discrete optimization via simulation through a Gaussian process-based search. Oper Res 62(6):1416–1438

    Article  Google Scholar 

  • Sundararajan S, Keerthi SS (2001) Predictive approach for choosing hyperparameters in Gaussian processes. Neural Comput 13(5):1103–1118

    Article  Google Scholar 

  • Tajbakhsh DS, Del Castillo E, Rosenberger JL (2014) A fully Bayesian approach to sequential optimization of computer metamodels for process improvement. Qual Reliab Eng Int 30(4):449–462

    Article  Google Scholar 

  • Tan MHY (2014a) Robust parameter design with computer experiments using orthonormal polynomials. Technometrics (in press)

    Google Scholar 

  • Tan MHY (2014b) Stochastic polynomial interpolation for uncertainty quantification with computer experiments. Technometrics (in press)

    Google Scholar 

  • Tan MHY (2015) Monotonic quantile regression with Bernstein polynomials for stochastic simulation. Technometrics (in press)

    Google Scholar 

  • Thiart C, Ngwenya MZ, Haines LM (2014) Investigating ‘optimal’ kriging variance estimation using an analytic and a bootstrap approach. J S Afr Inst Min Metall 114:613–618

    Google Scholar 

  • Toal DJJ, Bressloff NW, Keane AJ (2008) Kriging hyperparameter tuning strategies. AIAA J 46(5):1240–1252

    Article  Google Scholar 

  • Toropov VV, Schramm U, Sahai A, Jones R, Zeguer T (2005) Design optimization and stochastic analysis based on the moving least squares method. In: 6th world congress of structural and multidisciplinary optimization, Rio de Janeiro, paper no. 9412

    Google Scholar 

  • Tuo RC, Wu FJ, Yuc D (2014) Surrogate modeling of computer experiments with different mesh densities. Technometrics 56(3):372–380

    Article  Google Scholar 

  • Ulaganathan S, Couckuyt I, Dhaene T, Laermans E (2014) On the use of gradients in Kriging surrogate models. In: Tolk A, Diallo SY, Ryzhov IO, Yilmaz L, Buckley S, Miller JA (eds) Proceedings of the 2014 winter simulation conference, Savannah, pp 2692–2701

    Google Scholar 

  • Van Beers WCM, Kleijnen JPC (2003) Kriging for interpolation in random simulation. J Oper Res Soc 54:255–262

    Article  Google Scholar 

  • Van Beers WCM, Kleijnen JPC (2008) Customized sequential designs for random simulation experiments: Kriging metamodeling and bootstrapping. Eur J Oper Res 186(3):1099–1113

    Article  Google Scholar 

  • Viana FAC, Haftka RT (2009) Cross validation can estimate how well prediction variance correlates with error. AIAA J 47(9):2266–2270

    Article  Google Scholar 

  • Viana FAC, Simpson TW, Balabanov V, Toropov V (2014) Metamodeling in multidisciplinary design optimization: how far have we really come? AIAA J 52(4):670–690

    Article  Google Scholar 

  • Vieira H, Sanchez S, Kienitz KH, Belderrain MCN (2011) Generating and improving orthogonal designs by using mixed integer programming. Eur J Oper Res 215:629–638

    Article  Google Scholar 

  • Vose D (2000) Risk analysis; a quantitative guide, 2nd edn. Wiley, Chichester

    Google Scholar 

  • Wackernagel H (2003) Multivariate geostatistics: an introduction with applications, 3rd edn. Springer, Berlin

    Book  Google Scholar 

  • Wang C, Duan Q, Gong W, Ye A, Di Z, Miao C (2014) An evaluation of adaptive surrogate modeling based optimization with two benchmark problems. Environ Model Softw 60:167–179

    Article  Google Scholar 

  • Wei P, Lu Z, Song J (2015) Variable importance analysis: a comprehensive review. Reliab Eng Syst Saf 142:399–432

    Article  Google Scholar 

  • Wit E, Van den Heuvel E, Romeijn J-W (2012) All models are wrong …: an introduction to model uncertainty, Statistica Neerlandica 66(3):217–236

    Article  Google Scholar 

  • Xie W, Nelson BL, Barton RR (2014) A Bayesian framework for quantifying uncertainty in stochastic simulation. Oper Res (in press)

    Google Scholar 

  • Xu J, Zhang S, Huang E, Chen C-H, Lee H, Celik N (2014) Efficient multi-fidelity simulation optimization. In: Tolk A, Diallo SY, Ryzhov IO, Yilmaz L, Buckley S, Miller JA (eds) Proceedings of the 2014 winter simulation conference, Savannah, pp 3940–3951

    Google Scholar 

  • Yang X, Chen H, Liu MQ (2014) Resolvable orthogonal array-based uniform sliced Latin hypercube designs. Stat Probab Lett 93:108–115

    Article  Google Scholar 

  • Yin J, Ng SH, Ng KM (2009) A study on the effects of parameter estimation on Kriging model’s prediction error in stochastic simulation. In: Rossini MD, Hill RR, Johansson B, Dunkin A, Ingalls RG (eds) Proceedings of the 2009 winter simulation conference, Austin, pp 674–685

    Google Scholar 

  • Yin J, Ng SH, Ng KM (2010) A Bayesian metamodeling approach for stochastic simulations. In: Johansson B, Jain S, Montoya-Torres J, Hugan J, Yücesan E (eds) Proceedings of the 2010 winter simulation conference, Baltimore, pp 1055–1066

    Google Scholar 

  • Yuan J, Ng SH (2015) An integrated approach to stochastic computer model calibration, validation and prediction. Trans Model Comput Simul 25(3), Article No. 18

    Google Scholar 

  • Zhang Z (2007) New modeling procedures for functional data in computer experiments. Doctoral dissertation, Department of Statistics, Pennsylvania State University, University Park

    Google Scholar 

  • Zhang Z, Li R, Sudjianto A (2007) Modeling computer experiments with multiple responses. SAE Int 2007-01-1655

    Google Scholar 

  • Zhou Q, Qian PZG, Zhou S (2011) A simple approach to emulation for computer models with qualitative and quantitative factors. Technometrics 53:266–273

    Article  Google Scholar 

  • Zuniga MM, Kucherenko S, Shah N (2013) Metamodelling with independent and dependent inputs. Comput Phys Commun 184(6):1570–1580

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management, Tilburg University, Tilburg, The Netherlands

    Jack P. C. Kleijnen

Authors
  1. Jack P. C. Kleijnen
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kleijnen, J.P.C. (2015). Kriging Metamodels and Their Designs. In: Design and Analysis of Simulation Experiments. International Series in Operations Research & Management Science, vol 230. Springer, Cham. https://doi.org/10.1007/978-3-319-18087-8_5

Download citation

Publish with us

Policies and ethics

Search

Navigation