Space-Filling Designs for Computer Experiments

  • Thomas J. Santner
  • Brian J. Williams
  • William I. Notz
Part of the Springer Series in Statistics book series (SSS)


This chapter and the next discuss how to select inputs at which to compute the output of a computer experiment to achieve specific goals. The inputs one selects constitute the “experimental design.” As in previous chapters, the inputs are referred to as “runs.” The region corresponding to the values of the inputs that is to be studied is called the experimental region. A point in this region corresponds to a specific set of values of the inputs. Thus, an experimental design is a specification of points (runs) in the experimental region at which the response is to be computed.


  1. Atkinson AC, Donev AN (1992) Optimum experimental designs. Clarendon Press, OxfordzbMATHGoogle Scholar
  2. Ba S, Joseph VR (2011) Multi-layer designs for computer experiments. J Am Stat Assoc 106:1139–1149MathSciNetCrossRefGoogle Scholar
  3. Ba S, Myers WR, Brenneman WA (2015) Optimal sliced Latin hypercube designs. Technometrics 57(4):479–487MathSciNetCrossRefGoogle Scholar
  4. Bates RA, Buck RJ, Riccomagno E, Wynn HP (1996) Experimental design and observation for large systems. J R Stat Soc Ser B 58:77–94MathSciNetzbMATHGoogle Scholar
  5. Bernardo MC, Buck RJ, Liu L, Nazaret WA, Sacks J, Welch WJ (1992) Integrated circuit design optimization using a sequential strategy. IEEE Trans Comput Aided Des 11:361–372CrossRefGoogle Scholar
  6. Box GE, Draper NR (1987) Empirical model-building and response surfaces. Wiley, New York, NYzbMATHGoogle Scholar
  7. Box G, Hunter W, Hunter J (1978) Statistics for experimenters. Wiley, New York, NYzbMATHGoogle Scholar
  8. Bratley P, Fox BL, Niederreiter H (1994) Algorithm 738: programs to generate Niederreiter’s low-discrepancy sequences. ACM Trans Math Softw 20:494–495CrossRefGoogle Scholar
  9. Butler NA (2001) Optimal and orthogonal Latin hypercube designs for computer experiments. Biometrika 88:847–857MathSciNetCrossRefGoogle Scholar
  10. Chapman WL, Welch WJ, Bowman KP, Sacks J, Walsh JE (1994) Arctic sea ice variability: model sensitivities and a multidecadal simulation. J Geophys Res 99(C1):919–935CrossRefGoogle Scholar
  11. Chen RB, Wang W, Wu CFJ (2011) Building surrogates with overcomplete bases in computer experiments with applications to bistable laser diodes. IEE Trans 182:978–988Google Scholar
  12. Cioppa TM, Lucas TW (2007) Efficient nearly orthogonal and space-filling Latin hypercubes. Technometrics 49:45–55MathSciNetCrossRefGoogle Scholar
  13. Craig PC, Goldstein M, Rougier JC, Seheult AH (2001) Bayesian forecasting for complex systems using computer simulators. J Am Stat Assoc 96:717–729MathSciNetCrossRefGoogle Scholar
  14. Dean AM, Voss D, Draguljic D (2017) Design and analysis of experiments, 2nd edn. Springer, New York, NYCrossRefGoogle Scholar
  15. Dette H, Pepelyshev A (2010) Generalized Latin hypercube designs for computer experiments. Technometrics 52:421–429MathSciNetCrossRefGoogle Scholar
  16. Draguljić D, Santner TJ, Dean AM (2012) Non-collapsing spacing-filling designs for bounded polygonal regions. Technometrics 54:169–178MathSciNetCrossRefGoogle Scholar
  17. Fang KT, Lin DKJ, Winker P, Zhang Y (2000) Uniform design: theory and application. Technometrics 42:237–248MathSciNetCrossRefGoogle Scholar
  18. Fang KT, Li R, Sudjianto A (2006) Design and modeling for computer experiments. Chapman & Hall/CRC, Boca Raton, FLGoogle Scholar
  19. Halton JH (1960) On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals. Numer Math 2(1):84–90MathSciNetCrossRefGoogle Scholar
  20. Handcock MS (1991) On cascading Latin hypercube designs and additive models for experiments. Commun Stat Theory Methods 20(2):417–439MathSciNetCrossRefGoogle Scholar
  21. Hayeck GT (2009) The kinematics of the upper extremity and subsequent effects on joint loading and surgical treatment. PhD thesis, Cornell University, Ithaca, NYGoogle Scholar
  22. Hedayat A, Sloane N, Stufken J (1999) Orthogonal arrays. Springer, New York, NYCrossRefGoogle Scholar
  23. Hickernell FJ (1998) A generalized discrepancy and quadrature error bound. Math Comput 67:299–322MathSciNetCrossRefGoogle Scholar
  24. John JA (1987) Cyclic designs. Chapman & Hall, New York, NYCrossRefGoogle Scholar
  25. John PWM (1980) Incomplete block designs. M. Dekker, Inc., New York, NYzbMATHGoogle Scholar
  26. Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stat Plann Inf 26:131–148MathSciNetCrossRefGoogle Scholar
  27. Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492MathSciNetCrossRefGoogle Scholar
  28. Joseph VR, Gul E, Ba S (2015) Maximum projection designs for computer experiments. Biometrika 102(2):371–380MathSciNetCrossRefGoogle Scholar
  29. Kennedy MC, O’Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87:1–13MathSciNetCrossRefGoogle Scholar
  30. Kennedy MC, O’Hagan A (2001) Bayesian calibration of computer models (with discussion). J R Stat Soc Ser B 63:425–464MathSciNetCrossRefGoogle Scholar
  31. Liefvendahl M, Stocki R (2006) A study on algorithms for optimization of Latin hypercubes. J Stat Plann Inf 136:3231–3247MathSciNetCrossRefGoogle Scholar
  32. Loeppky JL, Sacks J, Welch WJ (2009) Choosing the sample size of a computer experiment: a practical guide. Technometrics 51(4):366–376MathSciNetCrossRefGoogle Scholar
  33. Loeppky JL, Moore LM, Williams BJ (2012) Projection array based designs for computer experiments. J Stat Plann Inf 142:1493–1505MathSciNetCrossRefGoogle Scholar
  34. McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21:239–245MathSciNetzbMATHGoogle Scholar
  35. Mease D, Bingham D (2006) Latin hyperrectangle sampling for computer experiments. Technometrics 48:467–477MathSciNetCrossRefGoogle Scholar
  36. Morris MD, Mitchell TJ (1995) Exploratory designs for computational experiments. J Stat Plann Inf 43:381–402CrossRefGoogle Scholar
  37. Niederreiter H (1988) Low-discrepancy and low-dispersion sequences. J Number Theory 30:51–70MathSciNetCrossRefGoogle Scholar
  38. Niederreiter H (1992) Random number generation and quasi-Monte Carlo methods. SIAM, Philadelphia, PACrossRefGoogle Scholar
  39. Owen AB (1992a) A central limit theorem for Latin hypercube sampling. J R Stat Soc Ser B Methodol 54:541–551MathSciNetzbMATHGoogle Scholar
  40. Owen AB (1992b) Orthogonal arrays for computer experiments, integration and visualization. Stat Sinica 2:439–452MathSciNetzbMATHGoogle Scholar
  41. Owen AB (1995) Randomly permuted (t, m, s)-nets and (t, s) sequences. In: Niederreiter H, Shiue PJS (eds) Monte Carlo and quasi-Monte Carlo methods in scientific computing. Springer, New York, NY, pp 299–317CrossRefGoogle Scholar
  42. Park JS (1994) Optimal Latin-hypercube designs for computer experiments. J Stat Plann Inf 39:95–111MathSciNetCrossRefGoogle Scholar
  43. Pukelsheim F (1993) Optimal design of experiments. Wiley, New York, NYzbMATHGoogle Scholar
  44. Qian PZ, Seepersad CC, Joseph VR, Allen JK, Wu CFJ (2006) Building surrogate models with details and approximate simulations. ASME J Mech Des 128:668–677CrossRefGoogle Scholar
  45. Qian PZG (2009) Nested Latin hypercube designs. Biometrika 96:957–970MathSciNetCrossRefGoogle Scholar
  46. Qian PZG (2012) Sliced Latin hypercube designs. J Am Stat Assoc 107:393–399MathSciNetCrossRefGoogle Scholar
  47. Qian PZG, Wu CFJ (2008) Bayesian hierarchical modeling for integrating low-accuracy and high-accuracy experiments. Technometrics 50(2):192–204MathSciNetCrossRefGoogle Scholar
  48. Raghavarao D (1971) Constructions and combinatorial problems in design of experiments. Wiley, New York, NYzbMATHGoogle Scholar
  49. Silvey SD (1980) Optimal design: an introduction to the theory for parameter estimation. Chapman & Hall, New York, NYCrossRefGoogle Scholar
  50. Sobol´ IM (1967) On the distribution of points in a cube and the approximate evaluation of integrals. USSR Comput Math Math Phys 7(4):86–112MathSciNetCrossRefGoogle Scholar
  51. Sobol´ IM (1976) Uniformly distributed sequences with an additional uniform property. USSR Comput Math Math Phys 16(5):236–242CrossRefGoogle Scholar
  52. Stein ML (1987) Large sample properties of simulations using Latin hypercube sampling. Technometrics 29:143–151MathSciNetCrossRefGoogle Scholar
  53. Stinstra E, den Hertog D, Stehouwer P, Vestjens A (2003) Constrained maximin designs for computer experiments. Technometrics 45(4):340–346MathSciNetCrossRefGoogle Scholar
  54. Street AP, Street DJ (1987) Combinatorics of experimental design. Oxford University Press, OxfordzbMATHGoogle Scholar
  55. Tan MHY (2013) Minimax designs for finite design regions. Technometrics 55:346–358MathSciNetCrossRefGoogle Scholar
  56. Tang B (1993) Orthogonal array-based Latin hypercubes. J Am Stat Assoc 88:1392–1397MathSciNetCrossRefGoogle Scholar
  57. Trosset MW (1999) Approximate maximin distance designs. In: ASA Proceedings of the section on physical and engineering sciences. American Statistical Association, Alexandria, VA, pp 223–227Google Scholar
  58. Vazquez E, Bect J (2011) Sequential search based on kriging: convergence analysis of some algorithms. Proceedings of the 58th world statistical congress of the ISI, pp 1241–1250Google Scholar
  59. Welch WJ (1985) ACED: algorithms for the construction of experimental designs. Am Stat 39:146CrossRefGoogle Scholar
  60. Welch WJ, Buck RJ, Sacks J, Wynn HP, Mitchell TJ, Morris MD (1992) Screening, predicting, and computer experiments. Technometrics 34:15–25CrossRefGoogle Scholar
  61. Wiens DP (1991) Designs for approximately linear regression: Two optimality properties of uniform designs. Stat Probab Lett 12:217–221MathSciNetCrossRefGoogle Scholar
  62. Williams BJ, Loeppky JL, Moore LM, Macklem MS (2011) Batch sequential design to achieve predictive maturity with calibrated computer models. Reliab Eng Syst Saf 96(9):1208–1219CrossRefGoogle Scholar
  63. Wu CFJ, Hamada M (2009) Experiments: planning, analysis, and parameter design optimization, 2nd edn. Wiley, New York, NYzbMATHGoogle Scholar
  64. Ye KQ (1998) Orthogonal column Latin hypercubes and their application in computer experiments. J Am Stat Assoc 93:1430–1439MathSciNetCrossRefGoogle Scholar
  65. Ye KQ, Li W, Sudjianto A (2000) Algorithmic construction of optimal symmetric Latin hypercube designs. J Stat Plann Inf 90(1):145–159MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Thomas J. Santner
    • 1
  • Brian J. Williams
    • 2
  • William I. Notz
    • 1
  1. 1.Department of StatisticsThe Ohio State UniversityColumbusUSA
  2. 2.Statistical Sciences GroupLos Alamos National LaboratoryLos AlamosUSA

Personalised recommendations