Sequential Design of Computer Experiments for Constrained Optimization

  • Brian J. Williams
  • Thomas J. Santner
  • William I. Notz
  • Jeffrey S. Lehman


This paper proposes a sequential method of designing computer or physical experiments when the goal is to optimize one integrated signal function subject to constraints on the integral of a second response function. Such problems occur, for example, in industrial problems where the computed responses depend on two types of inputs: manufacturing variables and noise variables. In industrial settings, manufacturing variables are determined by the product designer; noise variables represent field conditions which are modeled by specifying a probability distribution for these variables. The update scheme of the proposed method selects the control portion of the next input site to maximize a posterior expected “improvement” and the environmental portion of this next input is selected to minimize the mean square prediction error of the objective function at the new control site. The method allows for dependence between the objective and constraint functions. The efficacy of the algorithm relative to the single-stage design and relative to a design assuming independent responses is illustrated. Implementation issues for the deterministic and measurement error cases are discussed as are some generalizations of the method.


Computer Experiment Constraint Function Sequential Design Spatial Autoregressive Model Control Portion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



This work was sponsored, in part, by grants DMS-0406026 and DMS-0806134 (The Ohio State University) from the National Science Foundation. The authors would like to thank Han Gang for computational help.


  1. Bernardo, M. C., Buck, R., Liu, L., Nazaret, W. A., Sacks, J. & Welch, W. J. (1992). Integrated circuit design optimization using a sequential strategy, IEEE Transactions on Computer-Aided Design 11: 361–372.CrossRefGoogle Scholar
  2. Chang, P. B.,Williams, B. J., Bawa Bhalla, K. S., Belknap, T.W., Santner, T. J., Notz,W. I.&Bartel, D. L. (2001). Robust design and analysis of total joint replacements: Finite element model experiments with environmental variables, Journal of Biomechanical Engineering 123: 239– 246.CrossRefGoogle Scholar
  3. Currin, C.,Mitchell,T. J.,Morris,M. D.&Ylvisaker,D. (1991). Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments, Journal of the American Statistical Association 86: 953–963.CrossRefMathSciNetGoogle Scholar
  4. Handcock,M. S.&Stein,M. L. (1993). Abayesian analysis of kriging, Technometrics 35: 403–410.CrossRefGoogle Scholar
  5. Haylock, R. G. & O’Hagan, A. (1996). On inference for outputs of computationally expensive algorithms with uncertainty on the inputs, in J. Bernardo, J. Berger, A. Dawid & A. Smith (eds), Bayesian Statistics, Vol. 5, Oxford University Press, pp. 629–637.Google Scholar
  6. Jones, D. R., Schonlau, M. & Welch, W. J. (1998). Efficient global optimization of expensive black–box functions, Journal of Global Optimization 13: 455–492.MATHCrossRefMathSciNetGoogle Scholar
  7. Kennedy, M. C. & O’Hagan, A. (2000). Predicting the output from a complex computer code when fast approximations are available, Biometrika 87: 1–13.MATHCrossRefMathSciNetGoogle Scholar
  8. Lehman, J. (2002). Sequential Design of Computer Experiments for Robust Parameter Design, PhD thesis, Department of Statistics, Ohio State University, Columbus, OH USA.Google Scholar
  9. Nelder, J. A. & Mead, R. (1965). A simplex method for function minimization, Computer Journal 7: 308–313.MATHGoogle Scholar
  10. O’Hagan, A. (1992). Some bayesian numerical analysis, in J. Bernardo, J. Berger, A. Dawid & A. Smith (eds), Bayesian Statistics, Vol. 4, Oxford University Press, pp. 345–363.Google Scholar
  11. Ong, L.-T. (2004). Stability of Uncemented Acetabular Components: Design, Patient-Dependent, and Surgical Effects & Complications Due to Trapped Interfacial Fluid, PhD thesis, Sibley School of Mechanical and Aerospace Engineering, Cornell University.Google Scholar
  12. Sacks, J., Schiller, S. B. & Welch,W. J. (1989). Designs for computer experiments, Technometrics 31: 41–47.CrossRefMathSciNetGoogle Scholar
  13. Santner, T. J., Williams, B. J. & Notz,W. I. (2003). The Design and Analysis of Computer Experiments, Springer Verlag, New York.MATHGoogle Scholar
  14. Schonlau, M., Welch, W. J. & Jones, D. R. (1997). Global versus local search in constrained optimization of computer models, Technical Report RR-97-11, University of Waterloo and General Motors.Google Scholar
  15. Schonlau, M., Welch, W. J. & Jones, D. R. (1998). Global versus local search in constrained optimization of computer models, in N. Flournoy, W. Rosenberger & W. Wong (eds), New Developments and Applications in Experimental Design, Vol. 34, Institute of Mathematical Statistics, pp. 11–25.Google Scholar
  16. Tang, B. (1993). Orthogonal array-based latin hypercubes, Journal of the American Statistical Association 88: 1392–1397.MATHCrossRefMathSciNetGoogle Scholar
  17. VerHoef, J. M.&Barry,R. P. (1998). Constructing and fittingmodels for cokriging and multivariable spatial prediction, Journal of Statistical Planning and Inference 69: 275–294.CrossRefMathSciNetGoogle Scholar
  18. Welch,W. J., Buck, R. J., Sacks, J.,Wynn, H. P.,Mitchell, T. J. &Morris, M. D. (1992). Screening, predicting, and computer experiments, Technometrics 34: 15–25.CrossRefGoogle Scholar
  19. Williams, B. J., Santner, T. J. & Notz,W. I. (2000a). Sequential design of computer experiments for constrained optimization of integrated response functions, Technical Report 658, Department of Statistics, The Ohio State University.Google Scholar
  20. Williams, B. J., Santner, T. J. & Notz, W. I. (2000b). Sequential design of computer experiments to minimize integrated response functions, Statistica Sinica 10: 1133–1152.MATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Brian J. Williams
    • 1
  • Thomas J. Santner
    • 2
  • William I. Notz
    • 2
  • Jeffrey S. Lehman
    • 3
  1. 1.Los Alamos National LaboratoryLos AlamosUSA
  2. 2.Department of StatisticsThe Ohio State UniversityColumbusUSA
  3. 3.JPMorganChaseHome Finance Marketing AnalyticsColumbusUSA

Personalised recommendations