Skip to main content
SpringerLink
Log in

A quasi-multistart framework for global optimization of expensive functions using response surface models

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

We present the AQUARS (A QUAsi-multistart Response Surface) framework for finding the global minimum of a computationally expensive black-box function subject to bound constraints. In a traditional multistart approach, the local search method is blind to the trajectories of the previous local searches. Hence, the algorithm might find the same local minima even if the searches are initiated from points that are far apart. In contrast, AQUARS is a novel approach that locates the promising local minima of the objective function by performing local searches near the local minima of a response surface (RS) model of the objective function. It ignores neighborhoods of fully explored local minima of the RS model and it bounces between the best partially explored local minimum and the least explored local minimum of the RS model. We implement two AQUARS algorithms that use a radial basis function model and compare them with alternative global optimization methods on an 8-dimensional watershed model calibration problem and on 18 test problems. The alternatives include EGO, GLOBALm, MLMSRBF (Regis and Shoemaker in INFORMS J Comput 19(4):497–509, 2007), CGRBF-Restart (Regis and Shoemaker in J Global Optim 37(1):113–135 2007), and multi level single linkage (MLSL) coupled with two types of local solvers: SQP and Mesh Adaptive Direct Search (MADS) combined with kriging. The results show that the AQUARS methods generally use fewer function evaluations to identify the global minimum or to reach a target value compared to the alternatives. In particular, they are much better than EGO and MLSL coupled to MADS with kriging on the watershed calibration problem and on 15 of the test problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abramson, M.A.: NOMADm Version 4.6 User’s Guide. Unpublished manuscript (2007)

  • Abramson M.A., Audet C.: Convergence of mesh adaptive direct search to second-order stationary points. SIAM J. Optim. 17(2), 606–619 (2006)

    Article  Google Scholar 

  • Aleman D.M., Romeijn H.E., Dempsey J.F.: A response surface approach to beam orientation optimization in intensity modulated radiation therapy treatment planning. INFORMS J. Comput. 21, 62–76 (2009)

    Article  Google Scholar 

  • Audet C., Dennis J.E. Jr.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(2), 188–217 (2006)

    Article  Google Scholar 

  • Björkman M., Holmström K.: Global optimization of costly nonconvex functions using radial basis functions. Optim. Eng. 1, 373–397 (2000)

    Article  Google Scholar 

  • Boender C.G.E., Rinnooy Kan A.H.G., Timmer G.T., Stougie L.: A stochastic method for global optimization. Math. Program. 22, 125–140 (1982)

    Article  Google Scholar 

  • Booker A.J., Dennis J.E., Frank P.D., Serafini D.B., Torczon V., Trosset M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Optim. 17(1), 1–13 (1999)

    Article  Google Scholar 

  • Buhmann M.D.: Radial Basis Functions. Cambridge University Press, Cambridge (2003)

    Book  Google Scholar 

  • Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-Region Methods. MPS-SIAM Series on Optimization (2000)

  • Conn A.R., Scheinberg K., Vicente L.N.: Introduction to Derivative-Free Optimization. SIAM, Philadelphia (2009)

    Book  Google Scholar 

  • Cressie N.: Statistics for Spatial Data. Wiley, New York (1993)

    Google Scholar 

  • Csendes T.: Nonlinear parameter estimation by global optimization—efficiency and reliability. Acta Cybern. 8, 361–370 (1988)

    Google Scholar 

  • Dixon L.C.W., Szegö G.: The global optimization problem: an introduction. In: Dixon, L.C.W., Szegö, G. (eds) Towards Global Optimization, vol. 2, pp. 1–15. North-Holland, Amsterdam (1978)

    Google Scholar 

  • Egea J.A., Vazquez E., Banga J.R., Marti R.: Improved scatter search for the global optimization of computationally expensive dynamic models. J. Global Optim. 43(2–3), 175–190 (2009)

    Article  Google Scholar 

  • Forrester A.I.J., Sobester A., Keane A.J.: Engineering Design via Surrogate Modelling a Practical Guide. Wiley, New York (2008)

    Book  Google Scholar 

  • Giunta A.A., Balabanov V., Haim D., Grossman B., Mason W.H., Watson L.T., Haftka R.T.: Aircraft multidisciplinary design optimisation using design of experiments theory and response surface modelling. Aeronaut. J. 101(1008), 347–356 (1997)

    Google Scholar 

  • Glover, F.: A template for scatter search and path relinking. In: Hao, J.-K., Lutton, E., Ronald, E., Schoenauer, M., Snyers, D. (eds.) Artificial Evolution, Lecture Notes in Computer Science, vol. 1363, pp. 13–54. Springer, Berlin (1998)

  • Gutmann H.-M.: A radial basis function method for global optimization. J. Global Optim. 19(3), 201–227 (2001)

    Article  Google Scholar 

  • Holmström K.: An adaptive radial basis algorithm (ARBF) for expensive black-box global optimization. J. Global Optim. 41(3), 447–464 (2008)

    Article  Google Scholar 

  • Huang D., Allen T.T., Notz W.I., Zeng N.: Global optimization of stochastic black-box systems via sequential kriging meta-models. J. Global Optim. 34(3), 441–466 (2006)

    Article  Google Scholar 

  • Jakobsson S., Patriksson M., Rudholm J., Wojciechowski A.: A method for simulation based optimization using radial basis functions. Optim. Eng. 11(4), 501–532 (2009)

    Article  Google Scholar 

  • Jones D.R., Schonlau M., Welch W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4), 455–492 (1998)

    Article  Google Scholar 

  • Kleijnen J.P.C.: Design and Analysis of Simulation Experiments. Springer, Berlin (2008)

    Google Scholar 

  • Kleijnen J.P.C.: Kriging metamodeling in simulation: a review. Eur. J. Oper. Res. 192(3), 707–716 (2009)

    Article  Google Scholar 

  • Kolda T.G., Lewis R.M., Torczon V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45(3), 385–482 (2003)

    Article  Google Scholar 

  • Laguna M., Marti R.: Scatter Search: Methodology and Implementations in C. Kluwer, Boston (2003)

    Book  Google Scholar 

  • Lasdon L., Duarte A., Glover F., Laguna M., Marti R.: Adaptive memory programming for constrained global optimization. Comput. Oper. Res. 37(8), 1500–1509 (2010)

    Article  Google Scholar 

  • Lophaven, S.N., Nielsen, H.B., Søndergaard, J.: DACE: A Matlab Kriging Toolbox, Version 2.0. Technical Report IMM-TR-2002-12, Informatics and Mathematical Modelling, Technical University of Denmark, DK-2800 Kgs. Lyngby(2002)

  • Marsden A.L., Wang M., Dennis J.E. Jr., Moin P.: Optimal aeroacoustic shape design using the surrogate management framework. Optim. Eng. 5(2), 235–262 (2004)

    Article  Google Scholar 

  • Myers R.H., Montgomery D.C.: Response Surface Methodology: Process and Product Optimization Using Designed Experiments. Wiley, New York (1995)

    Google Scholar 

  • Nocedal J., Wright S.J.: Numerical Optimization. Springer, New York (1999)

    Book  Google Scholar 

  • Oeuvray R., Bierlaire M.: BOOSTERS: a derivative-free algorithm based on radial basis functions. Int. J. Model. Simul. 29(1), 26–36 (2009)

    Google Scholar 

  • Oeuvray, R.: Trust-Region Methods Based on Radial Basis Functions With Application To Biomedical Imaging. Ph.D. thesis, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne (2005)

  • Powell M.J.D.: The theory of radial basis function approximation in 1990. In: Light, W. (eds) Advances in Numerical Analysis: Wavelets, Subdivision Algorithms and Radial Basis Functions, vol. 2, pp. 105–210. Oxford University Press, Oxford (1992)

    Google Scholar 

  • Powell M.J.D.: UOBYQA: unconstrained optimization by quadratic approximation. Math. Program. 92, 555–582 (2002)

    Article  Google Scholar 

  • Powell M.J.D.: The NEWUOA software for unconstrained optimization without derivatives. In: Di Pillo, G., Roma, M. (eds) Large-Scale Nonlinear Optimization, pp. 255–297. Springer, USA (2006)

    Chapter  Google Scholar 

  • Regis R.G., Shoemaker C.A.: A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput. 19(4), 497–509 (2007)

    Article  Google Scholar 

  • Regis R.G., Shoemaker C.A.: Improved strategies for radial basis function methods for global optimization. J. Global Optim. 37(1), 113–135 (2007)

    Article  Google Scholar 

  • Rinnooy Kan A.H.G., Timmer G.T.: Stochastic global optimization methods, part II: multi level methods. Math. Program. 39, 57–78 (1987)

    Article  Google Scholar 

  • Sacks J., Welch W.J., Mitchell T.J., Wynn H.P.: Design and analysis of computer experiments. Stat. Sci. 4, 409–435 (1989)

    Article  Google Scholar 

  • Schoen F.: A wide class of test functions for global optimization. J. Global Optim. 3, 133–137 (1993)

    Article  Google Scholar 

  • Sendin J.O.H, Banga J.R., Csendes T.: Extensions of a multistart clustering algorithm for constrained global optimization problems. Ind. Eng. Chem. Res. 48(6), 3014–3023 (2009)

    Article  Google Scholar 

  • Shoemaker C.A., Regis R.G., Fleming R.C.: Watershed calibration using multistart local optimization and evolutionary optimization with radial basis function approximation. Hydrol. Sci. J. 52(3), 450–465 (2007)

    Article  Google Scholar 

  • Simpson T.W., Mauery T.M., Korte J.J., Mistree F.: Kriging metamodels for global approximation in simulation-based multidisciplinary design optimization. AIAA J. 39(12), 2233–2241 (2001)

    Article  Google Scholar 

  • The Mathworks, Inc.: Matlab Optimization Toolbox: User’s Guide, Version 4. Natick, MA (2009)

  • Ugray Z., Lasdon L., Plummer J., Glover F., Kelley J., Marti R.: Scatter search and local NLP solvers: a multistart framework for global optimization. INFORMS J. Comput. 19(3), 328–340 (2007)

    Article  Google Scholar 

  • Viana, F.A.C.: SURROGATES Toolbox User’s Guide, Version 2.1, http://sites.google.com/site/felipeacviana/surrogatestoolbox (2010)

  • Villemonteix J., Vazquez E., Walter E.: An informational approach to the global optimization of expensive-to-evaluate functions. J. Global Optim. 44(4), 509–534 (2009)

    Article  Google Scholar 

  • Ye K.Q., Li W., Sudjianto A.: Algorithmic construction of optimal symmetric latin hypercube designs. J. Stat. Plan. Inference 90, 145–159 (2000)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Saint Joseph’s University, Philadelphia, PA, 19131, USA

    Rommel G. Regis

  2. School of Civil and Environmental Engineering, Center for Applied Mathematics, Cornell University, Ithaca, New York, 14853, USA

    Christine A. Shoemaker

  3. School of Operations Research and Information Engineering, Cornell University, Ithaca, New York, 14853, USA

    Christine A. Shoemaker

Authors
  1. Rommel G. Regis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christine A. Shoemaker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rommel G. Regis.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Regis, R.G., Shoemaker, C.A. A quasi-multistart framework for global optimization of expensive functions using response surface models. J Glob Optim 56, 1719–1753 (2013). https://doi.org/10.1007/s10898-012-9940-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-012-9940-1

Keywords

Search

Navigation