Skip to main content
SpringerLink
Log in

Kriging Is Well-Suited to Parallelize Optimization

  • Chapter
Computational Intelligence in Expensive Optimization Problems

Part of the book series: Adaptation Learning and Optimization ((ALO,volume 2))

Abstract

The optimization of expensive-to-evaluate functions generally relies on metamodel-based exploration strategies. Many deterministic global optimization algorithms used in the field of computer experiments are based on Kriging (Gaussian process regression). Starting with a spatial predictor including a measure of uncertainty, they proceed by iteratively choosing the point maximizing a criterion which is a compromise between predicted performance and uncertainty. Distributing the evaluation of such numerically expensive objective functions on many processors is an appealing idea. Here we investigate a multi-points optimization criterion, the multipoints expected improvement (\(q-{\mathbb E}I\)), aimed at choosing several points at the same time. An analytical expression of the \(q-{\mathbb E}I\) is given when q = 2, and a consistent statistical estimate is given for the general case. We then propose two classes of heuristic strategies meant to approximately optimize the \(q-{\mathbb E}I\), and apply them to the classical Branin-Hoo test-case function. It is finally demonstrated within the covered example that the latter strategies perform as good as the best Latin Hypercubes and Uniform Designs ever found by simulation (2000 designs drawn at random for every q ∈ [1,10]).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abrahamsen, P.: A review of gaussian random fields and correlation functions, 2nd edn. Tech. Rep. 917, Norwegian Computing Center, Olso (1997)

    Google Scholar 

  2. Antoniadis, A., Berruyer, J., Carmona, R.: Régression non linéaire et applications. Economica, Paris (1992)

    Google Scholar 

  3. Baker, C., Watson, L.T., Grossman, B., Mason, W.H., Haftka, R.T.: Parallel global aircraft configuration design space exploration. Practical parallel computing, 79–96 (2001)

    Google Scholar 

  4. Bishop, C.: Neural Networks for Pattern Recognition. Oxford Univ. Press, Oxford (1995)

    Google Scholar 

  5. Blum, C.: Ant colony optimization: introduction and recent trends. Physics of Life Review 2, 353–373 (2005)

    Article  Google Scholar 

  6. development Core Team R: R: A language and environment for statistical computing (2006), http://www.R-project.org

  7. Cressie, N.: Statistics for spatial data. Wiley series in probability and mathematical statistics (1993)

    Google Scholar 

  8. Dreyfus, G., Martinez, J.M.: Réseaux de neurones. Eyrolles (2002)

    Google Scholar 

  9. Eiben, A., Smith, J.: Introduction to Evolutionary Computing. Springer, Heidelberg (2003)

    MATH  Google Scholar 

  10. Emmerich, M., Giannakoglou, K., Naujoks, B.: Single-and multiobjective optimization assisted by gaussian random field metamodels. IEEE Transactions on Evolutionary Computation 10(4), 421–439 (2006)

    Article  Google Scholar 

  11. Geman, D., Jedynak, B.: An active testing model for tracking roads in satellite images. Tech. rep., Institut National de Recherches en Informatique et Automatique (INRIA) (December 1995)

    Google Scholar 

  12. Genton, M.: Classes of kernels for machine learning: A statistics perspective. Journal of Machine Learning Research 2, 299–312 (2001)

    Article  Google Scholar 

  13. Ginsbourger, D.: Multiples métamodèles pour l’approximation et l’optimisation de fonctions numériques multivariables. PhD thesis, Ecole Nationale Supérieure des Mines de Saint-Etienne (2009)

    Google Scholar 

  14. Ginsbourger, D., Le Riche, R., Carraro, L.: A multipoints criterion for parallel global optimization of deterministic computer experiments. In: Non-Convex Programming 2007 (2007)

    Google Scholar 

  15. Goria, S.: Evaluation d’un projet minier: approche bayésienne et options réelles. PhD thesis, Ecole des Mines de Paris (2004)

    Google Scholar 

  16. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Heidelberg (2001)

    MATH  Google Scholar 

  17. Henkenjohann, N., Göbel, R., Kleiner, M., Kunert, J.: An adaptive sequential procedure for efficient optimization of the sheet metal spinning process. Qual. Reliab. Engng. Int. 21, 439–455 (2005)

    Article  Google Scholar 

  18. Huang, D., Allen, T., Notz, W., Miller, R.: Sequential Kriging optimization using multiple fidelity evaluations. Sructural and Multidisciplinary Optimization 32, 369–382 (2006)

    Article  Google Scholar 

  19. Huang, D., Allen, T., Notz, W., Zheng, N.: Global optimization of stochastic black-box systems via sequential Kriging meta-models. Journal of Global Optimization 34, 441–466 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  20. Jones, D.: A taxonomy of global optimization methods based on response surfaces. Journal of Global Optimization 21, 345–383 (2001)

    Article  MATH  Google Scholar 

  21. Jones, D., Pertunen, C., Stuckman, B.: Lipschitzian optimization without the lipschitz constant. Journal of Optimization Theory and Application 79(1) (1993)

    Google Scholar 

  22. Jones, D., Schonlau, M., Welch, W.: Efficient global optimization of expensive black-box functions. Journal of Global Optimization 13, 455–492 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  23. Journel, A.: Fundamentals of geostatistics in five lessons. Tech. rep., Stanford Center for Reservoir Forecasting (1988)

    Google Scholar 

  24. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE Intl. Conf. on Neural Networks, vol. 4, pp. 1942–1948 (1995)

    Google Scholar 

  25. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)

    Article  MathSciNet  Google Scholar 

  26. Knowles, J.: Parego: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Transactions on Evolutionnary Computation (2005)

    Google Scholar 

  27. Koehler, J., Owen, A.: Computer experiments. Tech. rep., Department of Statistics, Stanford University (1996)

    Google Scholar 

  28. Kracker, H.: Methoden zur analyse von computerexperimenten mit anwendung auf die hochdruckblechumformung. Master’s thesis, Dortmund University (2006)

    Google Scholar 

  29. Krige, D.: A statistical approach to some basic mine valuation problems on the witwatersrand. J. of the Chem., Metal. and Mining Soc. of South Africa 52(6), 119–139 (1951)

    Google Scholar 

  30. Martin, J., Simpson, T.: A monte carlo simulation of the Kriging model. In: 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, NY, AIAA, AIAA-2004-4483, August 30 - September 2 (2004)

    Google Scholar 

  31. Martin, J., Simpson, T.: Use of Kriging models to approximate deterministic computer models. AIAA Journal 43(4), 853–863 (2005)

    Article  Google Scholar 

  32. Matheron, G.: Principles of geostatistics. Economic Geology 58, 1246–1266 (1963)

    Article  Google Scholar 

  33. Matheron, G.: La théorie des variables régionalisées et ses applications. Tech. rep., Centre de Morphologie Mathématique de Fontainebleau, Ecole Nationale Supérieure des Mines de Paris (1970)

    Google Scholar 

  34. O’Hagan, A.: Bayesian analysis of computer code outputs: a tutorial. Reliability Engineering and System Safety 91(91), 1290–1300 (2006)

    Article  Google Scholar 

  35. Paciorek, C.: Nonstationary gaussian processes for regression and spatial modelling. PhD thesis, Carnegie Mellon University (2003)

    Google Scholar 

  36. Ponweiser, W., Wagner, T., Biermann, D., Vincze, M.: Multiobjective optimization on a limited budget of evaluations using model-assisted S-metric selection. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 784–794. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  37. Praveen, C., Duvigneau, R.: Radial basis functions and Kriging metamodels for aerodynamic optimization. Tech. rep., INRIA (2007)

    Google Scholar 

  38. Queipo, N., Verde, A., Pintos, S., Haftka, R.: Assessing the value of another cycle in surrogate-based optimization. In: 11th Multidisciplinary Analysis and Optimization Conference, AIAA (2006)

    Google Scholar 

  39. Rasmussen, C., Williams, K.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  40. Ripley, B.: Stochastic Simulation. John Wiley and Sons, New York (1987)

    Book  MATH  Google Scholar 

  41. Roustant, O., Ginsbourger, D., Deville, Y.: The DiceKriging package: Kriging-based metamodeling and optimization for computer experiments. In: The UseR! Conference, Agrocampus-Ouest, Rennes, France (2009)

    Google Scholar 

  42. Sacks, J., Welch, W., Mitchell, T., Wynn, H.: Design and analysis of computer experiments. Statistical Science 4(4), 409–435 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  43. Santner, T., Williams, B., Notz, W.: The Design and Analysis of Computer Experiments. Springer, Heidelberg (2003)

    MATH  Google Scholar 

  44. Sasena, M.: Flexibility and efficiency enhancements for constrained global design optimization with Kriging approximations. PhD thesis, University of Michigan (2002)

    Google Scholar 

  45. Sasena, M.J., Papalambros, P., Goovaerts, P.: Exploration of metamodeling sampling criteria for constrained global optimization. Journal of Engineering Optimization (2002)

    Google Scholar 

  46. Sasena, M.J., Papalambros, P.Y., Goovaerts, P.: Global optimization of problems with disconnected feasible regions via surrogate modeling. In: Proceedings of the 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Atlanta, GA (2002)

    Google Scholar 

  47. Schonlau, M.: Computer experiments and global optimization. PhD thesis, University of Waterloo (1997)

    Google Scholar 

  48. Schonlau, M., Welch, W., Jones, D.: A data-analytic approach to bayesian global optimization. In: Proceedings of the A.S.A. (1997)

    Google Scholar 

  49. Ulmer, H., Streichert, F., Zell, A.: Evolution strategies assisted by gaussian processes with improved pre-selection criterion. Tech. rep., Center for Bioinformatics Tuebingen, ZBIT (2003)

    Google Scholar 

  50. Villemonteix, J.: Optimisation de fonctions coûteuses: Modèles gaussiens pour une utilisation théorie et pratique industrielle. PhD thesis, Université Paris-sud XI, Faculté des Sciences d’Orsay (2008)

    Google Scholar 

  51. Villemonteix, J., Vazquez, E., Walter, E.: An informational approach to the global optimization of expensive-to-evaluate functions. Journal of Global Optimization 44(4), 509–534 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  52. Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia (1990)

    MATH  Google Scholar 

  53. Williams, C., Rasmussen, C.: Gaussian processes for regression. In: Advances in Neural Information Processing Systems, vol. 8 (1996)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Département 3MI, Ecole Nationale Supérieure des Mines, 158 cours Fauriel, Saint-Etienne, France

    David Ginsbourger & Laurent Carraro

  2. CNRS (UMR 5146) and Département 3MI, Ecole Nationale Supérieure des Mines, 158 cours Fauriel, Saint-Etienne, France

    Rodolphe Le Riche

Authors
  1. David Ginsbourger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rodolphe Le Riche
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Laurent Carraro
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Mechanical Engineering and Science-Faculty of Engineering, Kyoto University, Yoshida-honmachi, Sakyo-Ku,, 606-8501, Kyoto, Japan

    Yoel Tenne

  2. Advanced Technology Centre, Rolls-Royce Singapore Pte Ltd, 50 Nanyang Avenue, Block N2, Level B3C,Unit 05-08, 639798, Singapore

    Chi-Keong Goh

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Ginsbourger, D., Le Riche, R., Carraro, L. (2010). Kriging Is Well-Suited to Parallelize Optimization. In: Tenne, Y., Goh, CK. (eds) Computational Intelligence in Expensive Optimization Problems. Adaptation Learning and Optimization, vol 2. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10701-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-10701-6_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-10700-9

  • Online ISBN: 978-3-642-10701-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation