Exotic Bayesian Optimization

  • Francesco ArchettiEmail author
  • Antonio Candelieri
Part of the SpringerBriefs in Optimization book series (BRIEFSOPTI)


GO and BO have been considered first in “essentially unconstrained” conditions, where the solution was searched for within a bounded-box search space. Recently, due to methodological and application reasons, there has been an increasing interest in constrained global optimization (CGO).


  1. Abdolshah, M., Shilton, A., Rana, S., Gupta, S., Venkatesh, S.: Multi-objective Bayesian optimisation with preferences over objectives (2019). arXiv preprint arXiv:1902.04228
  2. Bachoc, F., Helbert, C., Picheny, V.: Gaussian process optimization with failures: classification and convergence proof (2019)Google Scholar
  3. Basudhar, A., Dribusch, C., Lacaze, S., Missoum, S.: Constrained efficient global optimization with support vector machines. Struct. Multidisciplinary Optim. 46, 201–221 (2012). Scholar
  4. Berkenkamp, F., Krause, A., Schoellig, A.P.: Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics (2016). arXiv preprint arXiv:1602.04450. doi:10.1177
  5. Calvin, J.M., Žilinskas, A.: On efficiency of bicriteria optimization. In: AIP Conference Proceedings, vol. 2070, no. 1, p. 020035. AIP Publishing (2019, February)Google Scholar
  6. Candelieri, A., Archetti, F.: Sequential model-based optimization with black-box constraints: feasibility determination via machine learning. In: AIP Conference Proceedings. p. 020010 (2019)Google Scholar
  7. Candelieri, A., Galuzzi, B.G., Giordani, I., Perego, R., Archetti, F.: Optimizing partially defined black-box functions under unknown constraints via Sequential Model Based Optimization: an application to Pump Scheduling Optimization in Water Distribution Networks. To appear in Proceedings of Learning and Intelligent Optimization conference (LION 13) (2019)Google Scholar
  8. Candelieri, A., Giordani, I., Archetti, F., Barkalov, K., Meyerov, I., Polovinkin, A., Sysoyev, A., Zolotykh, N.: Tuning hyperparameters of a SVM-based water demand forecasting system through parallel global optimization. Comput Oper Res (2018). Scholar
  9. Costabal, F.S., Perdikaris, P., Kuhl, E., Hurtado, D.E.: Multi-fidelity classification using Gaussian processes: accelerating the prediction of large-scale computational models (2019). arXiv preprint arXiv:1905.03406
  10. Digabel, S.L., Wild, S.M.: A taxonomy of constraints in simulation-based optimization (2015). arXiv preprint arXiv:1505.07881
  11. Emmerich, M., Klinkenberg, J.W.: The computation of the expected improvement in dominated hypervolume of Pareto front approximations. Rapport technique, Leiden University, 34, 7–3 (2008)Google Scholar
  12. Feliot, P., Bect, J., Vazquez, E.: A Bayesian approach to constrained single-and multi-objective optimization. J. Global Optim. 67(1–2), 97–133 (2017)MathSciNetCrossRefGoogle Scholar
  13. Frazier, P.I.: Bayesian optimization. In: Recent Advances in Optimization and Modeling of Contemporary Problems, pp. 255–278. INFORMS (2018)Google Scholar
  14. Gardner, J.R., Kusner, M.J., Xu, Z.E., Weinberger, K.Q., Cunningham, J.P.: Bayesian Optimization with Inequality Constraints Jacob. In: ICML, pp. 937–945 (2014)Google Scholar
  15. Ghoreishi, S.F., Allaire, D.: Multi-information source constrained Bayesian optimization. Struct. Multidisciplinary Optim. 1–15 (2018). Scholar
  16. Ginsbourger, D., Riche, R. Le, Carraro, L.: A multi-points criterion for deterministic parallel global optimization based on Gaussian processes. In: International Conference on Nonconvex Programming, NCP07, Rouen, France. 1–30 (2008)Google Scholar
  17. Ginsbourger, D., Riche, R. Le: Dealing with asynchronicity in parallel Gaussian Process based global optimization ∗. In: 4th International Conference of the ERCIM WG on Computing & Statistics (ERCIM’11) (2011)Google Scholar
  18. Gramacy, R.B., Lee, H.K.H.: Optimization under unknown constraints. Bayesian Statistics, 9 (2011)Google Scholar
  19. Gramacy, R.B., Gray, G.A., Le Digabel, S., Lee, H.K., Ranjan, P., Wells, G., Wild, S.M.: Modeling an augmented Lagrangian for blackbox constrained optimization. Technometrics 58(1), 1–11 (2016)MathSciNetCrossRefGoogle Scholar
  20. Hernández-Lobato, J.M., Gelbart, M.A., Hoffman, M.W., Adams, R.P., Ghahramani, Z.: Predictive entropy search for bayesian optimization with unknown constraints (2015)Google Scholar
  21. Horn, D., Wagner, T., Biermann, D., Weihs, C., Bischl, B.: Model-based multi-objective optimization: Taxonomy, multi-point proposal, toolbox and benchmark. In: International Conference on Evolutionary Multi-Criterion Optimization, pp. 64–78. Springer, Cham (2015, March)Google Scholar
  22. Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998)Google Scholar
  23. Kandasamy, K., Dasarathy, G., Schneider, J., Póczos, B.: Multi-fidelity bayesian optimisation with continuous approximations. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 1799–1808. JMLR. org (2017, August)Google Scholar
  24. Kandasamy, K., Krishnamurthy, A., Schneider, J., Poczos, B.: Asynchronous parallel Bayesian optimisation via Thompson sampling (2018). arXiv preprint arXiv:1705.09236
  25. Kandasamy, K., Neiswanger, W., Schneider, J., Poczos, B., Xing, E.: Neural architecture search with Bayesian optimisation and optimal transport (2019). arXiv preprint arXiv:1802.07191
  26. Klein, A., Falkner, S., Bartels, S., Hennig, P., Hutter, F.: Fast Bayesian optimization of machine learning hyperparameters on large datasets.In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, Brookline, MA, 54, 528–536 (2017)Google Scholar
  27. Letham, B., Karrer, B., Ottoni, G., Bakshy, E.: Constrained Bayesian optimization with noisy experiments. Bayesian Anal. 14(2), 495–519 (2019)MathSciNetCrossRefGoogle Scholar
  28. Linz, D.D., Huang, H., Zabinsky, Z.B.: Multi-fidelity simulation optimization with level set approximation using probabilistic branch and bound. In: Winter Simulation Conference (WSC), IEEE, pp. 2057–2068 (2017, December)Google Scholar
  29. Perdikaris, P., Karniadakis, G.E.: Model inversion via multi-fidelity Bayesian optimization: a new paradigm for parameter estimation in haemodynamics, and beyond. J. R. Soc. Interface 13(118), 20151107 (2016)CrossRefGoogle Scholar
  30. Picheny, V., Gramacy, R. B., Wild, S., Le Digabel, S.: Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian. In: Advances in Neural Information Processing Systems, pp. 1435–1443 (2016)Google Scholar
  31. Poloczek, M., Wang, J., Frazier, P. Multi-information source optimization. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, 30, pp. 4291–4301. Curran Associates, Red Hook, NY, (2017)Google Scholar
  32. Rudenko, L.I.: Objective functional approximation in a partially defined optimization problem. J. Math. Sci. 72(5), 3359–3363 (1994)MathSciNetCrossRefGoogle Scholar
  33. Sacher, M., Duvigneau, R., Le Maitre, O., Durand, M., Berrini, E., Hauville, F., Astolfi, J.A.: A classification approach to efficient global optimization in presence of non-computable domains. Struct. Multidisciplinary Optim. 58(4), 1537–1557 (2018)MathSciNetCrossRefGoogle Scholar
  34. Sen, R., Kandasamy, K., Shakkottai, S.: Multi-fidelity black-box optimization with hierarchical partitions. In: International Conference on Machine Learning, pp. 4545–4554 (2018, July)Google Scholar
  35. Sergeyev, Y.D., Kvasov, D.E.: Deterministic Global Optimization: an Introduction to the Di-agonal Approach. Springer (2017)Google Scholar
  36. Sergeyev, Y.D., Kvasov, D.E., Khalaf, F.M.: A one-dimensional local tuning algorithm for solving GO problems with partially defined constraints. Optim. Lett. 1(1), 85–99 (2007)MathSciNetCrossRefGoogle Scholar
  37. Sui, Y., Gotovos, A., Burdick, J.W., Krause, A.: Safe exploration for optimization with gaussian processes. In: International Conference on Machine Learning (ICML), (SafeOpt) (2015)Google Scholar
  38. Sui, Y., Zhuang, V., Burdick, J.W., Yue, Y.: Stagewise safe Bayesian optimization with Gaussian processes (2018). arXiv preprint arXiv:1806.07555
  39. Tsai, Y.A., Pedrielli, G., Mathesen, L., Zabinsky, Z.B., Huang, H., Candelieri, A., Perego, R.: Stochastic optimization for feasibility determination: an application to water pump operation in water distribution network. In: Proceedings of the 2018 Winter Simulation Conference, pp. 1945–1956. IEEE Press (2018, December)Google Scholar
  40. Vapnik, V.: Statistical Learning Theory. Willey, New York (1998)zbMATHGoogle Scholar
  41. Wada, T., Hino, H.: Bayesian optimization for multi-objective optimization and multi-point search (2019). arXiv preprint arXiv:1905.02370
  42. Wu, J., Frazier, P.: The parallel knowledge gradient method for batch bayesian optimization. In: Advances in Neural Information Processing Systems, pp. 3126–3134 (2016)Google Scholar
  43. Zhang, S., Lyu, W., Yang, F., Yan, C., Zhou, D., Zeng, X., Hu, X.: An efficient multi-fidelity Bayesian optimization approach for analog circuit synthesis. In: DAC, pp. 64–1 (2019, June)Google Scholar

Copyright information

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer Science, Systems and CommunicationsUniversity of Milano-BicoccaMilanItaly

Personalised recommendations