Are Humans Bayesian in the Optimization of Black-Box Functions?

  • Antonio CandelieriEmail author
  • Riccardo Perego
  • Ilaria Giordani
  • Francesco Archetti
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11974)


Many real-world problems have complicated objective functions whose optimization requires sophisticated sequential decision-making strategies. Modelling human function learning has been the subject of intense research in cognitive sciences. The topic is relevant in black-box optimization where information about the objective and/or constraints is not available and must be learned through function evaluations. The Gaussian Process based Bayesian learning paradigm is central in the development of active learning approaches balancing exploration/exploitation in uncertain conditions towards effective generalization in large decision spaces. In this paper we focus on Bayesian Optimization and analyse experimentally how it compares to humans while searching for the maximum of an unknown 2D function. A set of controlled experiments with 53 subjects confirm that Gaussian Processes provide a general model to explain different patterns of learning enabled search and optimization in humans.


Bayesian Optimization Cognitive models Search strategy 


  1. Adam, S.P., Alexandropoulos, S.A.N., Pardalos, P.M., Vrahatis, M.N.: No free lunch theorem: a review. In: Demetriou, I.C., Pardalos, P.M. (eds.) Approximation and Optimization. SOIA, vol. 145, pp. 57–82. Springer, Cham (2019). Scholar
  2. Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)CrossRefGoogle Scholar
  3. Borji, A., Itti, L.: Bayesian optimization explains human active search. In: Advances in Neural Information Processing System 26 (NIPS 2013), pp. 55–63 (2013)Google Scholar
  4. Candelieri, A., Perego, R., Archetti, F.: Bayesian optimization of pump operations in water distribution systems. J. Glob. Optim. 71, 213–235 (2018)MathSciNetCrossRefGoogle Scholar
  5. Chapelle, O., Li, L.: An empirical evaluation of thompson sampling. In: Advances in Neural Information Processing Systems, pp. 2249–2257 (2011)Google Scholar
  6. Eggensperger, K., Lindauer, M., Hutter, F.: Pitfalls and best practices in algorithm configuration. J. Artif. Intell. Res. 64, 861–893 (2019)MathSciNetCrossRefGoogle Scholar
  7. Gershman, S.J.: Uncertainty and exploration. bioRxiv 265504 (2018).
  8. Gopnik, A., O’Grady, S., Lucas, C.G., Griffiths, T.L., Wente, A., Bridgers, S., Dahl, R.E.: Changes in cognitive flexibility and hypothesis search across human life history from childhood to adolescence to adulthood. Proc. Nat. Acad. Sci. 114(30), 7892–7899 (2017)CrossRefGoogle Scholar
  9. Kruschke, J.K.: Bayesian approaches to associative learning: from passive to active learning. Learn. Behav. 36(3), 210–226 (2008)CrossRefGoogle Scholar
  10. Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998)MathSciNetCrossRefGoogle Scholar
  11. Li, K., Malik, J.: Learning to optimize. (2016) arXiv preprint arXiv:1606.01885
  12. May, B.C., Korda, N., Lee, A., Leslie, D.S.: Optimistic Bayesian sampling in contextual-bandit problems. J. Mach. Learn. Res. 13(Jun), 2069–2106 (2012)MathSciNetzbMATHGoogle Scholar
  13. Mehlhorn, K., Newell, B.R., Todd, P.M., Lee, M.D., Morgan, K., Braithwaite, V.A., Gonzalez, C.: Unpacking the exploration–exploitation tradeoff: a synthesis of human and animal literatures. Decision 2(3), 191 (2015)CrossRefGoogle Scholar
  14. Gershman, S.J.: Quantifying mismatch in Bayesian optimization. In: NIPS Workshop on Bayesian Optimization: Black-Box Optimization and Beyond (2016)Google Scholar
  15. Schulz, E., Tenenbaum, J., Duvenaud, D.K., Speekenbrink, M., Gershman, S.J.: Probing the compositionality of intuitive functions. In: Advances in Neural Information Processing Systems, pp. 3729–3737 (2016)Google Scholar
  16. Schulz, E., Speekenbrink, M., Krause, A.: A tutorial on Gaussian process regression: modelling, exploring, and exploiting functions. J. Math. Psychol. 85, 1–16 (2018)MathSciNetCrossRefGoogle Scholar
  17. Srinivas, N., Krause, A., Kakade, S., Seeger, M.: Gaussian process optimization in the bandit setting: no regret and experimental design. In: Proceedings of the 27th International Conference on Machine Learning, pp. 1015–1022. Omnipress, June 2010Google Scholar
  18. Thompson, W.R.: On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3/4), 285–294 (1933)CrossRefGoogle Scholar
  19. Wilson, R.C., Geana, A., White, J.M., Ludvig, E.A., Cohen, J.D.: Humans use directed and random exploration to solve the explore–exploit dilemma. J. Exp. Psychol. Gen. 143(6), 2074 (2014)CrossRefGoogle Scholar
  20. Wu, C.M., Schulz, E., Speekenbrink, M., Nelson, J.D., Meder, B.: Generalization guides human exploration in vast decision spaces. Nat. Hum. Behav. 2(12), 915 (2018)CrossRefGoogle Scholar
  21. Zhigljavsky, A., Zilinskas, A.: Stochastic Global Optimization, vol. 9. Springer, Berlin (2007). Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.University of Milano-BicoccaMilanItaly

Personalised recommendations