Are Humans Bayesian in the Optimization of Black-Box Functions?
- 35 Downloads
Many real-world problems have complicated objective functions whose optimization requires sophisticated sequential decision-making strategies. Modelling human function learning has been the subject of intense research in cognitive sciences. The topic is relevant in black-box optimization where information about the objective and/or constraints is not available and must be learned through function evaluations. The Gaussian Process based Bayesian learning paradigm is central in the development of active learning approaches balancing exploration/exploitation in uncertain conditions towards effective generalization in large decision spaces. In this paper we focus on Bayesian Optimization and analyse experimentally how it compares to humans while searching for the maximum of an unknown 2D function. A set of controlled experiments with 53 subjects confirm that Gaussian Processes provide a general model to explain different patterns of learning enabled search and optimization in humans.
KeywordsBayesian Optimization Cognitive models Search strategy
- Borji, A., Itti, L.: Bayesian optimization explains human active search. In: Advances in Neural Information Processing System 26 (NIPS 2013), pp. 55–63 (2013)Google Scholar
- Chapelle, O., Li, L.: An empirical evaluation of thompson sampling. In: Advances in Neural Information Processing Systems, pp. 2249–2257 (2011)Google Scholar
- Gershman, S.J.: Uncertainty and exploration. bioRxiv 265504 (2018). https://doi.org/10.1101/265504
- Li, K., Malik, J.: Learning to optimize. (2016) arXiv preprint arXiv:1606.01885
- Gershman, S.J.: Quantifying mismatch in Bayesian optimization. In: NIPS Workshop on Bayesian Optimization: Black-Box Optimization and Beyond (2016)Google Scholar
- Schulz, E., Tenenbaum, J., Duvenaud, D.K., Speekenbrink, M., Gershman, S.J.: Probing the compositionality of intuitive functions. In: Advances in Neural Information Processing Systems, pp. 3729–3737 (2016)Google Scholar
- Srinivas, N., Krause, A., Kakade, S., Seeger, M.: Gaussian process optimization in the bandit setting: no regret and experimental design. In: Proceedings of the 27th International Conference on Machine Learning, pp. 1015–1022. Omnipress, June 2010Google Scholar