Bayesian Best-Arm Identification for Selecting Influenza Mitigation Strategies
Pandemic influenza has the epidemic potential to kill millions of people. While various preventive measures exist (i.a., vaccination and school closures), deciding on strategies that lead to their most effective and efficient use remains challenging. To this end, individual-based epidemiological models are essential to assist decision makers in determining the best strategy to curb epidemic spread. However, individual-based models are computationally intensive and it is therefore pivotal to identify the optimal strategy using a minimal amount of model evaluations. Additionally, as epidemiological modeling experiments need to be planned, a computational budget needs to be specified a priori. Consequently, we present a new sampling technique to optimize the evaluation of preventive strategies using fixed budget best-arm identification algorithms. We use epidemiological modeling theory to derive knowledge about the reward distribution which we exploit using Bayesian best-arm identification algorithms (i.e., Top-two Thompson sampling and BayesGap). We evaluate these algorithms in a realistic experimental setting and demonstrate that it is possible to identify the optimal strategy using only a limited number of model evaluations, i.e., 2-to-3 times faster compared to the uniform sampling method, the predominant technique used for epidemiological decision making in the literature. Finally, we contribute and evaluate a statistic for Top-two Thompson sampling to inform the decision makers about the confidence of an arm recommendation. Code related to this paper is available at: https://plibin-vub.github.io/epidemic-bandits.
KeywordsPandemic influenza Multi-armed bandits Fixed budget best-arm identification Preventive strategies Individual-based models
Pieter Libin and Timothy Verstraeten were supported by a PhD grant of the FWO (Fonds Wetenschappelijk Onderzoek - Vlaanderen). Kristof Theys, Jelena Grujic and Diederik Roijers were supported by a postdoctoral grant of the FWO. The computational resources were provided by an EWI-FWO grant (Theys, KAN2012 220.127.116.11.). We thank the anonymous reviewers for their insightful comments that allowed us to improve this work.
- 1.Audibert, J.Y., Bubeck, S.: Best arm identification in multi-armed bandits. In: COLT-23th Conference on Learning Theory (2010)Google Scholar
- 8.Chapelle, O., Li, L.: An empirical evaluation of Thompson sampling. In: Advances in Neural Information Processing Systems, pp. 2249–2257 (2011)Google Scholar
- 16.Garivier, A., Kaufmann, E.: Optimal best arm identification with fixed confidence. In: Conference on Learning Theory, pp. 998–1027 (2016)Google Scholar
- 21.Hoffman, M., Shahriari, B., Freitas, N.: On correlation and budget constraints in model-based bandit optimization with application to automatic machine learning. In: Artificial Intelligence and Statistics, pp. 365–374 (2014)Google Scholar
- 22.Honda, J., Takemura, A.: Optimality of Thompson sampling for Gaussian bandits depends on priors. In: AISTATS, pp. 375–383 (2014)Google Scholar
- 25.Kaufmann, E., Kalyanakrishnan, S.: Information complexity in bandit subset selection. In: Conference on Learning Theory, pp. 228–251 (2013)Google Scholar
- 26.Libin, P., Verstraeten, T., Theys, K., Roijers, D.M., Vrancx, P., Nowé, A.: Efficient evaluation of influenza mitigation strategies using preventive bandits. In: Sukthankar, G., Rodriguez-Aguilar, J.A. (eds.) AAMAS 2017. LNCS (LNAI), vol. 10643, pp. 67–85. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71679-4_5CrossRefGoogle Scholar
- 31.Russo, D.: Simple Bayesian algorithms for best arm identification. In: Conference on Learning Theory, pp. 1417–1418 (2016)Google Scholar
- 33.WHO: WHO guidelines on the use of vaccines and antivirals during influenza pandemics (2004)Google Scholar