Dynamic sample budget allocation in model-based optimization
- 263 Downloads
Model-based search methods are a class of optimization techniques that search the solution space by sampling from an underlying probability distribution “model,” which is updated iteratively after evaluating the performance of the samples at each iteration. This paper aims to improve the sampling efficiency of model-based methods by considering a generalization where a population of distribution models is maintained and subsequently propagated from generation to generation. A key issue in the proposed approach is how to efficiently allocate the sampling budget among the population of models to maximize the algorithm performance. We formulate this problem as a generalized max k-armed bandit problem, and derive an efficient dynamic sample allocation scheme based on Markov decision theory to adaptively allocate computational resources. The proposed allocation scheme is then further used to update the current population to produce an improving population of models. Our preliminary numerical results indicate that the proposed procedure may considerably reduce the number of function evaluations needed to obtain high quality solutions, and thus further enhance the value of model-based methods for optimization problems that require expensive function evaluations for performance evaluation.
KeywordsMarkov decision processes Max k-armed bandit Global optimization
Unable to display preview. Download preview PDF.
- 1.Bertsekas D.P.: Dynamic Programming and Optimal Control, vol 1 and 2. Athena Scientific, Belmont (1995)Google Scholar
- 3.Cicirello, V., Smith, S.F.: The max k-armed bandit: a new model for exploration applied to search heuristic selection. In: Proceedings of the 20th National Conference on Artificial Intelligence (AAAI-05) (2005)Google Scholar
- 5.Fu, M.C., Hu, J., Marcus, S. I.: Model-based randomized methods for global optimization. In: Proceedings of the 17th international symposium on mathematical theory of networks and systems. Kyoto, Japan (2006)Google Scholar
- 9.Larrañaga P., Lozano J.A.: Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer, Boston (2002)Google Scholar
- 11.Pintér J.D.: Global Optimization in Action. Kluwer, The Netherlands (1996)Google Scholar
- 12.Ross S.: Stochastic Processes 2nd edn. John Wiley, New York (1995)Google Scholar
- 13.Royden H.L.: Real Analysis 3rd edn. Prentice-Hall, Englewood Cliffs (1988)Google Scholar
- 15.Rubinstein R.Y., Kroese D.P.: The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning. Springer, New York (2004)Google Scholar
- 17.Srinivas M., Patnaik L.M.: Genetic algorithms: a survey. IEEE Comp. 27, 17–26 (1994)Google Scholar
- 18.Streeter, M., Smith, S.F.: A simple distribution-free approach to the max k-armed bandit problem. In: Proceedings of the 12th international conference on principles and practice of constraint programming. Lecture Notes in Computer Science 4204, pp. 560–574. Springer, Berlin (2006)Google Scholar
- 21.Zhang, H., Fu, M.C.: Applying model reference adaptive search to American-style option pricing. In: Proceedings of the 28th winter simulation conference, pp. 711–718 (2006)Google Scholar