Approximate Policy Optimization and Adaptive Control in Regression Models
- 122 Downloads
In this paper we use recent advances in approximate dynamic programming to develop an approximate policy optimization procedure that uses Monte Carlo simulations for numerical solution of dynamic optimization problems in economics. The procedure is applied to the classical problem of “learning by doing” in regression models, for which the value and extent of active experimentation are demonstrated in a variety of numerical studies.
Keywordsdynamic programming policy iteration rollout Monte Carlo learning by doing
Unable to display preview. Download preview PDF.
- Bellman, R. (1957). Dynamic Programming. Princeton, NJ:Princeton University Press.Google Scholar
- Bertsekas, D.P. (2000). Dynamic Programming and Optimal Control, 2nd edition. Belmont, MA: Athena Scientific.Google Scholar
- Bertsekas, D.P. and Tsitsiklis, J.N. (1996). Neuro-DynamicProgramming. Belmont, MA: Athena Scientific.Google Scholar
- Blume, L. and Easley, D. (1984). Rational expectationsequilibrium: An alternative approach. Journal of EconomicTheory 34, 116–129.Google Scholar
- Box, G.E.P. and Tiao, G.C. (1973). Bayesian Inference in Statistical Analysis. New York: Wiley.Google Scholar
- Kendrick, D. (1981). Stochastic Control for EconomicModels. New York: McGraw-Hill.Google Scholar
- Lai, T.L. and Robbins, H. (1979). Adaptive design and stochasticapproximation. Annals of Statistics 7, 1196–1221.Google Scholar
- Stokey, N. and Lucas, R.E. (1989). Recursive Methods in Economic Dynamics. Cambridge, MA: HarvardUniversity Press.Google Scholar
- Tesauro, G. and Galperin, G. (1996). On-line policy improvement using Monte-Carlo search. In Advances inNeural Information Processing Systems 9, 1068–1074. Cambridge, MA: MIT Press.Google Scholar
- Wieland, V. (1995). Optimal control with unknown parameters – a study of optimal learning strategies with anapplication to monetary policy. Ph.D. Thesis, Stanford University.Google Scholar
- Yan, X., Diaconis, P., Rusmevichientong, P. and Van Roy, B. (2005). Solitaire: Man versus machine. In Advances in Neural Information Processing Systems 17, in press. Cambridge, MA: MIT PressGoogle Scholar