Abstract
This paper studies fitted value iteration for continuous state numerical dynamic programming using nonexpansive function approximators. A number of approximation schemes are discussed. The main contribution is to provide error bounds for approximate optimal policies generated by the value iteration algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aruoba S.B., Fernandez-Villaverde J., Rubio-Ramirez J.F. (2006) Comparing solution methods for dynamic equilibrium economies. Journal of Economic Dynamics and Control 30(12): 2477–2508
Chambers M.J., Bailey R.J. (1996) A theory of commodity price fluctuations. Journal of Political Economy 104(5): 924–957
Deaton A., Laroque G. (1996) Comptetitive storage and commodity price dynamics. Journal of Political Economy 104(5): 896–923
Drummond, C. (1996). Preventing overshoot of splines with application to reinforcement learning. Computer Science Dept. Ottawa TR-96-05.
Gordon, G. J. (1995). Stable function approximation in dynamic programming. Proceedings of the 12th International Conference on Machine Learning.
Guestrin, C., Koller, D., & Parr, R. (2001). Max-norm projections for factored MDPs. International Joint Conference on Artificial Intelligence (Vol. 1, pp. 673–680).
Hernández-Lerma O., LasserreJ.B. (1999) Further topics in discrete-time Markov control processes. New York: Springer-Verlag
Judd K.L. (1992). Projection methods for solving aggregate growth models. Journal of Economic Theory 58(2): 410–452
Judd, K. L., & Solnick, A. (1994). Numerical dynamic programming with shape-preserving splines. Unpublished manuscript.
Lyche, T., & Mørken, K. (2002). Spline methods. Mimeo, University of Oslo.
Maldonado W.L., Svaiter B.F. (2001) On the accuracy of the estimated policy function using the Bellman contraction method. Economics Bulletin 15(3): 1–8
Munos R., Moore A. (1999) Variable resolution discretization in optimal control. Machine Learning 1: 1–24
Puterman M. (1994) Markov decision processes: Discrete stochastic dynamic programming. John Wiley & Sons, New York
Rust J. (1996) Numerical dynamic programming in economics. In: Amman H., Kendrick D., Rust J. (eds). Handbook of computational economics. Elsevier, North Holland
Rust J. (1997) Using randomization to break the curse of dimensionality. Econometrica 65(3): 487–516
Samuelson P.A. (1971) Stochastic speculative price. Proceedings of the National Academy of Science 68(2): 335–337
Santos M.S., Vigo-Aguiar J. (1998) Analysis of a numerical dynamic programming algorithm applied to economic models. Econometrica 66(2): 409–426
Santos M.S. (2000) Accuracy of numerical solutions using the Euler equation residuals. Econometrica 68(6): 1377–1402
Stokey N.L., Lucas R.E., Prescott E.C. (1989) Recursive methods in economic dynamics. Harvard University Press, Massachusetts
Tsitsiklis J.N., Van Roy B. (1996) Feature-based methods for large scale dynamic programming. Machine Learning 22: 59–94
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Stachurski, J. Continuous State Dynamic Programming via Nonexpansive Approximation. Comput Econ 31, 141–160 (2008). https://doi.org/10.1007/s10614-007-9111-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10614-007-9111-5