Computational Economics

, Volume 31, Issue 2, pp 141–160 | Cite as

Continuous State Dynamic Programming via Nonexpansive Approximation

  • John Stachurski


This paper studies fitted value iteration for continuous state numerical dynamic programming using nonexpansive function approximators. A number of approximation schemes are discussed. The main contribution is to provide error bounds for approximate optimal policies generated by the value iteration algorithm.


Numerical dynamic programming Nonexpansive approximation 

JEL Classification

C61 C63 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aruoba S.B., Fernandez-Villaverde J., Rubio-Ramirez J.F. (2006) Comparing solution methods for dynamic equilibrium economies. Journal of Economic Dynamics and Control 30(12): 2477–2508CrossRefGoogle Scholar
  2. Chambers M.J., Bailey R.J. (1996) A theory of commodity price fluctuations. Journal of Political Economy 104(5): 924–957CrossRefGoogle Scholar
  3. Deaton A., Laroque G. (1996) Comptetitive storage and commodity price dynamics. Journal of Political Economy 104(5): 896–923CrossRefGoogle Scholar
  4. Drummond, C. (1996). Preventing overshoot of splines with application to reinforcement learning. Computer Science Dept. Ottawa TR-96-05.Google Scholar
  5. Gordon, G. J. (1995). Stable function approximation in dynamic programming. Proceedings of the 12th International Conference on Machine Learning.Google Scholar
  6. Guestrin, C., Koller, D., & Parr, R. (2001). Max-norm projections for factored MDPs. International Joint Conference on Artificial Intelligence (Vol. 1, pp. 673–680).Google Scholar
  7. Hernández-Lerma O., LasserreJ.B. (1999) Further topics in discrete-time Markov control processes. New York: Springer-VerlagGoogle Scholar
  8. Judd K.L. (1992). Projection methods for solving aggregate growth models. Journal of Economic Theory 58(2): 410–452CrossRefGoogle Scholar
  9. Judd, K. L., & Solnick, A. (1994). Numerical dynamic programming with shape-preserving splines. Unpublished manuscript.Google Scholar
  10. Lyche, T., & Mørken, K. (2002). Spline methods. Mimeo, University of Oslo.Google Scholar
  11. Maldonado W.L., Svaiter B.F. (2001) On the accuracy of the estimated policy function using the Bellman contraction method. Economics Bulletin 15(3): 1–8Google Scholar
  12. Munos R., Moore A. (1999) Variable resolution discretization in optimal control. Machine Learning 1: 1–24Google Scholar
  13. Puterman M. (1994) Markov decision processes: Discrete stochastic dynamic programming. John Wiley & Sons, New YorkGoogle Scholar
  14. Rust J. (1996) Numerical dynamic programming in economics. In: Amman H., Kendrick D., Rust J. (eds). Handbook of computational economics. Elsevier, North HollandGoogle Scholar
  15. Rust J. (1997) Using randomization to break the curse of dimensionality. Econometrica 65(3): 487–516CrossRefGoogle Scholar
  16. Samuelson P.A. (1971) Stochastic speculative price. Proceedings of the National Academy of Science 68(2): 335–337CrossRefGoogle Scholar
  17. Santos M.S., Vigo-Aguiar J. (1998) Analysis of a numerical dynamic programming algorithm applied to economic models. Econometrica 66(2): 409–426CrossRefGoogle Scholar
  18. Santos M.S. (2000) Accuracy of numerical solutions using the Euler equation residuals. Econometrica 68(6): 1377–1402CrossRefGoogle Scholar
  19. Stokey N.L., Lucas R.E., Prescott E.C. (1989) Recursive methods in economic dynamics. Harvard University Press, MassachusettsGoogle Scholar
  20. Tsitsiklis J.N., Van Roy B. (1996) Feature-based methods for large scale dynamic programming. Machine Learning 22: 59–94Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Institute of Economic ResearchKyoto UniversityKyotoJapan

Personalised recommendations