Skip to main content
Log in

An approximation scheme for stochastic controls in continuous time

  • Original Paper
  • Area 2
  • Published:
Japan Journal of Industrial and Applied Mathematics Aims and scope Submit manuscript

Abstract

We propose a simple time-discretization scheme for multi-dimensional stochastic optimal control problems in continuous time. It is based on a probabilistic representation for the convolution of the value function by a probability density function. We show the convergence results under mild conditions on coefficients of the problems by Barles–Souganidis viscosity solution method. Resulting numerical methods allow us to use uncontrolled Markov processes to estimate the conditional expectations in the dynamic programming procedure. Moreover, it can be implemented without the interpolation of the value function or the adjustment of the diffusion matrix.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Barles, G., Jakobsen, E.R.: On the convergence rate of approximation schemes for Hamilton–Jacobi–Bellman equations. Math. Model. Numer. Anal. 36, 33–54 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  2. Barles, G., Jakobsen, E.R.: Error bounds for monotone approximation schemes for parabolic Hamilton–Jacobi–Bellman equations. Math. Comp. 76, 1861–1893 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  3. Barles, G., Souganidis, P.E.: Convergence of approximation schemes for fully nonlinear second order equations. Asymptot. Anal. 4, 271–283 (1991)

    MATH  MathSciNet  Google Scholar 

  4. Bonnans, F., Ottenwaelter, E., Zidani, H.: A fast algorithm for the two dimensional HJB equation of stochastic control. Math. Model. Numer. Anal. 38, 723–735 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  5. Bonnans, F., Zidani, H.: Consistency of generalized finite difference schemes for the stochastic HJB equation. SIAM J. Numer. Anal. 41, 1008–1021 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  6. Bouchard, B., Touzi, N.: Discrete-time approximation and Monte Carlo simulation of backward stochastic differential equations. Stoch. Process. Appl. 111, 175–206 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  7. Camilli, F., Falcone, M.: An approximation scheme for the optimal control of diffusion processes. Math. Model. Numer. Anal. 29, 97–122 (1995)

    MATH  MathSciNet  Google Scholar 

  8. Carlini, E., Falcone, M., Ferretti, R.: An efficient algorithm for Hamilton–Jacobi equations in high dimension. Comput. Vis. Sci. 7, 15–29 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  9. Debrabant, K., Jakobsen, E.R.: Semi-Lagrangian schemes for linear and fully non-linear diffusion equations. Math. Comput. 82, 1433–1462 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  10. Fahim, A., Touzi, N., Warin, X.: A probabilistic numerical method for fully nonlinear parabolic PDEs. Ann. Appl. Probab. 21, 1322–1364 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  11. Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  12. Graf, S., Luschgy, H.: Foundation of Quantization for Probability Distributions. Springer, Berlin (2000)

    Book  Google Scholar 

  13. Györfi, L., Kohler, M., Krzyzak, A., Walk, H.: A distribution-free theory of nonparametric regression. Springer, New York (2002)

    Book  MATH  Google Scholar 

  14. Kaise, H., McEneaney, W.M.: Idempotent expansions for continuous-time stochastic control. In: Proceedings of 49th IEEE conference on decision and control (2010)

  15. Kohn, R.V., Serfaty, S.: A deterministic-control-based approach to fully nonlinear parabolic and elliptic equations. Commun. Pure Appl. Math. 63, 1298–1350 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  16. Krylov, N.V.: Controlled Diffusion Processes. Springer, New York (1980)

    Book  MATH  Google Scholar 

  17. Krylov, N.V.: Approximating value functions for controlled degenerate diffusion processes by using piece-wise constant policies. Electron. J. Probab. 4, 1–19 (1999)

    Article  MathSciNet  Google Scholar 

  18. Kushner, H.J., Dupuis, P.: Numerical Methods for Stochastic Control Problems in Continuous Time. Springer, New York (2001)

    Book  MATH  Google Scholar 

  19. Lemor, J.P., Gobet, E., Warin, X.: Rate of convergence of an empirical regression method for solving generalized backward stochastic differential equations. Bernoulli 12, 889–916 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  20. Longstaff, F.A., Schwartz, E.S.: Valuing american options by simulation: a simple least-squares approach. Rev. Financ. Stud. 14, 113–147 (2001)

    Article  Google Scholar 

  21. Pagès, G., Pham, H., Printems, J.: An optimal Markovian quantization algorithm for multidimensional stochastic control problems. Stoch. Dyn. 4, 501–545 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  22. Pagès, G., Printems, J.: Optimal quadratic quantization for numerics: the Gaussian case. Monte Carlo Methods Appl. 9, 135–168 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  23. Pham, H.: Continuous-Time Stochastic Control and Optimization With Financial Applications. Springer, Berlin (2009)

    Book  MATH  Google Scholar 

Download references

Acknowledgments

The author is thankful to Takashi Tamura for helpful comments, as well as the anonymous referee for his/her careful reading of a previous version of this paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yumiharu Nakano.

Additional information

This study is partially supported by JSPS KAKENHI Grant Number 26800079.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nakano, Y. An approximation scheme for stochastic controls in continuous time. Japan J. Indust. Appl. Math. 31, 681–696 (2014). https://doi.org/10.1007/s13160-014-0157-1

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13160-014-0157-1

Keywords

Mathematics Subject Classification

Navigation