Abstract
This work is concerned with rates of convergence of numerical methods using Markov chain approximation for controlled diffusions with stopping (the first exit time from a bounded region). In lieu of considering the associated finite difference schemes for Hamilton-Jacobi-Bellman (HJB) equations, a purely probabilistic approach is used. There is an added difficulty due to the boundary condition, which requires the continuity of the first exit time with respect to the discrete parameter. To prove the convergence of the algorithm by Markov chain approximation method, a tangency problem might arise. A common approach uses certain conditions to avoid the tangency problem. Here, by modifying the value function, it is demonstrated that the tangency problem will not arise in the sense of convergence in probability and in L 1. In addition, controlled diffusions with a discount factor is also treated.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H. J. Kushner, Numiercal methods for stochastic control problems in continuous time, SIAM J. Control Optim., 1990, 28: 999–1048.
H. J. Kushner and P. Dupuis, Numerical Methods for Stochastic Control Problems in Continuous Time, 2nd Edition, Springer, New York, 2001.
G. Barles and E. R. Jakobsen, On the rate for approximation shemes for Hamilton-Jacobi-Bellman equations, M2AN Math. Model. Numer. Anal., 2002, 36: 33–54.
H. Dong and N. V. Krylov, On the rate of convergence of finite-difference approximations for parabolic Bellman equations with Lipschitz coefficients in cylindrical domains, Appl. Math. Optim., 2007, 56: 37–66.
E. R. Jakobsen, On the rate of convergence of approximation schemes for Bellman equations associated with optimal stopping time problems, Math. Methods Appl. Sci., 2003, 13: 613–644.
N. V. Krylov, On the rate of convergence of finite-difference approximations for Bellman’s equations with variable coefficients. Probab. Theory Related Fields, 2000, 117: 1–16.
J. Menaldi, Some estimates for finite difference approximations, SIAM J. Control Optim., 1989, 27: 579–607.
J. Zhang, Rate of convergence of finite difference approximations for degenerate ODEs, Math. Computation, 2006, 75: 1755–1778.
Q. S. Song, G. Yin, and Z. Zhang, Numerical method for controlled regime-switching diffusions and regime-switching jump diffusions, Automatica, 2006, 42: 1147–1157.
Q. S. Song and G. Yin, Rates of convergence of numerical methods for controlled regime-switching diffusions with stopping times in the costs, SIAM J. Control Optim., 2009, 48: 1831–1857.
H. J. Kushner, Consistency issues for numerical methods for variance control with applications to optimization in finance, IEEE Trans. Automatic Control, 2000, 44: 2283–2296.
W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, New York, NY, 1975.
P. Hall, C. C. Heyde, Martingale Limit Theory and Its Application, Academic Press, 1980.
V. Strassen, Almost sure behavior of sums of independent random variables and martingales, Proc. 5th Berkeley Symp. on Math. Statist. Probab., Vol. II(Part 1), Berkeley Univ. of Calif. Press, 1967, 315–343.
I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag, New York, 2004.
N. V. Krylov, Controlled Diffusion Processes, Springer-Verlag, 1980.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported in part by the National Science Foundation under Grant Nos. DMS-0624849 and DMS-0907753, and in part by the Natural Science Foundation of China under Grant No. #70871055.
Rights and permissions
About this article
Cite this article
Song, Q., Yin, G.G. Convergence rates of Markov chain approximation methods for controlled diffusions with stopping. J Syst Sci Complex 23, 600–621 (2010). https://doi.org/10.1007/s11424-010-0148-5
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-010-0148-5