Markov decision drift processes

  • Frank A. Van der Duyn Schouten


In Markov decision theory we distinguish
  1. (a)

    discrete-time Markov decision processes

  2. (b)

    semi-Markov decision processes

  3. (c)

    continuous time Markov decision processes.



Optimal Policy Sample Path Markov Decision Process Impulsive Control Compound Poisson Process 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Constantinides, G.M., (1976). Stochastic cash management with fixed and proportional transaction costs. Management Sci. 22, 1320–1331.MathSciNetMATHCrossRefGoogle Scholar
  2. Constantinides, G.M. and S.F. Richard (1978). Existence of optimal simple policies for discounted-cost inventory and cash management in continuous time. Operations Res. 26, 620–636.MathSciNetMATHCrossRefGoogle Scholar
  3. Davis, M.H.A., (1984). Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models. J.R. Statist.Soc. B. 46, 353–388.MATHGoogle Scholar
  4. Hordijk, A. and F.A. Van der Duyn Schouten (1984). Discretization and weak convergence in Markov decision drift processes. Math. Oper.Res. 9, 112–141.MathSciNetMATHCrossRefGoogle Scholar
  5. Hordijk, A. and F.A. Van der Duyn Schouten (1985). Markov decision drift processes; conditions for optimality obtained by discretization. Math. Oper. Res. 10, 160–173.MathSciNetMATHCrossRefGoogle Scholar
  6. Hordijk, A. and F.A. Van der Duyn Schouten (1983). Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model. Advances in Appl.Probability 15, 274–303.MATHCrossRefGoogle Scholar
  7. De Kok, A.G., H.C. Tijms and F.A. Van der Duyn Schouten (1984). Approximations for the single product production-inventory problem with compound Poisson demand and service-level constraints. Advances in Appl. Probability 16, 378–401.MATHCrossRefGoogle Scholar
  8. Lam, Y. and L.C. Thomas (1983). Adaptive control of M/M/1 queues-continuous time Markov decision process approach. J. Appl. Probability 20, 368–379.MathSciNetMATHCrossRefGoogle Scholar
  9. Luss, H., (1976). Maintenance policies when deterioration can be observed by inspections. Operations Res. 24, 359–366.MathSciNetMATHCrossRefGoogle Scholar
  10. Rosenfield, D., (1976). Markovian deterioration with uncertain information. Operations Res. 24, 141–155.MathSciNetMATHCrossRefGoogle Scholar
  11. Tijms, H.C. and F.A. Van der Duyn Schouten (1985). A Markov decision algorithm for optimal inspections and revisions in a maintenance system with partial information. European J. of Oper. Res. 21, 245–253.MATHCrossRefGoogle Scholar
  12. Van der Duyn Schouten, F.A., (1983). Markov decision processes with continuous time parameter. Math. Centre Tract 164. Mathematical Centre, Amsterdam.Google Scholar
  13. Vermes, D., (1985). Optimal control of piecewise deterministic processes. Stochastics 14, 165–208.MathSciNetMATHCrossRefGoogle Scholar
  14. Yushkevich, A.A., (1983). Continuous time Markov decision processes with interventions. Stochastics 9, 235–274.MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1986

Authors and Affiliations

  • Frank A. Van der Duyn Schouten
    • 1
  1. 1.Department of Actuarial Sciences and EconometricsAmsterdamThe Netherlands

Personalised recommendations