Markov decision drift processes

  • Frank A. Van der Duyn Schouten


In Markov decision theory we distinguish
  1. (a)

    discrete-time Markov decision processes

  2. (b)

    semi-Markov decision processes

  3. (c)

    continuous time Markov decision processes.



Optimal Policy Sample Path Markov Decision Process Impulsive Control Compound Poisson Process 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Constantinides, G.M., (1976). Stochastic cash management with fixed and proportional transaction costs. Management Sci. 22, 1320–1331.MathSciNetzbMATHCrossRefGoogle Scholar
  2. Constantinides, G.M. and S.F. Richard (1978). Existence of optimal simple policies for discounted-cost inventory and cash management in continuous time. Operations Res. 26, 620–636.MathSciNetzbMATHCrossRefGoogle Scholar
  3. Davis, M.H.A., (1984). Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models. J.R. Statist.Soc. B. 46, 353–388.zbMATHGoogle Scholar
  4. Hordijk, A. and F.A. Van der Duyn Schouten (1984). Discretization and weak convergence in Markov decision drift processes. Math. Oper.Res. 9, 112–141.MathSciNetzbMATHCrossRefGoogle Scholar
  5. Hordijk, A. and F.A. Van der Duyn Schouten (1985). Markov decision drift processes; conditions for optimality obtained by discretization. Math. Oper. Res. 10, 160–173.MathSciNetzbMATHCrossRefGoogle Scholar
  6. Hordijk, A. and F.A. Van der Duyn Schouten (1983). Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model. Advances in Appl.Probability 15, 274–303.zbMATHCrossRefGoogle Scholar
  7. De Kok, A.G., H.C. Tijms and F.A. Van der Duyn Schouten (1984). Approximations for the single product production-inventory problem with compound Poisson demand and service-level constraints. Advances in Appl. Probability 16, 378–401.zbMATHCrossRefGoogle Scholar
  8. Lam, Y. and L.C. Thomas (1983). Adaptive control of M/M/1 queues-continuous time Markov decision process approach. J. Appl. Probability 20, 368–379.MathSciNetzbMATHCrossRefGoogle Scholar
  9. Luss, H., (1976). Maintenance policies when deterioration can be observed by inspections. Operations Res. 24, 359–366.MathSciNetzbMATHCrossRefGoogle Scholar
  10. Rosenfield, D., (1976). Markovian deterioration with uncertain information. Operations Res. 24, 141–155.MathSciNetzbMATHCrossRefGoogle Scholar
  11. Tijms, H.C. and F.A. Van der Duyn Schouten (1985). A Markov decision algorithm for optimal inspections and revisions in a maintenance system with partial information. European J. of Oper. Res. 21, 245–253.zbMATHCrossRefGoogle Scholar
  12. Van der Duyn Schouten, F.A., (1983). Markov decision processes with continuous time parameter. Math. Centre Tract 164. Mathematical Centre, Amsterdam.Google Scholar
  13. Vermes, D., (1985). Optimal control of piecewise deterministic processes. Stochastics 14, 165–208.MathSciNetzbMATHCrossRefGoogle Scholar
  14. Yushkevich, A.A., (1983). Continuous time Markov decision processes with interventions. Stochastics 9, 235–274.MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1986

Authors and Affiliations

  • Frank A. Van der Duyn Schouten
    • 1
  1. 1.Department of Actuarial Sciences and EconometricsAmsterdamThe Netherlands

Personalised recommendations