Continuous Time Markov Decision Processes

Part of the Advances in Mechanics and Mathematics book series (AMMA, volume 14)

This chapter discusses continuous time Markov decision processes, where the state space and the action sets are all countable. First, we focus on the total reward criterion for a stationary model by applying the ideas and methods presented in Chapter 2 for DTMDPs. Similar results to those in Chapter 2 are obtained. Then, we deal with a nonstationary model with the total reward criterion. By dividing the time axis into shorter intervals, we obtain the standard results, such as the optimality equation and the relationship between the optimality of a policy and the optimality equation. Finally, we study the average criterion for a stationary CTMDP model by transforming it into a DTMDP model. Thus, the results in DTMDPs can be used directly for CTMDPs for the average criterion.


Discount Rate Optimal Policy Markov Decision Process Optimality Equation Average Criterion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media, LLC 2008

Personalised recommendations