Semi-Markov Decision Processes

Part of the Advances in Mechanics and Mathematics book series (AMMA, volume 14)

The underlying stochastic processes in DTMDPs are discrete time Markov chains, where the decision epochs are equally periodic or the length of adjacent decision epochs are not considered. Those in CTMDPs are continuous time Markov chains, where the decision is chosen every time. In this chapter, we study a stationary semi-Markov decision processes (SMDPs) model, where the underlying stochastic processes are semi-Markov processes. Here, the decision epoch is exactly the state transition epoch with its length being random. We transform the SMDP model into a stationary DTMDP model for either the total reward criterion or the average criterion, similarly to the stationary CTMDP model with the average criterion discussed in Section 4.3. Thus, the results in DTMDP can be used directly for SMDP for the discounted criterion, the total reward criterion, and the average criterion.


MARKOV Decision Process Reward Function Optimality Equation Average Criterion Discrete Time Markov Chain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media, LLC 2008

Personalised recommendations