Continuous-Time Markov Chains
- 2.8k Downloads
A continuous-time Markov chain (CTMC) is a discrete-time Markov chain with the modification that, instead of spending one time unit in a state, it remains in a state for an exponentially distributed time whose rate depends on the state. The methodology of CTMCs is based on properties of renewal and Poisson processes as well as discrete-time chains. CTMCs are natural candidates for modeling systems in real time such as production and inventory systems, computer and telecommunications networks, and miscellaneous input-output systems. Many continuous-time processes have discretetime analogues; for instance, birth-death and Brownian motion processes are continuous-time analogues of discrete-time random walks. One’s choice of a continuous- or discrete-time model for a system typically depends on how realistic it is, its ease in addressing the issues at hand, or in computing quantities of interest.
KeywordsMarkov Chain Stationary Distribution Invariant Measure Transition Rate Poisson Process
Unable to display preview. Download preview PDF.