Discrete-Time Markov Models

  • V.G. Kulkarni
Part of the Springer Text in Statistics book series (STS)


Consider a system that evolves randomly in time. Suppose this system is observed at times n = 0, 1, 2, 3,....Let X n be the (random) state of the system at time n. The sequence of random variables {X 0, X 1, X 2,...} is called a (discrete-time) stochastic process and is written as {X n , n ≥ 0}. Let S be the set of values that XI, can take for any n.


State Space Stationary Distribution Computational Problem Transition Probability Matrix Transition Diagram 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • V.G. Kulkarni
    • 1
  1. 1.Department of Operations ResearchUniversity of North CarolinaChapel HillUSA

Personalised recommendations