Discrete-Time Markov Models

  • Pierre Brémaud
Part of the Texts in Applied Mathematics book series (TAM, volume 31)


Sequences of independent and identically distributed random variables are stochastic processes, but they are not always interesting as stochastic models because they behave more or less in the same way. In order to introduce more variability, one can allow for some dependence on the past, in the manner of deterministic recurrence equations. Discrete-time homogeneous Markov chains possess the required feature, since they can always be represented—at least distributionwise—by a stochastic recurrence equation X n+1 = f(X n , Z n+1),where {Z n } n≥1 is an i.i.d sequence, independent of the initial state X 0.


Stationary Distribution Transition Matrix Recurrence Equation Markov Property Transition Graph 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • Pierre Brémaud
    • 1
  1. 1.Laboratoire des Signaux et SystèmesCNRS-ESEGif-sur-YvetteFrance

Personalised recommendations