Markov Processes

  • Randolph Nelson


In this chapter we consider an important type of stochastic process called the Markov process. A Markov process1 is a stochastic process that has a limited form of “historical” dependency. To precisely define this dependency, let {X(t) : tT} be a stochastic process defined on the parameter set T. We will think of T in terms of time, and the values that X(t) can assume are called states which are elements of a state space S.


Markov Chain Markov Process Stationary Distribution Sample Path Markov Property 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Bibliographic Notes

  1. [64]
    J.G. Kemeny and J.L. Snell. Finite Markov Chains. Springer-Verlag, 1976.Google Scholar
  2. [63]
    J.G. Kemeny, J. L. Snell, and A. W. Knapp. Denumerable Markov Chains. Springer-Verlag, 1976.Google Scholar
  3. [111]
    L. Takacs. Combinatorial Methods in the Theory of Stochastic Processes. Robert E. Krieger, 1977.zbMATHGoogle Scholar
  4. [127]
    E. Wong and B. Hajek. Stochastic Processes in Engineering Systems. Springer-Verlag, 1985.Google Scholar
  5. [58]
    S. Karlin and H. M. Taylor. A Second Course in Stochastic Processes. Academic Press, 1981.Google Scholar

Copyright information

© Springer Science+Business Media New York 1995

Authors and Affiliations

  • Randolph Nelson
    • 1
    • 2
  1. 1.OTA Limited PartnershipPurchaseUSA
  2. 2.Modeling MethodologyIBM T.J. Watson Research CenterYorktown HeightsUSA

Personalised recommendations