Markov Processes \(\star \)

  • Simon ŠircaEmail author
Part of the Graduate Texts in Physics book series (GTP)


Markov processes are introduced as memoryless stochastic processes and classified in four classes based on whether the time parameter is continuous or discrete and whether the sample space is continuous or discrete. Two of them are treated in more detail: discrete-time (“classical”) Markov chains and continuous-time, continuous-state Markov processes. Long-time behavior of the chains is discussed, establishing the conditions for the formation of equilibrium distributions. In the continuous case, the Markov propagator is defined along with a discussion of moment functions, characterizing functions, and time evolution of the moments. Two particular Markov processes, the Wiener and the Ornstein–Uhlenbeck process, are given special attention due to their relevance for the study of diffusion.


Markov Process Equilibrium Distribution Wiener Process Moment Function Planck Equation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    E. Parzer, Stochastic Processes (Holden-Day, San Francisco, 1962)Google Scholar
  2. 2.
    A.A. Markov, An example of statistical investigation of the text “Eugene Onegin” concerning the connection of samples in chains. Sci. Context 19, 591 (2006). English translation from original RussianCrossRefzbMATHGoogle Scholar
  3. 3.
    S. Meyn, R.L. Tweedie, Markov Chains and Stochastic Stability (Springer, Berlin, 1995)zbMATHGoogle Scholar
  4. 4.
    D.W. Stroock, An Introduction to Markov Processes, 2nd edn. (Springer, Berlin, 2014)CrossRefzbMATHGoogle Scholar
  5. 5.
    A. Papoulis, Probability, Random Variables and Stochastic Processes, 3rd edn. (McGraw-Hill, New York, 1991)zbMATHGoogle Scholar
  6. 6.
    C. Meyer, Matrix Analysis and Applied Linear Algebra (SIAM, Philadelphia, 2000)CrossRefGoogle Scholar
  7. 7.
    B. Hayes, First links in the Markov chain. Am. Sci. 101, 92 (2013)Google Scholar
  8. 8.
    D.T. Gillespie, Markov Processes. An Introduction for Physical Scientists (Academic Press Inc, Boston, 1992)zbMATHGoogle Scholar
  9. 9.
    A. Einstein, On the movement of small particles suspended in stationary liquids required by the molecular-kinetic theory of heat. Ann. Phys. 17, 549 (1905)CrossRefGoogle Scholar
  10. 10.
    P. Langevin, Sur la théorie du mouvement brownien, C. R. Acad. Sci. (Paris) 146, 530 (1908). See also the English translation D.S. Lemons, A. Gythiel, Paul Langevin’s 1908 paper “On the theory of Brownian motion”. Am. J. Phys. 65, 1079 (1997)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Faculty of Mathematics and PhysicsUniversity of LjubljanaLjubljanaSlovenia

Personalised recommendations