Markov Processes \(\star \)
Markov processes are introduced as memoryless stochastic processes and classified in four classes based on whether the time parameter is continuous or discrete and whether the sample space is continuous or discrete. Two of them are treated in more detail: discrete-time (“classical”) Markov chains and continuous-time, continuous-state Markov processes. Long-time behavior of the chains is discussed, establishing the conditions for the formation of equilibrium distributions. In the continuous case, the Markov propagator is defined along with a discussion of moment functions, characterizing functions, and time evolution of the moments. Two particular Markov processes, the Wiener and the Ornstein–Uhlenbeck process, are given special attention due to their relevance for the study of diffusion.
KeywordsMarkov Process Equilibrium Distribution Wiener Process Moment Function Planck Equation
- 1.E. Parzer, Stochastic Processes (Holden-Day, San Francisco, 1962)Google Scholar
- 7.B. Hayes, First links in the Markov chain. Am. Sci. 101, 92 (2013)Google Scholar
- 10.P. Langevin, Sur la théorie du mouvement brownien, C. R. Acad. Sci. (Paris) 146, 530 (1908). See also the English translation D.S. Lemons, A. Gythiel, Paul Langevin’s 1908 paper “On the theory of Brownian motion”. Am. J. Phys. 65, 1079 (1997)Google Scholar