Ergodic Markov Processes and Poisson Equations (Lecture Notes)

  • Alexander VeretennikovEmail author
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 208)


These are lecture notes on the subject defined in the title. As such, they do not pretend to be really new, perhaps, except for the Sect. 10 about Poisson equations with potentials; also, the convergence rate shown in (83)–(84) is possibly less known. Yet, the hope of the author is that these notes may serve as a bridge to the important area of Poisson equations ‘in the whole space’ and with a parameter, the latter theme not being presented here. Why this area is so important was explained in many papers and books including (Ethier and Kurtz, Markov Processes: Characterization and Convergence, New Jersey, 2005) [12], (Papanicolaou et al. Statistical Mechanics, Dynamical Systems and the Duke Turbulence Conference, vol. 3. Durham, N.C., 1977) [34], (Pardoux and Veretennikov, Ann. Prob. 31(3), 1166–1192, 2003) [35]: it provides one of the main tools in diffusion approximation in the area stochastic averaging. Hence, the aim of these lectures is to prepare the reader to ‘real’ Poisson equations—i.e. for differential operators instead of difference operators—and, indeed, to diffusion approximation. Among other presented topics, we mention coupling method and convergence rates in the Ergodic theorem.


Markov chains Ergodic theorem Limit theorems Coupling Discrete poisson equations 


  1. 1.
    Bernstein, S.: Sur les sommes de quantités dépendantes, Izv. AN SSSR, ser. VI, 20, 15–17, 1459–1478 (1926)Google Scholar
  2. 2.
    Bernstein, S.N.: Extension of a limit theorem of probability theory to sums of dependent random variables. Russian Math. Surv. 10, 65–114 (1944). (in Russian)Google Scholar
  3. 3.
    Bezhaeva, Z.I., Oseledets, V.I.: On the variance of sums for functions of a stationary Markov process. Theory Prob. Appl. 41(3), 537–541 (1997)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Borovkov, A.A.: Ergodicity and Stability of Stochastic Processes. Wiley, Chichester (1998)zbMATHGoogle Scholar
  5. 5.
    Borovkov, A.A., Mogul’skii, A.A.: Large deviation principles for sums of random vectors and the corresponding renewal functions in the inhomogeneous case. Siberian Adv. Math. 25(4), 255–267 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Butkovsky, O.A., Veretennikov, A.Yu.: On asymptotics for Vaserstein coupling of Markov chains. Stoch. Process. Appl. 123(9), 3518–3541 (2013)Google Scholar
  7. 7.
    Dembo, A., Zeitouni, O.: Large Deviations Techniques and Applications, 2nd edn. Springer, Berlin (1998)Google Scholar
  8. 8.
    Dobrushin, R.L.: Central limit theorem for non-stationary Markov chains. I II Theory Prob. Appl. 1, 65–80, 329–383 (1956). [English translation]Google Scholar
  9. 9.
    Doeblin, W.: Exposé de la théorie des chaines simples constantes de Markov à un nombre fini d’états. Mathématique de l’Union Interbalkanique 2, 77–105 & 78–80 (1938)Google Scholar
  10. 10.
    Doob, J.L.: Stochastic Processes. Wiley, New Jersey (1953)Google Scholar
  11. 11.
    Dynkin, E.B.: Markov Processes, vol. 1. Springer, Berlin (2012). (paperback)Google Scholar
  12. 12.
    Ethier, S.N., Kurtz, T.G.: Markov Processes: Characterization and Convergence, Wiley Series in Probability and Statistics, New Jersey (2005). (paperback)Google Scholar
  13. 13.
    Feng, J., Kurtz, T.G.: Large Deviations for Stochastic Processes. AMS (2006)Google Scholar
  14. 14.
    Freidlin, M.I., Wentzell, A.D.: Random Perturbations of Dynamical Systems. Springer, Berlin (1984)Google Scholar
  15. 15.
    Gnedenko, B.V.: Theory of Probability, 6th edn. Gordon and Breach Sci. Publ., Amsterdam (1997)Google Scholar
  16. 16.
    Griffeath, D.Z.: A maximal coupling for Markov chains. Wahrscheinlichkeitstheorie verw. Gebiete 31(2), 95–106 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Gulinsky, O.V., Veretennikov, AYu.: Large Deviations for Discrete - Time Processes with Averaging. VSP, Utrecht, The Netherlands (1993)zbMATHGoogle Scholar
  18. 18.
    Ibragimov, IbragimovLinnik I.A., Yu, V.: Linnik, Independent and Stationary Sequences of Random Variables. Wolters-Noordhoff Publishing, Groningen (1971)Google Scholar
  19. 19.
    Kalashnikov, V.V.: Coupling Method, its Development and Applications. In the Russian translation of E, Nummelin, General Irreducible Markov Chains and Non-negative Operators (1984)Google Scholar
  20. 20.
    Karlin, S.: A First Course in Stochastic Processes. Academic press, Cambridge (2014)Google Scholar
  21. 21.
    Kato, T.: Perturbation Theory for Linear Operators. Springer Science & Business Media, Berlin (2013)Google Scholar
  22. 22.
    Khasminsky, R.Z.: Stochastic Stability of Differential Equations, 2nd edn. Springer, Berlin (2012).
  23. 23.
    Kolmogorov, A.N.: A local limit theorem for classical Markov chains. (Russian) Izvestiya Akad. Nauk SSSR. Ser. Mat. 13, 281–300 (1949)Google Scholar
  24. 24.
    Kontoyiannis, I., Meyn, S.P.: Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. Appl. Probab. 13(1), 304–362 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Krasnosel’skii, M.A., Lifshits, E.A., Sobolev, A.V.: Positive Linear Systems: The method of positive operators. Helderman Verlag, Berlin (1989)zbMATHGoogle Scholar
  26. 26.
    Krylov, N.M., Bogolyubov, N.N.: Les propriétés ergodiques des suites des probabilités en chaîne. CR Math. Acad. Sci. Paris 204, 1454–1546 (1937)zbMATHGoogle Scholar
  27. 27.
    Krylov, N.V.: Introduction to the Theory of Random Processes. AMS, Providence (2002)CrossRefzbMATHGoogle Scholar
  28. 28.
    Kulik, A.: Introduction to Ergodic Rates for Markov Chains and Processes. Potsdam University Press, Potsdam (2015)zbMATHGoogle Scholar
  29. 29.
    Lindvall, T.: Lectures on the Coupling Method (Dover Books on Mathematics) (paperback) (2002)Google Scholar
  30. 30.
    Markov, A.A.: Extension of the limit theorems of probability theory to a sum of variables connected in a chain (1906, in Russian), reprinted in Appendix B of: R. Howard. Dynamic Probabilistic Systems, volume 1: Markov Chains. Wiley, New Jersey (1971)Google Scholar
  31. 31.
    Nagaev, S.V.: A central limit theorem for discrete-time Markov processes. Selected Translations in Math. Stat. and Probab. 7, 156–164 (1968)Google Scholar
  32. 32.
    Nagaev, S.V.: The spectral method and the central limit theorem for general Markov chains. Dokl. Math. 91(1), 56–59 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Nummelin, E.: General Irreducible Markov Chains (Cambridge Tracts in Mathematics) (paperback). CUP, Cambridge (2008)Google Scholar
  34. 34.
    Papanicolaou, S.V., Stroock, D., Varadhan, S.R.S.: Martingale approach to some limit theorems. In: Ruelle, D.: (ed.) Statistical Mechanics, Dynamical Systems and the Duke Turbulence Conference. Duke Univ. Math. Series, vol. 3. Durham, N.C. (1977).
  35. 35.
    Pardoux, E., Veretennikov, Yu, A.: On Poisson equation and diffusion approximation 2. Ann. Prob. 31(3), 1166–1192 (2003)Google Scholar
  36. 36.
    Puhalskii, A.: Large Deviations and Idempotent Probability. Chapman & Hall/CRC, Boca Raton (2001)Google Scholar
  37. 37.
    Rockafellar, R.T.: Convex Analysis, Princeton Landmarks in Mathematics and Physics (paperback) (1997)Google Scholar
  38. 38.
    Seneta, E.: Non-negative Matrices and Markov Chains, 2nd edn. Springer, New York (1981)CrossRefzbMATHGoogle Scholar
  39. 39.
    Seneta, E.: Markov and the Birth of Chain dependence theory. Int. Stat. Rev./Revue Internationale de Statistique 64(3), 255–263 (1996)zbMATHGoogle Scholar
  40. 40.
    Thorisson, H.: Coupling, Stationarity, and Regeneration. Springer, New York (2000)CrossRefzbMATHGoogle Scholar
  41. 41.
    Tutubalin, V.N.: Theory of Probability and Random Processes. MSU, Moscow (in Russian) (1992)zbMATHGoogle Scholar
  42. 42.
    Varadhan, S.R.S.: Large Deviations and Applications. SIAM, Philadelphia (1984)CrossRefzbMATHGoogle Scholar
  43. 43.
    Vaserstein, L.N.: Markov processes over denumerable products of spaces. Describ. Large Syst. Automata Probl. Inf. Trans. 5(3), 47–52 (1969)Google Scholar
  44. 44.
    Veretennikov, Yu, A.: Bounds for the mixing rate in the theory of stochastic equations. Theory Probab. Appl. 32, 273–281 (1987)Google Scholar
  45. 45.
    Veretennikov, Yu, A.: On large deviations for systems of Itô stochastic equations. Theory Probab. Appl. 36(4), 772–782 (1991)Google Scholar
  46. 46.
    Veretennikov, Yu, A.: On polynomial mixing bounds for stochastic differential equations. Stoch. Process. Appl. 70, 115–127 (1997)Google Scholar
  47. 47.
    Veretennikov, Yu, A.: On polynomial mixing and convergence rate for stochastic difference and differential equations. Theory Probab. Appl. 45(1), 160–163 (2001)Google Scholar
  48. 48.
    Veretennikov, Yu, A.: Parameter and Non-parametric Estimation for Markov Chains. Publ, House of Applied Studies, Mechanics and Mathematics Faculty, Mocow State University (2000). (in Russian)Google Scholar
  49. 49.
    Wentzell, A.D.: A Course in the Theory of Random Processes. McGraw-Hill, New York (1981)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.University of LeedsLeedsUK
  2. 2.National Research University Higher School of EconomicsMoscowRussian Federation
  3. 3.Institute for Information Transmission ProblemsMoscowRussian Federation

Personalised recommendations