Basic Notions and Illustrations

Summary

The probability space for a one sided Markov process with stationary transition mechanism is set up in the discrete and continuous time parameter case under appropriate conditions in section 1. The extension (if possible) to a two-sided process is discussed, as well as the Chapman-Kolmogorov equation for first order transition probabilities. A number of illustrative examples are taken up in the following sections. The asymptotic properties of transition probabilities for Markov chains (Markov processes with a countable state space) are considered in section 2. This motivates in part the later development of an ergodic theorem (in Chapter 4 section 2) for Markov processes with a general state space. The classical example of a sequence of independent random variables is taken up in section 3. There is a brief discussion of the theory of errors and then a derivation of the Poisson approximation to the Binomial distribution and the normal approximation to the distribution of a sum of independent random variables, both with error terms. The theorems on the Poisson and normal approximation are not only of independent interest but are also used later in Chapter 7 section 1 to obtain a remarkable result of Kolmogorov on the approximation of the distribution of a sum of independent and identically distributed random variables by an infinitely divisible distribution with error term. A brief discussion of the continuous parameter Poisson and Wiener (Brownian motion) processes is given in section 4. The classical result of Polya on recurrence of one and two dimensional and nonrecurrence of three dimensional random walks is given in section 5. A generalization (due to Dudley) for random walks on countable Abelian groups is then developed.