From Random Walks to Markov Chains

Abstract

The simplest random walk may be described as follows. A particle moves along a line by steps; each step takes it one unit to the right or to the left with probabilities p and q = 1 − p respectively where 0 < p < 1. For verbal convenience we suppose that each step is taken in a unit of time so that the nth step is made instantaneously at time n; furthermore we suppose that the possible positions of the particle are the set of all integers on the coordinate axis. This set is often referred to as the “integer lattice” on R ¹ = (−∞, ∞) and will be denoted by I. Thus the particle executes a walk on the lattice, back and forth, and continues ad infinitum. If we plot its position X _n as a function of the time n, its path is a zigzag line of which some samples are shown below in Figure 30.