Markov Transition Functions

Abstract

Much of the remainder of this book is devoted to Markov processes. A definition of the Markov property was given in Chapter 1, but now we will begin over again in a different spirit. Instead of starting with the general Markov process itself, we will first examine how the transition probability functions of Markov processes can be constructed and studied. While we are doing this in the present chapter and the next one, no probability spaces or random variables will be needed (although the word “probability” will be used informally as a guide to the motivation for the work). Later, in Chapter 8, we will actually construct the processes themselves, and then the connection with the Markov property defined in Chapter 1 will be established.