Stochastic Processes, Markov Chains and Master Equations

Abstract

The last chapter dealt with several basic aspects of stochastic processes through Brownian motion, which represents the oldest and the best known physical example of the Markov processes . In this chapter, we study general concepts and theoretical frameworks of Markov processes that can extend the ideas of the foregoing chapter to a variety of physical and nonphysical phenomena. The stochastic variables , denoted by $q\left( t \right)$ below, are not limited to the position and velocity of a Brownian particle as in the last chapter; they can represent the dynamical states of more complex systems such as microscopic conformational states of a biological molecule. For a biological complex, the Markov processes of utmost concern are mesoscopic degrees of freedom ${\mathcal{Q}}\left( t \right)$ that evolve under the associated effective Hamiltonian or free energy function ${\mathcal{F}}\left\{ {\mathcal{Q}} \right\}$ (Chap. 5). In this chapter and next we study the general mathematical framework of the stochastic processes without recourse to the microscopic dynamics, while giving basic introductions of joint probabilities and time correlation functions. Two particularly well-known and useful equations for the Markov processes, the master equation and the Fokker-Planck equations are derived and solved for a wide class of examples. Their relevance and applications are discussed.