Continuous-Time Markov Models

Abstract

This section introduces random point processes of which the simplest example is the homogeneous Poisson process. A random point process is, roughly speaking, a countable random set of points of the real line. In most applications to engineering and operations research, a point of a point process is the time of occurrence of some event, and this is why points are also called events. For instance, the arrival times of customers at the desk of a post office or jobs at the central processing unit of a computer are point-process events. In biology, an event can be the time of birth of an organism. In physiology, the firing time of a neuron is also an event. In general, point processes on the line appear in stochastic models where the state of a system is changed by the occurrence of some event. In this case one can use the phrase stochastic systems driven by point pmcesses, and if the state of the system is discrete, one sometimes prefers to talk about stochastic discrete event systems. The basic examples are the Poisson process and the continuous-time Markov chain.