Imbedded Markov Chain Models

Abstract

In the last chapter, we used Markov process models for queueing systems with Poisson arrivals and exponential service times. To model a system as a Markov process, we should be able to give complete distribution characteristics of the process beyond time t, using what we know about the process at t and changes that may occur after t, without referring back to the events before t. When arrivals are Poisson and service times are exponential, because of the memoryless property of the exponential distribution we are able to use the Markov process as a model. If the arrival rate is λ and service rate is μ, at any time point t, time to next arrival has the exponential distribution with rate λ, and if a service is in progress, the remaining service time has the exponential distribution with rate μ. If one or both of the arrival and service distributions are non-exponential, the memoryless property does not hold and a Markov model of the type discussed in the last chapter does not work. In this chapter, we discuss a method by which a Markov model can be constructed, not for all t, but for specific time points on the time axis.