Sojourn Times for Finite Semi-Markov Processes

Abstract

Semi-Markov modelling is an important tool in the reliability field as has been indicated by the examples in Section 1.2.2. Moreover, semi-Markov processes are also interesting from a mathematical viewpoint since they are obtained by relaxing one of the key assumptions of Markov theory: here, the distribution of the (conditional) holding time in any state need not be memoryless but it is arbitrary. Notice the deliberate use of the word memoryless here in order to account for both the discrete-, and the continuous-time case. As is well-known, in these two cases the holding time distributions are geometric and exponential respectively. In what follows, Y = { Y_t: t ≥ 0 } is a semi-Markov process with finite state space, S. The transitions of Y between the states in S are governed by the embedded Markov chain X = { X_n : n = 0, 1, … } whose transition probability matrix is denoted by R. The cumulative distribution functions of the holding times of Y in s are denoted by ${F_{s,s'}},s' \ne s$. These are the conditional distribution functions of the holding time in s given that the next state to be visited is s’. ${F_{s,s'}}$ is a proper distribution function for non-absorbing s only. The semi-Markov process Y will be assumed either irreducible of absorbing. We partition S into n+1 (≥ 3) non empty subsets if Y is absorbing, and then S = A₁ ∪ … ∪ A_n ∪ A_n+1 with A_n+1 denoting the set of all absorbing states; otherwise, the partitioning is S = A₁ ∪ … ∪ A_n, n ≥ 2. (In the latter case, A_n+1 is defined to be the empty set in order to be able to cover both cases by the same set of formulae.) Following the notation inroduced in Chapter 4, for i ∈ { 1, …, n},${T_{{A_i}}}_J = 0$ stands for the length of the jth sojourn of Y in A_i if A_i is visited by Y at least j times. We put ${T_{{A_i}}}_J = 0$ otherwise.