Perfect Simulation of a Class of Stochastic Hybrid Systems with an Application to Peer to Peer Systems

Abstract

In this paper we present a class of hybrid systems made of deterministic differential equations and random discrete jumps. We then show how to construct a simulation of such a stochastic hybrid system that provides perfect samples of its asymptotic behavior based on the extension to continuous state-space of coupling-from-the-past techniques introduced by Foss and Tweedie (Stoch Models 14:187–204, 1998) and using suitable envelope trajectories to tackle non-monotonicity. The applicability of the method is illustrated by showing how this framework can be used to model the Squirrel peer to peer system and by reporting a simulation study based on this approach. This paper provides both a framework on how to carry simulation based experimental studies of large and complex hybrid systems and its application in the Squirrel model demonstrating how versatile and powerful this approach can be over a typical example.

Appendices

Appendix

A. More on the Squirrel differential equation

This appendix is used to explain the construction of the differential equation

$$ \frac{dx}{dt} = f_N(x(t)) = \sigma N \left(1-\left(\frac{x(t)}{C}\right)^{\alpha}\right) - \theta x(t), $$

(15)

and to assess its properties (uniqueness of the solution, asymptotic behavior, rate of convergence).

2.1 A.1 Modeling the miss probability

We now explain how the miss probability of Squirrel can be modeled by

$$ 1- \left(\frac{x}{C}\right)^{\alpha}. $$

(16)

This model was first proposed in (Clévenot and Nain 2004; Clévenot-Perronnin 2005). In this section we will not only recall the theoretical grounds for this model, but we will also provide experimental results to validate the model and estimate α.

A uniform-popularity model would straightforwardly give a linear model $P({\mbox{hit}|N(t),x(t)})= \frac{x(t)}{C}$ (the hit probability being 1 - the miss probability). However, it was shown in (Breslau et al. 1999) that Web objects exhibit a Zipf-like popularity distribution, i.e., the k-th most popular object is requested with probability $\frac{\Omega}{k^{\beta}}$, where 0 < β < 1 is the skew parameter of the distribution and $\Omega = 1/\sum_{i=1}^C i^{-\beta}$ is a normalization factor. In (Breslau et al. 1999) it was shown that β is close to 0.8. Assuming that the x(t) present objects in cache are the x(t) most popular ones (since they are more likely to be requested), a rough approximation for the hit probability is

$$ P({\mbox{hit}|x(t)})= \sum\limits_{i=1}^{\lfloor x(t) \rfloor} \frac{\Omega}{i^\beta} \approx \frac{x(t)^{1-\beta}-1}{C^{1-\beta}-1}, $$

(17)

by using the approximation $\sum_{i=1}^{\lfloor x \rfloor} i^{-\beta}\approx \int_1^x t^{-\beta} dt = (x^{1-\beta}-1)/(1-\beta)$ for x ≥ 1. From Eq. 17 we get the rough approximation $P({\mbox{hit}|x(t)})= (\frac{x(t)}{C})^{\alpha}$ with α = 1 − β.

To validate this model we have conducted the following experiment. We have initially drawn x samples from a Zipf-like distribution. Then we drew an additional sample z. This sample z results in a hit if it belongs to the set of x initially drawn samples, in a miss otherwise. This latter step is repeated 1,000 times to get a 95% confidence interval no larger than 0.03 on the hit probability. Then we compared the experimental hit probability to the model given by Eq. 16 and we computed the value of α that best fitted the numerical results. Figure 9 show how perfectly Eq. 16 follows the experimental hit probability.

Fig. 9

Hit probability as a function of the number of cached objects x for the Zipf-like distribution with parameter β = 0.8 and with C = 1,000 possible objects

However, the value of α used to get this accuracy slightly differs from 1 − β. Actually for most values of β between 0.1 and 0.9 we found experimentally that α ≈ 1.2 − β for β < 1. We conjecture that this discrepancy is partly due to the fact that the x cached objects are not necessarily the x most popular ones. It may also be due to the approximations used to get Eq. 17.

2.2 A.2 Asymptote

Lemma 9

Any solution x(t) of the differential equation converges to a unique limit ℓ _N , for all initial conditions.

Proof

Setting $\frac{dx}{dt}=0$ in Eq. 15 we get f _N (x) = 0. This equation is verified by potential asymptotes for x(t).

Since $\frac{df_N}{dx}<0$ for all x ≥ 0 and for all N > 0, and since f _N (0) = σ N > 0 and f _N (C) = − θ C < 0, equation f _N (x) = 0 admits a unique positive solution denoted by ℓ _N :

$$ f_N(\ell_N)=0. $$

(18)

Therefore any solution x _N (t) of Eq. 15 admits a unique asymptote ℓ _N . □

Computing ℓ _N can be done by using a classical Newton method (Press et al. 1992), that converges very fast here.

2.3 A.3 Uniqueness of the solution

The differential equation (15) is not Lipschitz in 0 as soon as α < 1. Therefore, the classical techniques used to prove uniqueness do not apply here. However, uniqueness is required for our simulation method to work.

Lemma 10

The Eq. 15 admits a unique solution for any initial condition $x^0 \!{\stackrel{\rm{def}}{=}}\! x(0 ) \!\geq\! 0$.

Proof

Let us consider first the case $x^0 < \ell_N$. In that case, if x ₁(t) is one solution, $\frac{dx_1}{dt} = f_N(x_1(t)) > 0$ for all t ≥ 0, so that x ₁(t) is strictly increasing from x ⁰ to ℓ _N . Therefore, the inverse function $x^{-1}_1(u): [x^0,\ell_N) \to {\mathbb R}$ is well defined. If we consider another solution x ₂(t) then the derivative of the inverse functions $x^{-1}_1(u)$ and $x^{-1}_2(u)$ verify $\frac{dx_1^{-1}}{du}(u) = \frac{1}{f_N(u)} = \frac{dx_2^{-1}}{du}(u)$ for all $u \in [x^0,\ell_N)$. So that $x^{-1}_1(u) - x^{-1}_2(u)$ is constant and does not depend on u. When u = x ⁰, then $x^{-1}_1(x^0) = 0$ and $x^{-1}_2(x^0) = 0$. This implies that $x^{-1}_1(u) = x^{-1}_2(u)$ for all $u \in [x^0,\ell_N)$, so that x ₁(t) = x ₂(t) for all t ≥ 0. If $x^0 = \ell_N$, then the derivative of any solution remains equal to zero, so that there is a unique solution x(t) ≡ ℓ _N . Finally, the case $x^0 > \ell_N$ is similar to the case $x^0 < \ell_N$, with decreasing solutions instead of increasing ones. □

2.4 A.4 Rate of convergence

Let us define the rate of convergence γ _N (t) at time t of the ODE solution x _N (t) as follows: if x _N (t) < ℓ _N ,

$$ \gamma_N(t){\stackrel{\rm{def}}{=}} \lim\limits_{d\to 0} \frac{(x_N(t+d)-\ell_N)-(x_N(t)-\ell_N)}{d(\ell_N-x_N(t))}, $$

(19)

and $\gamma_N {\stackrel{\rm{def}}{=}} \lim_{d\to 0}\frac{(x_N(t+dt)-\ell_N)-(x_N(t)-\ell_N)}{dt(x_N(t)-\ell_N)}$ otherwise. Note that γ _N  > 0 since x _n (t) ≤ ℓ _N in steady state as shown in Appendix B.

We now simplify Eq. 19 when x _N (t) < ℓ _N . The case x _N (t) > ℓ _N is exactly similar and will be omitted.

Taking the limit as d →0 we get

$$ \gamma_N(t)=\lim\limits_{d\to 0} \frac{x_N(t+d)-x_N(t)}{d(\ell_N-x_N(t))},\\ $$

(20)

$$ \;\;\;\;\;\;\;\;\;= \frac{\frac{dx_N}{dt}(t)}{\ell_N-x_N(t)}, $$

(21)

$$ \;\;\;\;\;\;\;\;\; = \frac{\frac{dx_N}{dt}(t)-f_N(\ell_N)}{\ell_N-x_N(t)}, $$

(22)

where Eq. 22 is obtained using Eq. 18. Then when x _N (t) →ℓ _N we get the asymptotic rate of convergence,

$$ \gamma_N =\lim\limits_{x\to \ell_N} \frac{f_N(x)-f_N(\ell_N)}{\ell_N-x}, $$

(23)

$$ \;\;\;\;\; = -\frac{df}{dx}(\ell_N) = \frac{\sigma \alpha N + \theta (1-\alpha) \ell_N}{\ell_N}. $$

(24)

Lemma 11

If x _N (0) < ℓ _N , then for all t > 0, γ _N (t) ≥ γ _N and for all t > 0, $x_N(t) \geq \ell_N (1-\exp(-\gamma_N t))$.

Proof

Note that if x _N (0) < ℓ _N then (in the absence of jumps) we have x _N (t) < ℓ _N for all t > 0. From Eqs. 21 and 24 we have for all N:

$$\begin{array}{lll} \gamma_N(t) - \gamma_N &=& \frac{\frac{dx_N}{dt}(t)}{\ell_N-x_N(t)}+\frac{df}{dx}(\ell_N),\\ &=&\frac{f_N(x_N(t))+(\ell_N-x_N(t))\frac{df}{dx}(\ell_N)}{\ell_N-x_N(t)},\\ &=&\frac{\sigma N\left(1-\left(\frac{x_N(t)}{C}\right)^{\alpha}\right)-\theta x_N(t)-(\ell_N-x_N(t))(-\frac{\sigma N\alpha(\frac{\ell_N}{C})^{\alpha}}{\ell_N}-\theta)}{\ell_N-x_N(t)}. \end{array}$$

The numerator $g(x(t)) {\stackrel{\rm{def}}{=}} \sigma N\left(1\!-\!\left(\frac{x_N(t)}{C}\right)^{\alpha}\right)-\theta x_N(t)-(\ell_N\!-\!x_N(t))(-\frac{\sigma N\alpha(\frac{\ell_N}{C})^{\alpha}}{\ell_N}-\theta)$ is positive since:

$$ \frac{dg}{dx}(x(t)) = \frac{\sigma N \alpha \ell_N^{\alpha}x_N(t)^{\alpha} (-\ell_N^{1-\alpha}+x_N(t)^{1-\alpha})}{C^{\alpha}x_N(t)\ell_N} <0, $$

and lim _t→ ∞ g(x(t)) = 0. This proves the first inequality of the lemma.

Using Eq. 21 this first inequality gives

$$\begin{array}{rll} \frac{\frac{dx_N}{dt}(t)}{\ell_N-x_N(t)} &\geq& \gamma_N\\ \frac{dx_N}{dt}(t)&\geq& \gamma_N(\ell_N-x_N(t)). \end{array}$$

Let y(t) be the solution of the first-order linear ODE $\frac{dy}{dt}(t)= \gamma_N(\ell_N-y(t))$ with y(0) = 0. This solution is $y(t)=\ell_N(1-\exp(-\gamma_N t))$. We now prove the second inequality, i.e., that for all t > 0, x _N (t) ≥ y(t). Let us consider the derivative of the inverse functions (which are always well defined here):

$$ \frac{dy^{-1}}{du}(u)=\frac{1}{\gamma_N(\ell_N-u)}\geq \frac{dx_N^{-1}}{du}(u). $$

As a result we have $x_N^{-1}(u) \leq y^{-1}(u)$ for all u ≥ 0 since $x^{-1}_N(0)=y^{-1}(0)=0$. This implies that x _N (t) ≥ y(t) for all t ≥ 0.

2.5 A.5 Numerical integration

First, note that in the case α = 1 the ODE (15) admits a closed-form solution on [T _n ,T _n + 1):

$$ x(t)=\frac{\sigma N(T_n)}{\frac{\sigma N(T_n)}{C}+\theta}+\left(x(T_n)-\frac{\sigma N(T_n)}{\frac{\sigma N(T_n)}{C}+\theta}\right) e^{-(t-T_n)(\theta+\frac{\sigma N(T_n)}{C})}. $$

(25)

However, when $\alpha \not= 1$, the equation does not have a closed form solution and must be solved numerically. In this paper, we only consider a first order Runge–Kutta method. More sophisticated integration is possible, yielding a better precision with fewer discretization steps.

The fact that the equation may not be Lipschitz in 0, makes the computation of the error a little tricky. Here, we simply compute a bound on the error made by using the classical Euler integration method: x _n  = x _n − 1 + h f _N (x _n − 1). One has to consider the first step of integration apart. The error on the first step is bounded by h f _N (0) ≤ h σN. As for all subsequent steps, n, the error is bounded by |h ² x′′(a)| for some nh ≤ a ≤ (n + 1)h.

$$\begin{array}{lll} h^2 |x''(a)| &\leq& |h^2 \left[ \sigma N \left(1- \left(\frac{x_n}{C}\right)^\alpha \right) - \theta x_n \right] \left[ -\sigma N \alpha \frac{x_n^{\alpha-1}}{C^\alpha} - \theta \right]|,\\ &\leq & h^2 \frac{\sigma^{\alpha+1} N^{\alpha+1}}{C^\alpha} \alpha h^{\alpha-1}, \\ &\leq & \frac{\sigma^{\alpha+1} N^{\alpha+1}}{C^\alpha} \alpha h^{\alpha+1}. \end{array}$$

Therefore, the total error by integrating over a duration T is bounded by

$$e = h \sigma N + \alpha T \frac{\sigma^{\alpha+1} N^{\alpha+1}}{C^\alpha} h^{\alpha}.$$

B. More on the Squirrel Markov chain

The squirrel Markov chain S _n  = (N(T _n ),x(T _n )) is proved uniformly ergodic in Section 4 and (0,0) is an atom. However, its final class (the set of states reachable with positive probability) is not clearly stated. We define ${\cal C} = \cup \{ A : \exists m | P^m(s,A) > 0\}$ for all $s \in {\cal D}$

Lemma 12

The final reachable class ${\cal C} = \{ (N,x) | N \in \{ 1,\ldots,N_{max}\}\; , \; 0 < x < \ell_N \} \cup \{ (0,0)\} \cup \{ (1,0)\}$.

Proof

First note that state (0,0) is reachable from any state (N,x) ∈ {0,...N _max } ×[0,C] (see Section 4). Now, from state (0,0), one may reach state (N,x) for any 0 < N ≤ N _max and any x such that 0 < x < ℓ _N by letting N arrivals come almost simultaneously and let the time run, so that x may grow from 0 to ℓ _N . This means that ${W} {\stackrel{\rm{def}}{=}} \{ (N,x) | N \in \{1,\ldots,N_{max}\} , 0< x < \ell_N \} \subset {\cal C}$. Also, state (1,0) is the first state visited from state (0,0).

Next, we show that during the evolution of the Markov chain starting in (0,0), no state with x > ℓ _N may ever be reached. Starting in state (N,x) with x < ℓ _N , one may only reach (N′,x′) with N′ = N, or N′ = N − 1 or N′ = N + 1.

• 1. If N′ = N then x′ < ℓ _N and the state stays in W.

• 2. If N′ = N + 1 then x′ = x < ℓ _N  ≤ ℓ _N + 1 and the state stays in W.

• 3. If N′ = N − 1 then x′ may reach any value between x (N − 1)/N and ℓ _N − 1. The proof is over if one can show that (N − 1)x/N < ℓ _N − 1.

It remains to show that (N − 1)x/N < ℓ _N − 1 for any 0 < x < ℓ _N . It is enough to show that ℓ _N N′/N < ℓ _N′, with N′ = N − 1.

Let us consider f _N′(ℓ _N N′/N):

$$ \begin{array}{lll} f_{N'}\left(\ell_N \frac{N'}{N} \right) & = & \sigma N' \left(1-\left( \frac{N'\ell_N}{N C} \right)^\alpha \right) - \theta \frac{N'}{N} \ell_N,\\ & = & \sigma N' \left(\left( \frac{N'}{N} \right)^\alpha \left( 1-\left( \frac{\ell_N}{C} \right)^\alpha \right)+ 1-\left( \frac{N'}{N} \right)^\alpha \right) - \theta \frac{N'}{N} \ell_N, \\ &=& \left(1- \left( \frac{N'}{N} \right)^\alpha \right) \left( \sigma - \frac{ \theta \ell_N }{N} \right), \\ &=& \left(1-\left( \frac{N'}{N} \right)^\alpha \right) \sigma \left( \frac{\ell_N}{NC}\right)^\alpha ,\\ & \geq & 0. \end{array}$$

f _N′(ℓ _N N′/N) ≥ 0 means that ℓ _N N′/N  <  ℓ _N′ because f _N′ is decreasing and f _N′(ℓ _N ′) = 0 (see Appendix A.2). □