Modelling large timescale and small timescale service variability
 106 Downloads
Abstract
The performance of service units may depend on various randomly changing environmental effects. It is quite often the case that these effects vary on different timescales. In this paper, we consider small and large scale (short and long term) service variability, where the short term variability affects the instantaneous service speed of the service unit and a modulating background Markov chain characterizes the long term effect. The main modelling challenge in this work is that the considered small and long term variation results in randomness along different axes: short term variability along the time axis and long term variability along the work axis. We present a simulation approach and an explicit analytic formula for the service time distribution in the double transform domain that allows for the efficient computation of service time moments. Finally, we compare the simulation results with analytic ones.
Keywords
Short and long term service variability Brownian motion Markov modulation Performance analysis1 Introduction
Service speed variability is a problem that has been observed in many practical application scenarios. For example, in Kimber and Daly (1986), it has been observed for vehicular traffic. More recently this problem has been recognized in data centers (Guo et al. 2014). The effect of variability was also studied in Anjum and Perros (2015) with application to videostreaming. Most of the previous literature, however, focused only on largetimescale variability, where Markovmodulating models represent the random fluctuations of the environment. These set of models are commonly referred to as reward models and have been studied for a long time (Howard 1971).
The variation in the service speed can be modelled by dividing the jobs into “infinitesimal quantities of work to be done” and considering the “speed at which this infinitesimal work is performed”, i.e., the random amount of time needed to execute the infinitesimal amount of work. Then, once a model defines how speed changes over time, the complete system can be modelled in a straightforward way where the amount of work increases gradually along the analysis and the time required to execute the given amount of work is a random process.
If the service process depends on a timedependent random process, e.g., on a modulating background continuous time Markov chain (CTMC) representing the environmental state, whose “clock” evolves according to the time, then the natural performance analysis is based on the gradually increasing time and the randomly varying time dependent environment state.
However, in many real applications, variability is not easily predictable and works at different timescales. Modulating CTMCs (whose “clock” evolves according to the time) works very well to model variability where the parameters of the job execution remain constant for a longer random period of time, and there are few jumps during the execution of one job. Apart from this large scale variability, in this work, we also focus on variability that occurs at much smaller timescales, where the execution speed changes thousands, if not millions, of times during the execution of the main job, and combine it with the more classical modulation that works on a larger timescale.
The remainder of this paper is structured as follows. In Sect. 2 we start by considering only the small timescale variability. In Sect. 3 we additionally introduce also the large timescale variability. Section 4 is devoted to the mathematical analysis of the obtained small and large timescale system. The effects of the considered variability is studied in Sect. 5 through numerical examples, and Sect. 6 concludes the paper.
2 Small timescale variability
In this section, we omit the large timescale variability and instead focus only on small timescale variability. So we assume that the environmental state is unchanged for now.
We focus on the service of a job in a queue whose work requirement, W, is generally distributed according to probability density function \(f_W(x)\).
2.1 Moments of the scaled distribution
3 Combining large and small timescale variability
Large scale variability can be considered using a discrete state continuous time Markov modulating process (MMP), denoted by M(t). We assume the MMP is a continuous time Markov chain (CTMC) on a finite state space with infinitesimal generator matrix Q. In state i, the service is characterised by rate \(\mu _i\) and variance \(\sigma _i^2\).
Only considering large scale variability (that is, assuming \(\sigma _i\equiv 0, \forall i\)) would lead to a standard Markov reward model. However, including smallscale variability makes for an interesting and complex model.

Let \(a_1\) denote the time of the first transition of M(t). As long as X(w) is smaller than \(a_1\), X(w) evolves according to a Brownian motion with parameters \(\mu _i\) and \(\sigma _i^2\) [denoted by BM(\(\mu _i,\sigma _i^2\))].

At time \(a_1\), M(t) changes to some state j. Accordingly, assuming that the first passage of X(w) to \(a_1\) occurs at work amount \(w_1\), for \(w\ge w_1\), X(w) evolves according to a BM(\(\mu _j,\sigma _j\)) (starting from the point \(w_1\) and from level \(a_1\)).

This is repeated for further possible transitions of M(t) at times \(a_2,a_3,\dots \), up to the point u.

If W is random, generate the value of W, denoted by u.

X(w) starts from \(t=0\), \(w=0\), with \(M(0)=i\).

Generate the first transition time \(a_1\) of M(t).

X(w) runs as a BM\((\mu _i, \sigma _i^2)\) until either the value of X(w) reaches \(a_1\) or w reaches u, whichever occurs first.

If u occurred first, then the simulation is finished.

If \(X(w_1)=a_1\) for some \(w_1<W\), then we generate the next state j and also the next transition time \(a_2\) according to the MMP M(t), then continue X(w) as a Brownian motion with parameters \((\mu _j, \sigma _j^2)\) starting from the point \((w_1,a_1)\) until either the value of X(w) reaches \(a_2\) or reaches u, whichever occurs first.

We keep generating new transitions and new Brownian motion sections until we reach u. The service time of the job is \(T=X(u)=X(W)\).
4 Job completion in small and large timescale variable environment
Let X(w) denote the time needed to service a job of fixed size w. We aim to analyse the entire process \(\{X(w),w\ge 0\}\), and, based on that, derive performance measures for X(w) (for a fixed job size w), and also for X(W), where W is possibly random.
The system operates in a random environment characterized by the MMP \(M(t), t\ge 0\), which is a Markov chain with generator Q (and the variable t denotes the time of the MMP). The process X(w) starts from 0 at \(w=0\). When the MMP is in state i, the main process, X(w), is a Brownian motion with parameters \(\mu _i>0\) and \(\sigma _i>0\) (given for each state i). Whenever the MMP makes a transition at time \(t=a\), i.e., when X(w) reaches level a, the MMP switches to a new state k and the main process continues as a Brownian motion with parameters \(\mu _k>0\) and \(\sigma _k\) starting at level a. Then the same procedure continues until the job of size u gets completed.
Theorem 1
Proof

The first term in (14) is the probability that the main process reaches \(u=w\) before hitting level a averaged out according to the distribution of a.

In the second term, the MMP switches to state k at \(u<w\), with the main process at level \(X(w)=a\).

Even though the general idea is that the main process is increasing, using a second order approach means that in the short term, the main process may decrease as well. Hence we should care about negative values of x if possible. The formula (14) is consistent with the possibility that the process X(w) may decrease and the formula is valid for negative values of x as well.
For deterministic work requirement (\(W=w\)) we applied the numerical inverse Laplace transformation method from Horváth et al. (2018) with order 24. This numerical inverse Laplace transform procedure evaluates the Laplace transform function only in points with a positive real part.
5 Numerical examples
5.1 Simulation results
To study the effects of variability, we have applied the procedure outlined in Sect. 3 to simulate the behaviour of the queue with short and long scale variability. In particular, to find the intersection between the Brownian motion and the level determined by the time at which the modulating process changes state, we have discretised the work with a quantum \(\Delta x\), and during the period when the MMP stays in state i, for each quantum we have set the evolution of the time according to a normal distribution \(N(\mu _i \Delta x, \sigma _i^2 \Delta x)\) (following the procedure outlined at the beginning of Sect. 2). The MMP leaves state i at the first time instant in which the discretised BM crosses the level \(T_n\), where \(T_n\) is the time of the nth state transition of the MMP. When the nth state transition occurs in state i, then \(T_n=T_{n1}+\tau _i\), where \(T_{n1}\) is the time of the previous state transition and \(\tau _i\) is exponentially distributed with parameter \(q_{ii}\) (the ith diagonal element of the generator matrix \(\mathbf {Q}\) of the MMP). This simulation approach is indeed an approximation, but it can be made arbitrarily precise by choosing appropriately small values of \(\Delta x\) (at the cost of simulation time). Simulations were run for several choices of \(\Delta x\) to examine the error of this approximation.
In our numerical experiment, we have considered a twostate modulating process with jump rates \(q_{12}\) and \(q_{21}\), and studied the effects of different service speed and variability parameters \(\mu _i\) and \(\sigma _i\) (\(i=1,2\)). Apart from computing performance measures related to the service time distribution, we also included simulation results for the response time in an M/G/1 queue, where jobs arrive according to a Poisson process of rate \(\lambda \) and are served by a single server subject to short and long term variability according to a firstcomefirstserved discipline. Job sizes may be either deterministic or random. \(\lambda \) is set so that the queue is stable.

Base no variability, \(\mu _1 = \mu _2 = 2.4848\) and \(\sigma _1 = \sigma _2 = 0\).

Small (fixed) small term variability with \(\mu _1 = \mu _2 = 2.4848\) and \(\sigma _1 = \sigma _2 = 0.98773\).

Small (variable) small term variability with \(\mu _1 = \mu _2 = 2.4848\) and \(\sigma _1 = 0.4\) and \(\sigma _2 = 1.5\).

Large Long term variability is present, but no short term variability: \(\mu _1 = 2\), \(\mu _2 = 4\), \(\sigma _1 = \sigma _2 = 0\).

Small + Large both effects are combined: \(\mu _1 = 2,\mu _2 = 4,\sigma _1 = 0.4,\sigma _2 = 1.5.\)
Figure 4a shows the service time distribution for the different server variability configurations. For the Base case, as it is expected, all the probability mass is centered along \(\mu _1 E[W] = \mu _2 E[W] = 248.48\). For the Small (fixed) case, the introduction of small term variability destroys the deterministic behaviour, resulting in a smooth distribution still concentrated near \(\mu _1 E[W] = \mu _2 E[W] = 248.48\). For the Small (variable) case, the distribution is similar, with larger tails due to the long term variability in \(\sigma \). For the Large case, in state 1, service time is exactly \(\mu _1 E[W]=200\) ms, and in state 2, service time is exactly \(\mu _2 E[W]=400\) ms. The probability masses in Fig. 4a at 200 ms and 400 ms are associated with the cases when the MMP stays in state 1 (2, respectively) for the whole period of the service. The cases when the MMP experiences state transition during the service are represented by the continuously increasing part of the Large curve. The case that combines both small and large scale variability (Small + Large) further smooths the curves, and the effect is more evident near the two probability masses at 200 ms, and 400 ms.

Deterministic \(W=100\) ms,

Exponential with mean 100 ms,

Erlang with 4 stages with mean 100 ms,
 Hyperexponential with probability density functionwith parameters \(\lambda _1=1/(100(1+\sqrt{0.6})),\lambda _2=1/100((1\sqrt{0.6}))\)$$\begin{aligned} f_W(x)=\frac{1}{2}\lambda _1 e^{\lambda _1 x} + \frac{1}{2}\lambda _2 e^{\lambda _2 x}\quad x>0 \end{aligned}$$
 Pareto with probability density function$$\begin{aligned} f_W(x)=\left\{ \begin{array}{ll} \frac{\frac{5}{4}\cdot 20^{\frac{5}{4}}}{x^{\frac{9}{4}}}&{} \quad x>20, \\ 0 &{}\quad x<20 \end{array} \right. \end{aligned}$$
In particular, Fig. 6a shows the service time distribution for each job size distribution. The effect of service variability is more evident on job length distributions with a lower coefficient of variation. Figure 6b shows the effect on response time: indeed, combining the effect of service variability with a heavy tailed distribution, as for the Pareto case, can create very long queues which can lead to extremely long response times.
5.2 Comparison of analytical and simulation results
For the last batch of simulations, we compare empirical moments from the simulation to moments calculated using the double transform method of Sect. 4.
The system parameters are the same as in (25). Two different job size distributions are examined: deterministic and exponential. To test the inaccuracy of the simulation with finite discretization steps, we run each simulation with two different choices of \(\Delta x\): \(\Delta x=0.05\) ms and \(\Delta x=0.005\) ms.
Table 1 presents the moments of the service time distribution obtained from the simulator and the transform domain description. \(\Delta x=0.05\) ms corresponds to sim. 1 and \(\Delta x=0.005\) ms corresponds to sim. 2.
From Table 1, we observe increasing relative error for higher moments.
Comparison of the numerical analysis and the simulation results
Deterministic job size  

sim. 1  sim. 2  Transform  
\(E(X(W)M(0)=1)\)  215.5  215.8  213.0 
\(E(X(W)M(0)=2)\)  367.1  364.9  359.3 
\(E(X(W)^2M(0)=1)\)  \(4.689 \times 10^4\)  \(4.835\times 10^4\)  \(4.818\times 10^4\) 
\(E(X(W)^2M(0)=2)\)  \(1.383\times 10^5\)  \(1.368\times 10^5\)  \(1.332\times 10^5\) 
\(E(X(W)^3M(0)=1)\)  \(1.133\times 10^7\)  \(1.140\times 10^7\)  \(1.082\times 10^7\) 
\(E(X(W)^3M(0)=2)\)  \(5.303\times 10^7\)  \(5.23\times 10^7\)  \(5.054\times 10^7\) 
\(E(X(W)^4M(0)=1)\)  \(2.846\times 10^9\)  \(2.87\times 10^9\)  \(2.656\times 10^9\) 
\(E(X(W)^4M(0)=2)\)  \(2.060\times 10^{10}\)  \(2.03\times 10^{10}\)  \(1.950\times 10^{10}\) 
Exponential job size  

sim. 1  sim. 2  Transform  
\(E(X(W)M(0)=1)\)  226.1  224.9  219.3 
\(E(X(W)M(0)=2)\)  322.5  330.0  339.7 
\(E(X(W)^2M(0)=1)\)  \(1.081\times 10^5\)  \(1.108\times 10^5\)  \(1.055\times 10^5\) 
\(E(X(W)^2M(0)=2)\)  \(2.030\times 10^5\)  \(2.089\times 10^5\)  \(2.165\times 10^5\) 
\(E(X(W)^3M(0)=1)\)  \(8.171\times 10^7\)  \(8.820\times 10^7\)  \(8.226\times 10^7\) 
\(E(X(W)^3M(0)=2)\)  \(1.892\times 10^8\)  \(1.981\times 10^8\)  \(2.007\times 10^8\) 
\(E(X(W)^4M(0)=1)\)  \(8.680\times 10^{10}\)  \(9.810\times 10^{10}\)  \(9.044\times 10^{10}\) 
\(E(X(W)^4M(0)=2)\)  \(2.352\times 10^{11}\)  \(2.580\times 10^{11}\)  \(2.443\times 10^{11}\) 
6 Conclusions
In this work, we have introduced a queue with a service model where a modulating background Markov process models the large timescale variability, and a secondorder fluid process models the service capacity on small timescale. The resulting service model can be interpreted as a certain type of leveldependent Brownian motion.
We have presented both a simulation approach for the service time and response time of a job for various job size distributions and a double Laplace transform domain analysis of the service time distribution. A numerical example illustrates the effect of small and large scales of service variability. Using that example, we compared the results obtained from simulation and the Laplace transform domain analytical description.
Notes
Acknowledgements
Open access funding provided by Budapest University of Technology and Economics (BME).
References
 Anjum, B., & Perros, H. (2015). Bandwidth estimation for video streaming under percentile delay, jitter, and packet loss rate constraints using traces. Computer Communications, 57, 73–84. https://doi.org/10.1016/j.comcom.2014.08.018.CrossRefGoogle Scholar
 Breuer, L. (2012). Occupation times for Markovmodulated Brownian motion. Journal of Applied Probability, 49(2), 549–565. https://doi.org/10.1239/jap/1339878804.CrossRefGoogle Scholar
 Guo, J., Liu, F., Huang, X., Lui, J. C., Hu, M., Gao, Q., & Jin, H. (2014). On efficient bandwidth allocation for traffic variability in datacenters. In: Proceedings—IEEE INFOCOM, pp. 1572–1580.Google Scholar
 Horváth, I., Talyigás, Z., & Telek, M. (2018). An optimal inverse Laplace transform method without positive and negative overshoot—An integral based interpretation. Electronic Notes in Theoretical Computer Science, 337, 87–104.Google Scholar
 Howard, R. (1971). Dynamic probabilistic systems, Volume II: SemiMarkov and decision processes. New York: Wiley.Google Scholar
 Karandikar, R. L., & Kulkarni, V. G. (1995). Secondorder fluid flow models: Reflected Brownian motion in a random environment. Operations Research, 43(1), 77–88. https://doi.org/10.1287/opre.43.1.77.CrossRefGoogle Scholar
 Kimber, R., & Daly, P. (1986). Timedependent queueing at road junctions: Observation and prediction. Transportation Research Part B: Methodological, 20(3), 187–203. https://doi.org/10.1016/01912615(86)900160.CrossRefGoogle Scholar
 Schilling, R., & Partzsch, L. (2012). Brownian motion: An introduction to stochastic processes. Berlin: De Gruyter.Google Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.