Delay asymptotics and bounds for multitask parallel jobs

Abstract

We study delay of jobs that consist of multiple parallel tasks, which is a critical performance metric in a wide range of applications such as data file retrieval in coded storage systems and parallel computing. In this problem, each job is completed only when all of its tasks are completed, so the delay of a job is the maximum of the delays of its tasks. Despite the wide attention this problem has received, tight analysis is still largely unknown since analyzing job delay requires characterizing the complicated correlation among task delays, which is hard to do. We first consider an asymptotic regime where the number of servers, n, goes to infinity, and the number of tasks in a job, $k^{(n)}$, is allowed to increase with n. We establish the asymptotic independence of any $k^{(n)}$ queues under the condition $k^{(n)}= o(n^{1/4})$. This greatly generalizes the asymptotic independence type of results in the literature, where asymptotic independence is shown only for a fixed constant number of queues. As a consequence of our independence result, the job delay converges to the maximum of independent task delays. We next consider the non-asymptotic regime. Here, we prove that independence yields a stochastic upper bound on job delay for any n and any $k^{(n)}$ with $k^{(n)}\le n$. The key component of our proof is a new technique we develop, called “Poisson oversampling.” Our approach converts the job delay problem into a corresponding balls-and-bins problem. However, in contrast with typical balls-and-bins problems where there is a negative correlation among bins, we prove that our variant exhibits positive correlation.

Appendices

Appendix A: Proof of Lemma 3

Lemma 3 (Restated)

$$\begin{aligned} d_{TV}\Bigl (\widetilde{\pi }^{(n,k^{(n)})},\hat{\pi }^{(k^{(n)})}\Bigr )=O\Biggl (\biggl (\frac{k^{(n)}}{n^{1/4}}\biggr )^2\Biggr ). \quad {(17)\, (\mathrm{Restated})} \end{aligned}$$

Proof

This proof has a similar flavor to the proofs of Lemmas 1 and 2. Recall that $\left( \widetilde{\varvec{W}}^{(n,k^{(n)})}(t),t\ge 0\right)$, the workload processes of the first $k^{(n)}$ queues in the system $\widetilde{\mathcal {S}}^{(n)}$, are $k^{(n)}$ independent M/G/1 queues each with arrival rate $\widetilde{\lambda }^{(n)}$ and service time distribution G. We couple this with $\left( \hat{\varvec{W}}^{(k^{(n)})}(t),t\ge 0\right)$, where $\hat{\varvec{W}}^{(k^{(n)})}(t)=\bigl (\hat{W}_1(t),\dots ,\hat{W}_{k^{(n)}}(t)\bigr )$ is the workload vector of $k^{(n)}$ independent M/G/1 queues each with arrival rate $\lambda$ and service time distribution G. Then, $\hat{\pi }^{(k^{(n)})}$ is its stationary distribution. We will prove the bound on $d_{TV}\bigl (\widetilde{\pi }^{(n,k^{(n)})},\hat{\pi }^{(k^{(n)})}\bigr )$ by showing that $\left( \widetilde{\varvec{W}}^{(n,k^{(n)})}(t),t\ge 0\right)$ and $\left( \hat{\varvec{W}}^{(k^{(n)})}(t),t\ge 0\right)$ are close.

Now we specify the coupling. All the queues start from empty, i.e., $\hat{W}_i(0)=\widetilde{W}^{(n)}_i(0)=0$ for all $i=1,2,\dots ,k^{(n)}$. When there is a task arrival to some queue of $\hat{\varvec{W}}^{(k^{(n)})}$, we let a task arrive to the corresponding queue of $\widetilde{\varvec{W}}^{(n,k^{(n)})}$ with probability $\frac{\widetilde{\lambda }^{(n)}}{\lambda }$, and let these two tasks require the same service time. So with probability $1-\frac{\widetilde{\lambda }^{(n)}}{\lambda }$ there is no task arrival to $\widetilde{\varvec{W}}^{(n,k^{(n)})}$.

We pick a time $\tau ^{(n)}=O\Bigl (\frac{n^{1/2}}{k^{(n)}}\Bigr )$. Let $\hat{\pi }^{(k^{(n)})}_{\tau ^{(n)}}$ denote the distribution of $\hat{\varvec{W}}^{(k^{(n)})}(\tau ^{(n)})$. Then,

$$\begin{aligned} d_{TV}\Bigl (\widetilde{\pi }^{(n,k^{(n)})},\hat{\pi }^{(k^{(n)})}\Bigr )\le & {} d_{TV}\Bigl (\widetilde{\pi }^{(n,k^{(n)})}_{\tau ^{(n)}},\hat{\pi }^{(k^{(n)})}_{\tau ^{(n)}}\Bigr )\\&+\,d_{TV}\Bigl (\widetilde{\pi }^{(n,k^{(n)})}_{\tau ^{(n)}},\widetilde{\pi }^{(n,k^{(n)})}\Bigr )+d_{TV}\Bigl (\hat{\pi }^{(k^{(n)})}_{\tau ^{(n)}},\hat{\pi }^{(k^{(n)})}\Bigr ). \end{aligned}$$

Noting Lemma 2, we have

$$\begin{aligned} d_{TV}\Bigl (\widetilde{\pi }_{\tau ^{(n)}}^{(n,k^{(n)})},\widetilde{\pi }^{(n,k^{(n)})}\Bigr )&=O\Biggl (\biggl (\frac{k^{(n)}}{n^{1/4}}\biggr )^2\Biggr ),\end{aligned}$$

(33)

$$\begin{aligned} d_{TV}\Bigl (\hat{\pi }_{\tau ^{(n)}}^{(k^{(n)})},\hat{\pi }^{(k^{(n)})}\Bigr )&=O\Biggl (\biggl (\frac{k^{(n)}}{n^{1/4}}\biggr )^2\Biggr ). \end{aligned}$$

(34)

Next we bound $d_{TV}\Bigl (\widetilde{\pi }^{(n,k^{(n)})}_{\tau ^{(n)}},\hat{\pi }^{(k^{(n)})}_{\tau ^{(n)}}\Bigr )$ using arguments similar to those in the proof of Lemma 1. By the coupling, $\hat{\varvec{W}}^{(k^{(n)})}(t)$ and $\widetilde{\varvec{W}}^{(n,k^{(n)})}(t)$ are different for some $t\in [0,\tau ^{(n)}]$ only when some task arrives to $\hat{\varvec{W}}^{(k^{(n)})}$ but not to $\widetilde{\varvec{W}}^{(n,k^{(n)})}$. We denote this event by $\mathcal {E}$. Then,

$$\begin{aligned} d_{TV}\Bigl (\widetilde{\pi }^{(n,k^{(n)})}_{\tau ^{(n)}},\hat{\pi }^{(k^{(n)})}_{\tau ^{(n)}}\Bigr )\le \mathbb {P}(\mathcal {E}). \end{aligned}$$

So the remainder of this proof is dedicated to bounding $\mathbb {P}(\mathcal {E})$.

Consider the time interval $[0,\tau ^{(n)}]$. Let A be the number of task arrivals to $\hat{\varvec{W}}^{(k^{(n)})}$ during this time interval. Then,

$$\begin{aligned} \mathbb {P}(\mathcal {E})&=\sum _{j=0}^{\infty }\mathbb {P}(A=j)\mathbb {P}(\mathcal {E}\mid A=j)\nonumber \\&\le \sum _{j=0}^{\infty }\frac{(\lambda k^{(n)}\tau ^{(n)})^je^{-k^{(n)}\lambda \tau ^{(n)}}}{j!}j\biggl (1-\frac{\widetilde{\lambda }^{(n)}}{\lambda }\biggr ) \nonumber \\&= k^{(n)}\tau ^{(n)}(\lambda -\widetilde{\lambda }^{(n)}), \end{aligned}$$

(35)

where we have used a union bound for (35). By definition,

$$\begin{aligned} \widetilde{\lambda }^{(n)}&=\frac{\varLambda ^{(n)}}{ k^{(n)}}\Biggl (1-\frac{\left( {\begin{array}{c}n-k^{(n)}\\ k^{(n)}\end{array}}\right) }{\left( {\begin{array}{c}n\\ k^{(n)}\end{array}}\right) }\Biggr )\\&\ge \frac{\varLambda ^{(n)}}{ k^{(n)}}\Biggl (1-\biggl (1-\frac{k^{(n)}}{n}\biggr )^{k^{(n)}}\Biggr )\\&=\frac{\varLambda ^{(n)}}{ k^{(n)}}\biggl (\frac{(k^{(n)})^2}{n}+O\biggl (\frac{(k^{(n)})^4}{n^2}\biggr )\biggr )\\&=\lambda +O\biggl (\frac{(k^{(n)})^2}{n}\biggr ). \end{aligned}$$

Therefore,

$$\begin{aligned} \mathbb {P}(\mathcal {E})&= O\Biggl (\biggl (\frac{k^{(n)}}{n^{1/4}}\biggr )^2\Biggr ), \end{aligned}$$

which completes the proof. $\square$

Appendix B: Proof of Corollary 1

Corollary

1 (Restated) Consider an n-server system in the limited fork–join model with $k^{(n)}=o(n^{1/4})$, job arrival rate $\varLambda ^{(n)}=n\lambda /k^{(n)}$, and exponentially distributed service times with mean $1/\mu$. Then, the steady-state job delay, $T^{(n)}$, converges as:

$$\begin{aligned} \lim _{n\rightarrow \infty }\sup _{\tau \ge 0}\left| \mathbb {P}\bigl (T^{(n)}\le \tau \bigr )-\left( 1-e^{-(\mu -\lambda )\tau }\right) ^{k^{(n)}}\right| =0. \quad {(8)\,(\mathrm{Restated})} \end{aligned}$$

Specifically, if $k^{(n)}\rightarrow \infty$ as $n\rightarrow \infty$, then

$$\begin{aligned} \frac{T^{(n)}}{H_{k^{(n)}}/(\mu -\lambda )}\Rightarrow 1,\quad \text {as }n\rightarrow \infty , \quad {(9)\, (\mathrm{Restated})} \end{aligned}$$

where $H_{k^{(n)}}$ is the $k^{(n)}$-th harmonic number, and further,

$$\begin{aligned} \lim _{n\rightarrow \infty }\frac{\mathbb {E}\bigl [T^{(n)}\bigr ]}{H_{k^{(n)}}/(\mu -\lambda )}=1. \quad {(10)\,(\mathrm{Restated})} \end{aligned}$$

Proof

When the service times are exponentially distributed, each queue is an M/M/1 queue, and thus the cdf of the task delay at each queue, F, is given by

$$\begin{aligned} F(\tau )=1-e^{-(\mu -\lambda )\tau }. \end{aligned}$$

Then, the convergence in (8) directly follows from Theorem 1.

To prove the weak convergence of $\frac{T^{(n)}}{H_{k^{(n)}}/(\mu -\lambda )}$ in (9), we first note that

$$\begin{aligned} \frac{\hat{T}^{(n)}}{H_{k^{(n)}}/(\mu -\lambda )}\Rightarrow 1,\quad \text {as }n\rightarrow \infty , \end{aligned}$$

which is a direct implication of a standard result in the asymptotic theory of extremes (see, for example, Theorem 8.12 in [7]). Combining this with (8) yields (9).

To prove the convergence of the expectation in (10), we actually need the stochastic dominance shown in Theorem 2. The expectation in (10) can be written as

$$\begin{aligned} \frac{\mathbb {E}\bigl [T^{(n)}\bigr ]}{H_{k^{(n)}}/(\mu -\lambda )}&=\int _{0}^{\infty }\mathbb {P}\biggl (\frac{T^{(n)}}{H_{k^{(n)}}/(\mu -\lambda )}> \tau \biggr )\mathrm{d}\tau . \end{aligned}$$

By Theorem 2, for any $\tau \ge 0$,

$$\begin{aligned} \mathbb {P}\biggl (\frac{T^{(n)}}{H_{k^{(n)}}/(\mu -\lambda )}> \tau \biggr )\le \mathbb {P}\biggl (\frac{\hat{T}^{(n)}}{H_{k^{(n)}}/(\mu -\lambda )}> \tau \biggr ). \end{aligned}$$

Since

$$\begin{aligned} \frac{\hat{T}^{(n)}}{H_{k^{(n)}}/(\mu -\lambda )}\Rightarrow 1,\quad \text {as }n\rightarrow \infty , \end{aligned}$$

and

$$\begin{aligned} \frac{\mathbb {E}\bigl [\hat{T}^{(n)}\bigr ]}{H_{k^{(n)}}/(\mu -\lambda )}=\int _{0}^{\infty }\mathbb {P}\biggl (\frac{\hat{T}^{(n)}}{H_{k^{(n)}}/(\mu -\lambda )}> \tau \biggr )\mathrm{d}\tau =1, \end{aligned}$$

by the General Lebesgue Dominated Convergence Theorem (see, for example, Theorem 19 in [38]), we can take the limit inside the integral and, using (9), get

$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{\mathbb {E}\bigl [T^{(n)}\bigr ]}{H_{k^{(n)}}/(\mu -\lambda )}&=\int _{0}^{\infty }\lim _{n\rightarrow \infty }\mathbb {P}\biggl (\frac{T^{(n)}}{H_{k^{(n)}}/(\mu -\lambda )}> \tau \biggr )\mathrm{d}\tau \\&=\int _{0}^1 1 \mathrm{d}\tau \\&=1, \end{aligned}$$

which completes the proof. $\square$

Appendix C: Non-independence result for $k^{(n)}=\varvec{\varTheta }(n)$

Theorem 3

Consider an n-server system in the limited fork–join model with $k^{(n)}=\varTheta (n)$, job arrival rate $\varLambda ^{(n)}=n\lambda /k^{(n)}$, and exponentially distributed service times with rate $\mu$. Let $\pi ^{(n,2)}$ denote the joint distribution of the steady-state queue lengths for any two queues in the n-server system. Let $\hat{\pi }^{(2)}$ denote the joint distribution of the steady-state queue lengths of two independent M/M/1 queues, each with load $\rho$. Then, there exist $\epsilon >0$ and $n_0>0$ such that, for any $n>n_0$, $d_{TV}\bigl (\pi ^{(n,2)}, \hat{\pi }^{(2)}\bigr )>\epsilon$.

Proof

We assume that $k^{(n)}= p n$ for a constant p with $0<p\le 1$. Then the job arrival rate is given by $\varLambda ^{(n)}= \lambda /p$, which is a constant, so we rewrite $\varLambda ^{(n)}$ as $\varLambda$ for conciseness.

Let $\epsilon =\frac{p\lambda (1-\rho )^2}{2(11\varLambda +8\mu )}$. We will specify $n_0$ later. Suppose by contradiction that $d_{TV}\bigl (\pi ^{(n,2)},\hat{\pi }^{(2)}\bigr )\le \epsilon$ for all $n>n_0$. We will show that this assumption contradicts the balance equations of the first two queues in the limited fork–join system with n servers.

We first write out the balance equations for the Markov chain formed by the queue lengths of the first two queues. Consider a job arrival to this n-server system. Let $p_0^{(n)}$ be the probability that no task arrives to the first two queues, and $p_1^{(n)}$ be the probability that exactly one task arrives to the first two queues. Let $p_2^{(n)}=1-p_0^{(n)}-p_1^{(n)}$ be the probability that two tasks arrive to the first two queues. We can compute these probabilities as follows:

$$\begin{aligned} p_0^{(n)}&= \left( {\begin{array}{c}n-2\\ k\end{array}}\right) \Bigm / \left( {\begin{array}{c}n\\ k\end{array}}\right) \rightarrow p_0:=(1-p)^2 \quad \text {as }n\rightarrow \infty ,\\ p_1^{(n)}&= \frac{2\left( {\begin{array}{c}n-2\\ k-1\end{array}}\right) }{\left( {\begin{array}{c}n\\ k\end{array}}\right) } \rightarrow p_1:=2p(1-p) \quad \text {as }n \rightarrow \infty ,\\ p_2^{(n)}&= \frac{\left( {\begin{array}{c}n-2\\ k-2\end{array}}\right) }{\left( {\begin{array}{c}n\\ k\end{array}}\right) } \rightarrow p_2:=p^2 \quad \text {as }n \rightarrow \infty . \end{aligned}$$

Recall that the joint distribution of the steady-state queue lengths of the first two queues is $\pi ^{(n,2)}$. Then, the balance equation of the first two queues for the state (1, 1) can be written as

$$\begin{aligned} 0&=\pi ^{(n,2)}(1,1) \cdot (p_1^{(n)}\varLambda +p_2^{(n)}\varLambda +2\mu )\nonumber \\&\quad -\biggl (\frac{1}{2}\pi ^{(n,2)}(0,1)p_1^{(n)}\varLambda +\frac{1}{2}\pi ^{(n,2)}(1,0)p_1^{(n)}\varLambda \nonumber \\&\quad +\pi ^{(n,2)}(0,0)p_2^{(n)}\varLambda +\pi ^{(n,2)}(1,2)\mu +\pi ^{(n,2)}(2,1)\mu \biggr ). \end{aligned}$$

(36)

Let the right-hand side of (36) be denoted by $\mathcal {R}(\pi ^{(n,2)})$. Let

$$\begin{aligned} a_1&=(p_1^{(n)}-p_1)\varLambda \biggl (\pi ^{(n,2)}(1,1)-\frac{1}{2}\pi ^{(n,2)}(0,1)-\frac{1}{2}\pi ^{(n,2)}(1,0)\biggr )\\&\quad +(p_2^{(n)}-p_2)\varLambda \biggl (\pi ^{(n,2)}(1,1)-\frac{1}{2}\pi ^{(n,2)}(0,1)\\&\quad -\frac{1}{2}\pi ^{(n,2)}(1,0)-\pi ^{(n,2)}(0,0)\biggr ),\\ a_2&=(\pi ^{(n,2)}(1,1)-\hat{\pi }^{(2)}(1,1))(p_1\varLambda +p_2\varLambda +2\mu )\\&\quad -\frac{1}{2}(\pi ^{(n,2)}(0,1)-\hat{\pi }^{(2)}(0,1)+\pi ^{(n,2)}(1,0)-\hat{\pi }^{(2)}(1,0))p_1\varLambda \\&\quad -(\pi ^{(n,2)}(0,0)-\hat{\pi }^{(2)}(0,0))p_2\varLambda \\&\quad -(\pi ^{(n,2)}(1,2)-\hat{\pi }^{(2)}(1,2)+\pi ^{(n,2)}(2,1)-\hat{\pi }^{(2)}(2,1))\mu . \end{aligned}$$

Then since $\hat{\pi }^{(2)}(q_1,q_2)=(1-\rho )^2\rho ^{q_1+q_2}$ for any $(q_1,q_2)\in \mathbb {Z}_+^2$,

$$\begin{aligned} \mathcal {R}(\pi ^{(n,2)})&=a_1+a_2+\hat{\pi }^{(2)}(1,1) \cdot (p_1\varLambda +p_2\varLambda +2\mu )\nonumber \\&\quad -\biggl (\frac{1}{2}\hat{\pi }^{(2)}(0,1)p_1\varLambda +\frac{1}{2}\hat{\pi }^{(2)}(1,0)p_1\varLambda \nonumber \\&\quad +\hat{\pi }^{(2)}(0,0)p_2\varLambda +\hat{\pi }^{(2)}(1,2)\mu +\hat{\pi }^{(2)}(2,1)\mu \biggr )\nonumber \\&=a_1+a_2-p \lambda (1-\rho )^4. \end{aligned}$$

(37)

We choose $n_0$ such that, for any $n> n_0$, $|p_1^{(n)}-p_1|\le \epsilon$ and $|p_2^{(n)}-p_2|\le \epsilon$. Then, it is not hard to see that $|a_1|\le 3\varLambda \epsilon$. By the assumption that $d_{TV}\bigl (\pi ^{(n,2)},\hat{\pi }^{(2)}\bigr )\le \epsilon$, we have that $|a_2|\le 8(\varLambda +\mu ) \epsilon$. By the choice of $\epsilon$, $|a_1+a_2|\le (11\varLambda +8\mu )\epsilon =\frac{1}{2}p\lambda (1-\rho )^2$. Therefore, $\mathcal {R}(\pi ^{(n,2)})<0$ by (37), which contradicts the balance equation (36). This completes the proof of Theorem 3. $\square$

Appendix D: Definition and some properties of association

Definition 1

(Association [9]) We say random variables $X_1$, $X_2,\dots ,X_m$ are associated if for all (entrywisely) nondecreasing functions f and g,

$$\begin{aligned}&\mathbb {E}[f(X_1,X_2,\dots ,X_m)g(X_1,X_2,\dots ,X_m)] \nonumber \\&\quad \ge \mathbb {E}[f(X_1,X_2,\dots ,X_m)]\mathbb {E}[g(X_1,X_2,\dots ,X_m)]. \end{aligned}$$

(38)

Lemma 4

[9] Associated random variables have the following properties:

• (P1) Nondecreasing functions of associated random variables are associated.

• (P2) If two sets of associated random variables are independent of one another, then their union is a set of associated random variables.

• (P3) If a sequence of random vectors $\varvec{X}(u)\Rightarrow \varvec{X}$ as $u\rightarrow \infty$ and, for each u, the entries of $\varvec{X}(u)$ are associated, then the entries of $\varvec{X}$ are associated.