Blind three dimensional deconvolution via convex optimization

Abstract

In this paper we discuss recovering two signals from their convolution in 3 dimensions. One of the signals is assumed to lie in a known subspace and the other one is assumed to be sparse. Various applications such as super resolution, radar imaging, and direction of arrival estimation can be described in this framework. We introduce a method to estimate parameters of a signal in a low-dimensional subspace which is convolved with another signal comprised of some impulses in time domain. We transform the problem to a convex optimization in the form of a positive semi-definite program using lifting and the atomic norm. We demonstrate that unknown parameters can be recovered by lowpass observations. Numerical simulations show excellent performance of the proposed method.

Appendices

Appendix A

1.1 Proof of the primary problem SDP

By defining one atom in the form of $\varvec{A}\left( {\tau ,\varvec{b}} \right) = \varvec{c}\left( \tau \right)\varvec{b}^{\varvec{*}}$ with $\tau_{i} \in \left[ {0,1} \right)$, $\varvec{b} \in {\mathbb{R}}^{L}$, $\left\| \varvec{b} \right\|_{2} = 1$ and $\varvec{Z} = \mathop \sum \limits_{k} a_{k} \varvec{A}\left( {\varvec{\tau}_{k} , \varvec{b}_{k} } \right), a_{k} \ge 0$. We have:

$$ \left\| \varvec{Z} \right\|_{{\mathcal{A}}} = \mathop { \inf }\limits_{{\varvec{T },\varvec{ W} \in {\mathbb{C}}^{L \times L} }} \left\{ {\frac{1}{2}{\text{Tr}}\left( {{\text{S}}\left( {\mathbf{T}} \right)} \right) + \frac{1}{2}{\text{Tr}}\left( \varvec{W} \right)|\left[ {\begin{array}{*{20}c} {{\text{S}}\left( {\mathbf{T}} \right)} & \varvec{Z} \\ {\varvec{Z}^{H} } & \varvec{W} \\ \end{array} } \right]{ \succcurlyeq }0} \right\}. $$

Proof

We show the right hand side of the above equation with $SDP\left( \varvec{Z} \right)$. assuming $\varvec{Z} = \mathop \sum \limits_{k} a_{k} \varvec{c}\left( {\varvec{\tau}_{k} } \right)\varvec{b}_{k}^{\varvec{*}} , a_{k} \ge 0$ and $S\left( \varvec{T} \right) = \mathop \sum \limits_{k} a_{k} \varvec{c}\left( {\varvec{\tau}_{k} } \right)\varvec{c}\left( {\varvec{\tau}_{k} } \right)^{*}$ we have:

$$ \left[ {\begin{array}{*{20}c} {{\text{S}}\left( {\mathbf{T}} \right)} & \varvec{Z} \\ {\varvec{Z}^{\varvec{*}} } & \varvec{W} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {\mathop \sum \limits_{k} a_{k} \varvec{c}\left( {\varvec{\tau}_{k} } \right)\varvec{c}\left( {\varvec{\tau}_{k} } \right)^{*} } & {\mathop \sum \limits_{k} a_{k} \varvec{c}\left( {\varvec{\tau}_{k} } \right)\varvec{b}_{k}^{\varvec{*}} } \\ {\mathop \sum \limits_{k} a_{k} \varvec{c}\left( {\varvec{\tau}_{k} } \right)^{*} \varvec{b}_{k} } & {\mathop \sum \limits_{k} a_{k} \varvec{b}_{k} \varvec{b}_{k}^{\varvec{*}} } \\ \end{array} } \right] = \mathop \sum \limits_{k} a_{k} \left[ {\begin{array}{*{20}c} {\varvec{c}\left( {\varvec{\tau}_{k} } \right)} \\ {\varvec{b}_{k} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\varvec{c}\left( {\varvec{\tau}_{k} } \right)^{*} } & {\varvec{b}_{k}^{\varvec{*}} } \\ \end{array} } \right]{ \succcurlyeq }0 $$

Therefore:

$$ \frac{1}{2}{\text{Tr}}\left( {{\text{S}}\left( {\mathbf{T}} \right)} \right) + \frac{1}{2}{\text{Tr}}\left( \varvec{W} \right) = \frac{1}{2}\mathop \sum \limits_{k} a_{k} + \frac{1}{2}\mathop \sum \limits_{k} a_{k} = \left\| \varvec{Z} \right\|_{{\mathcal{A}}} $$

Which is: $SDP\left( \varvec{Z} \right) \le \left\| \varvec{Z} \right\|_{{\mathcal{A}}}$.□

We assume $\left[ {\begin{array}{*{20}c} {{\text{S}}\left( {\mathbf{T}} \right)} & \varvec{Z} \\ {\varvec{Z}^{\varvec{*}} } & \varvec{W} \\ \end{array} } \right]{ \succcurlyeq }0$, while $S\left( \varvec{T} \right) = \varvec{VDV}^{\varvec{*}}$ and $\varvec{D} = {\text{diag}}\left( {d_{i} } \right), d_{i} > 0$. we can see that $\varvec{Z}$ is in the range space of $\varvec{V}$. So $\varvec{Z} = \varvec{VB}$ and recall that $\varvec{b}_{k}$ is the kth column of $\varvec{B}^{\varvec{T}}$. Also $\varvec{W}$ is positive semi-definite, therefore $\varvec{W} = \varvec{B}^{\varvec{*}} \varvec{EB}$ where $\varvec{E}$ a is positive semi-definite matrix too. We have:

$$ \left[ {\begin{array}{*{20}c} {{\text{S}}\left( {\mathbf{T}} \right)} & \varvec{Z} \\ {\varvec{Z}^{\varvec{*}} } & \varvec{W} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} \varvec{V} & {\mathbf{0}} \\ {\mathbf{0}} & {\varvec{B}^{\varvec{*}} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} \varvec{D} & \varvec{I} \\ \varvec{I} & \varvec{E} \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\varvec{V}^{\varvec{*}} } & {\mathbf{0}} \\ {\mathbf{0}} & \varvec{B} \\ \end{array} } \right] \succ 0 $$

Thus:

$$ \begin{aligned} \frac{1}{2}{\text{Tr}}\left( {{\text{S}}\left( {\mathbf{T}} \right)} \right) + \frac{1}{2}{\text{Tr}}\left( \varvec{W} \right) & = \frac{1}{2}{\text{Tr}}\left( \varvec{D} \right) + \frac{1}{2}{\text{Tr}}\left( \varvec{W} \right) \ge \sqrt {{\text{Tr}}\left( \varvec{D} \right) \cdot {\text{Tr}}\left( \varvec{W} \right)} \ge \sqrt {\left( {\mathop \sum \limits_{k} d_{k} } \right) \cdot \left( {\mathop \sum \limits_{k} d_{k}^{ - 1} \left\| {\varvec{b}_{k} } \right\|^{2} } \right)} \\ & \ge \mathop \sum \limits_{k} \left\| {\varvec{b}_{k} } \right\| \ge \left\| \varvec{Z} \right\|_{{\mathcal{A}}} \\ \end{aligned} $$

Therefore, $\left\| \varvec{Z} \right\|_{{\mathcal{A}}} \le SDP\left( \varvec{Z} \right)$ and we conclude that $\left\| \varvec{Z} \right\|_{{\mathcal{A}}} = SDP\left( \varvec{Z} \right)$.

However, in contrary to the proposed SDP in Tang et al. (2013), since $S\left( \varvec{T} \right)$ might not have a description as $\varvec{VDV}^{\varvec{*}}$, we cannot guarantee that $\left\| \varvec{Z} \right\|_{{\mathcal{A}}} = SDP\left( \varvec{Z} \right)$.

Appendix B

2.1 Proof of Proposition 1

Fist we show that each $\varvec{q}$ satisfying (15) and (16) is dual feasible. We have:

$$ \begin{aligned} \left\| {\varvec{Z}_{{\mathbf{0}}} } \right\|_{{\mathcal{A}}} & \ge \left\| {\varvec{Z}_{{\mathbf{0}}} } \right\|_{{\mathcal{A}}} \left\| {\chi^{*} \left( \varvec{q} \right)} \right\|_{{\mathcal{A}}}^{{ \star }} \ge \left\langle {\chi^{*} \left( \varvec{q} \right)\varvec{,Z}_{{\mathbf{0}}} } \right\rangle _{{\mathbb{R}}} \\ & = \mathop \sum \limits_{k = 1}^{K} {\text{Re}}\left( {a_{k}^{*} {\text{trace}}\left( {\varvec{c}\left( {\varvec{\tau}_{k} } \right)\varvec{h}_{k}^{{\mathbf{H}}} \chi^{*} \left( \varvec{q} \right)} \right)} \right) = \mathop \sum \limits_{k = 1}^{K} {\text{Re}}\left( {a_{k}^{*} \varvec{h}_{k}^{{\mathbf{H}}} \varvec{Q}\left( {\varvec{\tau}_{k} } \right)} \right) \\ & = \mathop \sum \limits_{k = 1}^{K} {\text{Re}}\left( {a_{k}^{*} {\text{sign}}\left( {a_{k} } \right)} \right) = \mathop \sum \limits_{k = 1}^{K} \left| {a_{k} } \right| \ge \left\| {\varvec{Z}_{{\mathbf{0}}} } \right\|_{{\mathcal{A}}} \\ \end{aligned} $$

Therefore, $\left\langle {\chi^{*} \left( \varvec{q} \right) ,\varvec{Z}_{{\mathbf{0}}} } \right\rangle _{{\mathbb{R}}} = \left\| {\varvec{Z}_{{\mathbf{0}}} } \right\|_{{\mathcal{A}}}$ and based on strong duality $\varvec{q}$ is the optimum dual for the problem which $\varvec{Z}_{{\mathbf{0}}}$ is its primary optimum.

To show uniqueness of the solution, assume $\tilde{\varvec{Z}} = \sum\nolimits_{k} {\tilde{a}_{k} \varvec{c}\left( {\tilde{\varvec{\tau }}_{k} } \right)\tilde{\varvec{h}}_{k}^{\text{T}} }$ is another solution of the problem and $\varvec{Z}_{{\mathbf{0}}}$ and $\tilde{\varvec{Z}}$ have the same support set T. Set of atoms in ${\mathbf{\mathcal{T}}}$ are independent so they should be equal. Therefore we have:

$$ \left\langle {\chi^{*} \left( \varvec{q} \right),\tilde{\varvec{Z}}} \right\rangle_{{\mathbb{R}}} = \mathop \sum \limits_{{\varvec{\tau}_{k} \in {\mathbf{\mathcal{T}}}}} {\text{Re}}\left( {\tilde{a}_{k}^{*} \tilde{\varvec{h}}_{k}^{{\mathbf{H}}} \varvec{Q}\left( {\tilde{\varvec{\tau }}_{k} } \right)} \right) + \mathop \sum \limits_{{\varvec{\tau}_{j} \notin {\mathbf{\mathcal{T}}}}} {\text{Re}}\left( {\tilde{a}_{j}^{*} \tilde{\varvec{h}}_{j}^{{\mathbf{H}}} \varvec{Q}\left( {\tilde{\varvec{\tau }}_{j} } \right)} \right) = \mathop \sum \limits_{{\varvec{\tau}_{k} \in {\mathbf{\mathcal{T}}}}} \left| {\tilde{a}_{k} } \right| + \mathop \sum \limits_{{\varvec{\tau}_{j} \notin {\mathbf{\mathcal{T}}}}} \left| {\tilde{a}_{j} } \right| = \left\| {\tilde{\varvec{Z}}} \right\|_{{\mathcal{A}}} $$

Which contradicts the strong duality. Therefore, the solution of (8) is unique.

Appendix C

3.1 Proof of Lemma 2

Assume that $\left( {2\pi n_{1}\upkappa} \right)^{2} \le 13$ and $B$ is defined as:

$$ \begin{aligned} B & : = \left\| {\varvec{S}_{\varvec{n}} } \right\| = \frac{1}{{M^{3} }}\left\| {s_{{n_{1} }} s_{{n_{2} }} s_{{n_{3} }} \left( {\varvec{\nu}_{\varvec{n}}\varvec{\nu}_{\varvec{n}}^{\text{H}} } \right) \otimes \left( {\varvec{b}_{\varvec{n}} \varvec{b}_{\varvec{n}}^{\text{H}} - \varvec{I}_{L} } \right)} \right\| \\ & \le \frac{1}{{M^{3} }}\mathop {\hbox{max} }\limits_{{n_{1} }} \left| {s_{{n_{1} }} } \right|. \mathop {\hbox{max} }\limits_{{n_{2} }} \left| {s_{{n_{2} }} } \right|. \mathop {\hbox{max} }\limits_{{n_{3} }} \left| {s_{{n_{3} }} } \right|\left\| {\varvec{\nu}_{\varvec{n}} } \right\|_{2}^{2} \hbox{max} \left\{ {\left\| {\varvec{b}_{\varvec{n}} } \right\|_{2}^{2} ,\left\| {\varvec{I}_{L} } \right\|} \right\} \\ & \le \frac{1}{{M^{3} }}\left( {K + 3K\left( {2\pi n_{1}\upkappa} \right)^{2} } \right)\hbox{max} \left\{ {\mu L,1} \right\} \le \frac{1}{{M^{3} }}K\left( {40} \right)\mu L = \frac{40\mu KL}{{M^{3} }} \\ \end{aligned} $$

Also:

$$ \begin{aligned} \sigma^{2} & : = \left\| {\mathop \sum \limits_{{\varvec{n} \in J}} {\mathbb{E}}\left[ {\varvec{S}_{\varvec{n}} \varvec{S}_{\varvec{n}}^{\text{H}} } \right]} \right\| \\ & = \frac{1}{{M^{3} }}\left\| {\mathop \sum \limits_{{\varvec{n} \in J}} {\mathbb{E}}\left\{ {s_{{n_{1} }}^{2} s_{{n_{2} }}^{2} s_{{n_{3} }}^{2} \left[ {\left( {\varvec{\nu}_{\varvec{n}}\varvec{\nu}_{\varvec{n}}^{\text{H}} } \right) \otimes \left( {\varvec{b}_{\varvec{n}} \varvec{b}_{\varvec{n}}^{H} - \varvec{I}_{L} } \right)} \right]\left[ {\left( {\varvec{\nu}_{\varvec{n}}\varvec{\nu}_{\varvec{n}}^{\text{H}} } \right) \otimes \left( {\varvec{b}_{\varvec{n}} \varvec{b}_{\varvec{n}}^{H} - \varvec{I}_{L} } \right)} \right]^{\text{H}} } \right\}} \right\| \\ & \le \frac{1}{{M^{6} }}40\mu KL\left\| {\mathop \sum \limits_{{\varvec{n} \in J}} s_{{n_{1} }}^{2} s_{{n_{2} }}^{2} s_{{n_{3} }}^{2} \left( {\varvec{\nu}_{\varvec{n}}\varvec{\nu}_{\varvec{n}}^{\text{H}} } \right) \otimes \varvec{I}_{L} } \right\| \\ & \le \frac{40\mu KL}{{M^{3} }}\left\| {\frac{1}{{M^{3} }}\mathop \sum \limits_{{\varvec{n} \in J}} s_{{n_{1} }} s_{{n_{2} }} s_{{n_{3} }} \left( {\varvec{\nu}_{\varvec{n}}\varvec{\nu}_{\varvec{n}}^{\text{H}} } \right) \otimes \varvec{I}_{L} } \right\| \le \frac{40\mu KL}{{M^{3} }}\left\| {{\bar{\boldsymbol{\varGamma }}}} \right\| \le \frac{42\mu KL}{{M^{3} }} \\ \end{aligned} $$

Now, Bernstein’s non-commutative inequality gives:

$$ {\mathbb{P}}\left\{ {\left\| {{\varvec{\Gamma}} - {\bar{\boldsymbol{\varGamma }}}} \right\| >\upvarepsilon_{1} } \right\} \le 2d{ \exp }\left( {\frac{{ - t^{2} /2}}{{\sigma^{2} + Bt/3}}} \right) $$

will give:

$$ {\mathbb{P}}\left\{ {\left\| {{\varvec{\Gamma}} - {\bar{\boldsymbol{\varGamma }}}} \right\| >\upvarepsilon_{1} } \right\} \le 8KL{ \exp }\left( {\frac{{ -\upvarepsilon_{1}^{2} /2}}{{\frac{42\mu KL}{{M^{3} }} + Bt/3}}} \right) \le 8KL{ \exp }\left( {\frac{{ -\upvarepsilon_{1}^{2} M^{3} }}{100\mu KL}} \right) \le \delta_{1} $$

Therefore, if $M^{3} \ge \frac{100\mu KL}{{\upvarepsilon_{1}^{2} }}{ \log }\left( {\frac{8KL}{{\delta_{1} }}} \right)$ then $\left\| {{\varvec{\Gamma}} - {\bar{\boldsymbol{\varGamma }}}} \right\| \le\upvarepsilon_{1}$ with probability at least $1 - \delta_{1}$.

Appendix D

4.1 Proof of Lemma 6

We use the matrix Bernstein inequality to prove the Lemma. First define:

$$ \begin{aligned} \varvec{Y}_{\varvec{n}}^{{\left( \varvec{i} \right)}} & = \frac{1}{{M^{3} }}\left( {j2\pi\upkappa} \right)^{{i_{1} + i_{2} + i_{3} }} n_{1}^{{i_{1} }} n_{2}^{{i_{2} }} n_{3}^{{i_{3} }} s_{{n_{1} }} s_{{n_{2} }} s_{{n_{3} }} e^{{j2\pi\varvec{\tau}^{\text{T}} \varvec{n}}}\varvec{\nu}_{\varvec{n}} \otimes \left( {\varvec{b}_{\varvec{n}} \varvec{b}_{\varvec{n}}^{\text{H}} - \varvec{I}_{L} } \right) \\ & & \quad \varvec{E}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right) - \bar{\varvec{E}}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right) = \mathop \sum \limits_{{\varvec{n} \in J}} \varvec{Y}_{\varvec{n}}^{{\left( \varvec{i} \right)}} \\ \end{aligned} $$

(D.1)

The following elementary bounds (Tang et al. 2013; Candes and Fernandez-Granda 2014) are useful.

$$ \begin{aligned} & \left\| {s_{{n_{1} }} } \right\|_{\infty } \le 1 \\ & \left| {2\pi n\upkappa} \right| \le 4\quad {\text{when}}\quad M \ge 2 \\ & \left\| {\varvec{\nu}_{\varvec{n}} } \right\|_{2}^{2} \le 40K\quad {\text{when}}\quad M \ge 4 \\ \end{aligned} $$

Thus, we have

$$ \begin{aligned} \left\| {\varvec{Y}_{\varvec{n}}^{{\left( \varvec{i} \right)}} } \right\| & = \left\| {\varvec{Y}_{{n_{1} ,n_{2} ,n_{3} }}^{{\left( {i_{1} ,i_{2} ,i_{3} } \right)}} } \right\| \le \frac{1}{{M^{3} }}\left( 4 \right)^{{i_{1} + i_{2} + i_{3} }} \left\| {\varvec{\nu}_{\varvec{n}} } \right\|_{2} \left\| {\varvec{b}_{\varvec{n}} \varvec{b}_{\varvec{n}}^{H} - \varvec{I}_{L} } \right\| \le \frac{1}{{M^{3} }}\left( 4 \right)^{{i_{1} + i_{2} + i_{3} }} \sqrt {40K } max\left\{ {\mu L ,1} \right\} \\ & \le \frac{{2^{{3 + 2\left( {i_{1} + i_{2} + i_{3} } \right)}} \sqrt {K } \mu L}}{{M^{3} }}: = R \\ \end{aligned} $$

(D.2)

Also we can write

$$ \begin{aligned} \left\| {\mathop \sum \limits_{{\varvec{n} \in J}} {\mathbb{E}}\left[ {\varvec{Y}_{\varvec{n}}^{{\left( \varvec{i} \right)^{\text{H}} }} \varvec{Y}_{\varvec{n}}^{{\left( \varvec{i} \right)}} } \right]} \right\| & = \frac{1}{{M^{6} }}\left\| {\mathop \sum \limits_{{\varvec{n} \in J}} {\mathbb{E}}\left[ {\mathop \prod \limits_{k = 1}^{3} \left( {\left| {s_{{n_{k} }} } \right|^{2} \left( {\left| {2\pi n_{k}\upkappa} \right|} \right)^{{2i_{k} }} } \right)\left( {\varvec{\nu}_{\varvec{n}}^{\text{H}} \otimes \left( {\varvec{b}_{\varvec{n}} \varvec{b}_{\varvec{n}}^{\text{H}} - \varvec{I}_{L} } \right)} \right)\left( {\varvec{\nu}_{\varvec{n}} \otimes \left( {\varvec{b}_{\varvec{n}} \varvec{b}_{\varvec{n}}^{\text{H}} - \varvec{I}_{L} } \right)} \right)} \right]} \right\| \\ & \le \frac{1}{{M^{6} }}\left( 4 \right)^{{2\left( {i_{1} + i_{2} + i_{3} } \right)}} \left\| {\mathop \sum \limits_{{\varvec{n} \in J}} \left\| {\varvec{\nu}_{\varvec{n}} } \right\|_{2}^{2} {\mathbb{E}}\left[ {\left( {\varvec{b}_{\varvec{n}} \varvec{b}_{\varvec{n}}^{\text{H}} - \varvec{I}_{L} } \right)^{2} } \right]} \right\| \le \frac{{\left( 4 \right)^{{2\left( {i_{1} + i_{2} + i_{3} } \right)}} \mu L}}{{M^{6} }}\left\| {\mathop \sum \limits_{{\varvec{n} \in J}} \left\| {\varvec{\nu}_{\varvec{n}} } \right\|_{2}^{2} \varvec{I}_{L} } \right\| \\ & = \frac{{\left( 4 \right)^{{2\left( {i_{1} + i_{2} + i_{3} } \right)}} \mu L}}{{M^{6} }}\mathop \sum \limits_{{\varvec{n} \in J}} \left\| {\varvec{\nu}_{\varvec{n}} } \right\|_{2}^{2} \le \frac{{\left( 4 \right)^{{2\left( {i_{1} + i_{2} + i_{3} } \right)}} \mu L}}{{M^{6} }}\left( {4M + 1} \right)^{3} \left( {40K} \right) \\ & \le \frac{{5000 \times \left( 4 \right)^{{2\left( {i_{1} + i_{2} + i_{3} } \right)}} \mu KL}}{{M^{3} }}: = \sigma^{2} \\ \end{aligned} $$

(D.3)

Applying matrix Bernstein inequality (Rectangular Case) (Tropp 2012; Yang et al. 2016) and using $\left( {D.2} \right)$ and $\left( {D.3} \right)$, we can see that for a fixed $\left( {i_{1} ,i_{2} ,i_{3} } \right)$,

$$ {\mathbb{P}}\left\{ {\left\| {\mathop \sum \limits_{{\varvec{n} \in J}} \varvec{Y}_{\varvec{n}}^{{\left( \varvec{i} \right)}} } \right\| >\upvarepsilon_{2} } \right\} \le \left( {4KL + L} \right){ \exp }\left( {\frac{{ - 3\upvarepsilon_{2}^{2} }}{{8\sigma^{2} }}} \right) $$

In order to make this failure probability less than $\delta_{2}$, we require that:

$$ \log \left( {\left( {4KL + L} \right) \cdot { \exp }\left( {\frac{{ - 3\upvarepsilon_{2}^{2} }}{{8\sigma^{2} }}} \right)} \right) \le \log \delta_{2} $$

Which results in the following bound on $M$

$$ M^{3} \ge \frac{{4000 \times 4^{{2\left( {i_{1} + i_{2} + i_{3} } \right)}} \mu LK}}{{3\upvarepsilon_{2}^{2} }}\log \left( {\frac{4KL + L}{{\delta_{2} }}} \right) $$

Using a union bound for $i_{1} ,i_{2} ,i_{3} = 0,1,2,3$, we have

$$ {\mathbb{P}}\left\{ {\left\| {\mathop \sum \limits_{{\varvec{n} \in J}} \varvec{Y}_{\varvec{n}}^{{\left( \varvec{i} \right)}} } \right\| >\upvarepsilon_{2} ,\quad i_{1} ,i_{2} ,i_{3} = 0, \ldots ,3} \right\} \le 4^{3} \delta_{2} $$

Provided that $M^{3} \ge \frac{{4000 \times 4^{{2\left( {i_{1} + i_{2} + i_{3} } \right)}} \mu LK}}{{3\upvarepsilon_{2}^{2} }}\log \left( {\frac{4KL + L}{{\delta_{2} }}} \right)$.

Appendix E

5.1 Proof of Lemma 7

According to Lemma 6, conditioned on the events $\upvarepsilon_{{1,\upvarepsilon_{1} }}$ with ${\upvarepsilon}_{1} \epsilon \left({0,\left. {\frac{1}{4} } \right]} \right.$ and

$$ \mathop {\bigcap }\limits_{{\varvec{\tau}_{d} \in\Omega _{grid} }} \left\{ {\left\| {\varvec{E}^{{\left( \varvec{i} \right)}} \left( {\varvec{\tau}_{d} } \right) - \bar{\varvec{E}}^{{\left( \varvec{i} \right)}} \left( {\varvec{\tau}_{d} } \right)} \right\| \le\upvarepsilon_{2} ,\quad i_{1} ,i_{2} ,i_{3} = 0,1,2,3 } \right\} $$

We have

$$ \begin{aligned} \left\| {\left( {\varvec{E}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right) - \bar{\varvec{E}}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right)} \right)^{\text{H}} \varvec{L}} \right\| & \le \left\| {\varvec{E}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right) - \bar{\varvec{E}}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right)} \right\|\left\| \varvec{L} \right\| \le \left\| {\varvec{E}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right) - \bar{\varvec{E}}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right)} \right\|\left\| {{\varvec{\Gamma}}^{ - 1} } \right\| \\ & \le\upvarepsilon_{2} \times 2\left\| {{\bar{\boldsymbol{\varGamma }}}^{ - 1} } \right\| \le 2.066\upvarepsilon_{2} \le 2.1\upvarepsilon_{2} \\ \end{aligned} $$

Where the second inequality uses the fact that $\varvec{L}$ is a submatrix of ${\varvec{\Gamma}}^{ - 1}$, and the third and fourth inequalities follow from Lemmas 4 and 1, respectively.

Using a union bound for $i_{1} ,i_{2} ,i_{3} = 0,1,2,3$, we have

$$ {\mathbb{P}}\left\{ {\mathop {\sup }\limits_{{\varvec{\tau}_{l} \in\Omega _{grid} }} \left\| {\left( {\varvec{E}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right) - \bar{\varvec{E}}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right)} \right)^{H} \varvec{L}} \right\| \le 2.1\upvarepsilon_{2} ,\quad i_{1} ,i_{2} ,i_{3} = 0, \ldots ,3} \right\} \le {\mathbb{P}}\left( {\varepsilon^{C}_{{1,\upvarepsilon_{1} }} } \right) + 64\delta_{2} \left| {\Omega _{grid} } \right|. $$

Appendix F

6.1 Proof of Lemma 9

There exist a numerical constant C such that

$$ \begin{aligned} \left\| {\varvec{E}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right)} \right\|_{F}^{2} & = \left\| {\varvec{E}^{{\left( {i_{1} ,i_{2} ,i_{3} } \right)}} \left(\varvec{\tau}\right)} \right\|_{F}^{2} = \left\| {\upkappa^{{i_{1} + i_{2} + i_{3} }} \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {K^{{\left( {i_{1} ,i_{2} ,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{1} } \right)^{*} } \\ \vdots \\ {K^{{\left( {i_{1} ,i_{2} ,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{K} } \right)^{*} } \\ \end{array} } \\ {\begin{array}{*{20}c} {\kappa K^{{\left( {i_{1} + 1,i_{2} ,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{1} } \right)^{*} } \\ \vdots \\ \begin{aligned} \kappa K^{{\left( {i_{1} + 1,i_{2} ,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{K} } \right)^{*} \hfill \\ \begin{array}{*{20}c} {\begin{array}{*{20}c} {\kappa K^{{\left( {i_{1} ,i_{2} + 1,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{1} } \right)^{*} } \\ \vdots \\ {\kappa K^{{\left( {i_{1} ,i_{2} + 1,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{K} } \right)^{*} } \\ \end{array} } \\ {\begin{array}{*{20}c} {\kappa K^{{\left( {i_{1} ,i_{2} ,i_{3} + 1} \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{1} } \right)^{*} } \\ \vdots \\ {\kappa K^{{\left( {i_{1} ,i_{2} ,i_{3} + 1} \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{K} } \right)^{*} } \\ \end{array} } \\ \end{array} \hfill \\ \end{aligned} \\ \end{array} } \\ \end{array} } \right] \otimes \varvec{I}_{L} } \right\|_{F}^{2} \\ & = L\left\| {\upkappa^{{i_{1} + i_{2} + i_{3} }} \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {K^{{\left( {i_{1} ,i_{2} ,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{1} } \right)^{*} } \\ \vdots \\ {K^{{\left( {i_{1} ,i_{2} ,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{K} } \right)^{*} } \\ \end{array} } \\ {\begin{array}{*{20}c} {\kappa K^{{\left( {i_{1} + 1,i_{2} ,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{1} } \right)^{*} } \\ \vdots \\ \begin{aligned} \kappa K^{{\left( {i_{1} + 1,i_{2} ,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{K} } \right)^{*} \hfill \\ \begin{array}{*{20}c} {\begin{array}{*{20}c} {\kappa K^{{\left( {i_{1} ,i_{2} + 1,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{1} } \right)^{*} } \\ \vdots \\ {\kappa K^{{\left( {i_{1} ,i_{2} + 1,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{K} } \right)^{*} } \\ \end{array} } \\ {\begin{array}{*{20}c} {\kappa K^{{\left( {i_{1} ,i_{2} ,i_{3} + 1} \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{1} } \right)^{*} } \\ \vdots \\ {\kappa K^{{\left( {i_{1} ,i_{2} ,i_{3} + 1} \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{K} } \right)^{*} } \\ \end{array} } \\ \end{array} \hfill \\ \end{aligned} \\ \end{array} } \\ \end{array} } \right]} \right\|_{2}^{2} \le CL \\ \end{aligned} $$

Where the inequality follows from proof of Lemma IV.9 in Tang et al. (2013). Conditioned on ${\varvec{\upvarepsilon}}_{{1,\upvarepsilon_{1} }}$ with ${\varvec{\upvarepsilon}}_{1} \epsilon \left({0,\frac{1}{4}} \right]$, if $\left\{ {\varvec{\tau}_{k} } \right\}$ are well separated to the sequence $\left\{ {K^{{\left( {i_{1} ,i_{2} ,i_{3} } \right)}} \left( {\varvec{\tau}-\varvec{\tau}_{k} } \right)} \right\}$ decreases, for some numerical constant C we have

$$ \left\| {\left( {\varvec{L} - \bar{\varvec{L}} \otimes \varvec{I}_{L} } \right)^{H} \bar{\varvec{E}}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right)} \right\|_{F}^{2} \le \left\| {\left( {\varvec{L} - \bar{\varvec{L}} \otimes \varvec{I}_{L} } \right)^{H} } \right\|^{2} \left\| {\bar{\varvec{E}}^{{\left( \varvec{i} \right)}} \left(\varvec{\tau}\right)} \right\|_{F}^{2} \le \left( {2 \times 1.034^{2}\upvarepsilon_{1} } \right)^{2} CL: = CL\upvarepsilon_{1}^{2} $$

Where the second inequality uses Lemmas 1 and 4 and the fact that $\varvec{L} - \bar{\varvec{L}} \otimes \varvec{I}_{L}$ is a submatrix of ${\varvec{\Gamma}}^{ - 1} - {\bar{\boldsymbol{\varGamma }}}^{ - 1}$.