Construction of Tight Frames on Graphs and Application to Denoising

Abstract

Given a neighborhood graph representation of a finite set of points $x_i\in \mathbb {R}^d,i=1,\ldots ,n,$ we construct a frame (redundant dictionary) for the space of real-valued functions defined on the graph. This frame is adapted to the underlying geometrical structure of the x _i, has finitely many elements, and these elements are localized in frequency as well as in space. This construction follows the ideas of Hammond et al. (Appl Comput Harmon Anal 30:129–150, 2011), with the key point that we construct a tight (or Parseval) frame. This means we have a very simple, explicit reconstruction formula for every function f defined on the graph from the coefficients given by its scalar product with the frame elements. We use this representation in the setting of denoising where we are given noisy observations of a function f defined on the graph. By applying a thresholding method to the coefficients in the reconstruction formula, we define an estimate of f whose risk satisfies a tight oracle inequality.

Appendix

1.1 Proof of Theorem 3

Theorem 3 states a oracle-type inequality which captures the relation of soft thresholding estimators $\hat {f}_{S_s}=\sum _{k,l} S_s \left (\left \langle \,y,\varPsi _{kl} \right \rangle ,t_{kl}\right ) \varPsi _{kl}$ defined in (20.16) to the collection of keep-or-kill estimators on a Parseval frame. This result is known in the literature (see Candès 2006, Section 9), but we provide a short self-contained proof for completeness, modulo a technical result from Donoho and Johnstone (1994) for soft thresholding of a single one-dimensional Gaussian variable, which is basic for the Proof of Theorem 3.

Lemma 1

For 0 ≤ δ ≤ 1∕2, $t=\sqrt {2\log (\delta ^{-1})}$ and $X\sim \mathscr {N}(\mu ,1)$

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} {{\mathbf{E}}_{X}\left({\left(S_s(X,t)-\mu\right)^2}\right)}&\displaystyle \leq &\displaystyle (2\log ( \delta^{-1})+1)(\delta+\min(1,\mu^2) )\\ &\displaystyle =&\displaystyle (t^2+1)\left( \exp \left(-\frac{t^2}{2}\right) +\min(1,\mu^2) \right). \end{array} \end{aligned} $$

(20.23)

The proof of this lemma can be found in appendix 1 of Donoho and Johnstone (1994). Now we are able to prove Theorem 3.

Proof

First note that for y = τx, τ > 0, we have

$$\displaystyle \begin{aligned} S_s(y,u)=\tau S_s\left(x, \frac{u}{\tau}\right). \end{aligned} $$

(20.24)

Secondly we remark that

$$\displaystyle \begin{aligned} \frac{\left\langle\,y,\varPsi_{kl} \right\rangle}{\sigma\left\lVert \varPsi_{kl} \right\rVert}\sim\mathscr{N}\left(\frac{a_{kl}}{\sigma\left\lVert \varPsi_{kl} \right\rVert},1\right). \end{aligned} $$

(20.25)

Considering now the risk of the soft thresholding estimator $\hat {f}_{S_s}$ we get

$$\displaystyle \begin{aligned} \begin{array}{rcl} {\mathbf{E}\left(\left\lVert \hat{f}_{S_{s}}-f \right\rVert^2\right)} &\displaystyle =&\displaystyle {\mathbf{E}\left(\left\lVert \sum_{k,l} \left({S_{s}}\left(\left\langle\,y,\varPsi_{kl} \right\rangle, t_{kl}\right)-a_{kl}\right) \varPsi_{kl} \right\rVert^2\right)}\\ &\displaystyle \leq &\displaystyle {\mathbf{E}\left(\sum_{k,l}\left({S_{s}} \left(\left\langle\,y,\varPsi_{kl} \right\rangle, t_{kl} \right)-a_{kl}\right)^2\right)}\\ &\displaystyle =&\displaystyle \sum_{k,l} {\mathbf{E}\left(\left({S_{s}}\left(\left\langle\,y,\varPsi_{kl} \right\rangle, t_{kl}\right)-a_{kl}\right)^2\right)}. \end{array} \end{aligned} $$

(20.26)

by using inequality (20.6). By applying (20.24) and then (20.23) with $t=\sqrt {2\log (n)}$ it follows that

$$\displaystyle \begin{aligned} \begin{array}{rcl} &\displaystyle &\displaystyle {\mathbf{E}\left(\left\lVert \hat{f}_{S_s}-f \right\rVert^2\right)} \leq \sum_{k,l} \sigma^2\left\lVert \varPsi_{kl} \right\rVert^2 {\mathbf{E}\left(\left({S_{s}}\left(\frac{\left\langle\,y,\varPsi_{kl} \right\rangle}{\sigma\left\lVert \varPsi_{kl} \right\rVert}, \sqrt{2\log(n)}\right)-\frac{a_{kl}}{\sigma\left\lVert \varPsi_{kl} \right\rVert}\right)^2\right)}\\ &\displaystyle &\displaystyle \quad \leq \sum_{k,l} \sigma^2\left\lVert \varPsi_{kl} \right\rVert^2 (2\log(n)+1)\left(\exp \left(-\frac{2\log(n)}{2}\right) +\min \left(1,\frac{a_{kl}^2}{\sigma^2\left\lVert \varPsi_{kl} \right\rVert^2}\right) \right)\\ &\displaystyle &\displaystyle \quad = \sum_{k,l} (2\log(n)+1)\left( \frac{1}{n}\sigma^2\left\lVert \varPsi_{kl} \right\rVert^2 +\min \left(\sigma^2\left\lVert \varPsi_{kl} \right\rVert^2,a_{kl}^2\right)\right )\\ &\displaystyle &\displaystyle \quad = (2\log(n)+1) \left( \frac{1}{n} \sum_{k,l} \sigma^2\left\lVert \varPsi_{kl} \right\rVert^2 +\sum_{k,l} \min \left(\sigma^2\left\lVert \varPsi_{kl} \right\rVert^2,a_{kl}^2\right) \right). \end{array} \end{aligned} $$

(20.27)

Recalling the Parseval frame property $\sum _{k,l} \left \lVert \varPsi _{kl} \right \rVert ^2=n$, we finally obtain

$$\displaystyle \begin{aligned} \begin{array}{rcl} {\mathbf{E}\left(\left\lVert \hat{f}_{S_s}-f \right\rVert^2\right)}&\displaystyle \leq &\displaystyle (2\log(n)+1) \left( \frac{1}{n}n \sigma^2 +\sum_{k,l} \min \left(\sigma^2\left\lVert \varPsi_{kl} \right\rVert^2,a_{kl}^2\right) \right)\\ &\displaystyle =&\displaystyle (2\log(n)+1) \left( \sigma^2 +\sum_{k,l} \min \left(\sigma^2\left\lVert \varPsi_{kl} \right\rVert^2,a_{kl}^2\right) \right). \end{array} \end{aligned} $$

(20.28)

where we recognize the upper bound $\sum _{k,l} \min \left (\sigma ^2\left \lVert \varPsi _{kl} \right \rVert ^2,a_{kl}^2 \right )=OB(f)$ for the oracle. □