Bidirectionally aligned sparse representation for single image super-resolution

Abstract

It has been demonstrated that the sparse representation based framework is one of the most popular and promising ways to handle the single image super-resolution (SISR) issue. However, due to the complexity of image degradation and inevitable existence of noise, the coding coefficients produced by imposing sparse prior only are not precise enough for faithful reconstructions. In order to overcome it, we present an improved SISR reconstruction method based on the proposed bidirectionally aligned sparse representation (BASR) model. In our model, the bidirectional similarities are first modeled and constructed to form a complementary pair of regularization terms. The raw sparse coefficients are additionally aligned to this pair of standards to restrain sparse coding noise and therefore result in better recoveries. On the basis of fast iterative shrinkage-thresholding algorithm, a well-designed mathematic implementation is introduced for solving the proposed BASR model efficiently. Thorough experimental results indicate that the proposed method performs effectively and efficiently, and outperforms many recently published baselines in terms of both objective evaluation and visual fidelity.

Appendix A: Solving BASR on the basis of FISTA

Here, we provide the main process for solving the proposed BASR model (11) on the basis of FISTA [3] which is an effective algorithm for linear inverse problems with dense matrix data. Even though this part is more detailed when compared with that in subsection 3.4, it is still a sketch only. For comprehensive details, we direct you to [3, 11].

First of all, for the sake of convenience, let us divide (11) into two functions and define them respectively

$$ \begin{array}{l} f\left({\boldsymbol{\alpha}}_i\right)\overset{\Delta}{=}{\left\Vert {\boldsymbol{y}}_i-{\boldsymbol{H}\boldsymbol{D}}_{t_i}{\boldsymbol{\alpha}}_i\right\Vert}_2^2,\kern0.5em \nabla f\left({\boldsymbol{\alpha}}_i\right)=2{\boldsymbol{D}}_{t_i}^T\left({\boldsymbol{H}}^T{\boldsymbol{H}\boldsymbol{x}}_i-{\boldsymbol{H}}^T{\boldsymbol{y}}_i\right)\\ {} g\left({\boldsymbol{\alpha}}_i\right)\overset{\Delta}{=}{\mu}_1{\left\Vert {\boldsymbol{\alpha}}_i-{\boldsymbol{\varphi}}_i\right\Vert}_1+{\mu}_2{\left\Vert {\boldsymbol{\alpha}}_i-{\boldsymbol{\psi}}_i\right\Vert}_1\end{array} $$

(13)

Then, according to the mathematic results derived in [3], model (11) can be approximated by considering the following formula at the given point ${\boldsymbol{\alpha}}_i^{(l)}$(i.e., the sparse coefficient in the l-th iteration)

$$ \begin{array}{c}{\left\{{\boldsymbol{\alpha}}_i^{\left( l+1\right)}\right\}}_{i=1}^Q=\rho \left({\boldsymbol{\alpha}}_i^{(l)}, L\right)\overset{\Delta}{=} \arg \underset{{\boldsymbol{\alpha}}_i^{\left( l+1\right)}}{ \min}\left\{\omega \left({\boldsymbol{\alpha}}_i^{\left( l+1\right)},{\boldsymbol{\alpha}}_i^{(l)}, L\right)\right\}\\ {}= \arg \underset{{\boldsymbol{\alpha}}_i^{\left( l+1\right)}}{ \min}\left\{ f\left({\boldsymbol{\alpha}}_i^{(l)}\right)+\left\langle {\boldsymbol{\alpha}}_i^{\left( l+1\right)}-{\boldsymbol{\alpha}}_i^{(l)},\nabla f\left({\boldsymbol{\alpha}}_i^{(l)}\right)\right\rangle +\frac{L}{2}{\left\Vert {\boldsymbol{\alpha}}_i^{\left( l+1\right)}-{\boldsymbol{\alpha}}_i^{(l)}\right\Vert}_2^2+ g\left({\boldsymbol{\alpha}}_i^{\left( l+1\right)}\right)\right\}, for\ i=1,2,\dots, Q\end{array} $$

(14)

It has been proved that the above formula admits a unique minimizer. By getting rid of the constant terms, function ρ can be reformulated into a briefer from

$$ \rho \left({\boldsymbol{\alpha}}_i^{(l)}, L\right)= \arg \underset{{\boldsymbol{\alpha}}_i^{\left( l+1\right)}}{ \min}\left\{ g\left({\boldsymbol{\alpha}}_i^{\left( l+1\right)}\right)+\frac{L}{2}{\left\Vert {\boldsymbol{\alpha}}_i^{\left( l+1\right)}-\left({\boldsymbol{\alpha}}_i^{(l)}-\frac{1}{L}\nabla f\left({\boldsymbol{\alpha}}_i^{(l)}\right)\right)\right\Vert}_2^2\right\} $$

(15)

Notation L here is involved to control the magnitude of step-size. After taking the derivative of (15) with respect to ${\boldsymbol{\alpha}}_i^{\left( l+1\right)}$ and making it equal to zero, we can get

$$ {\boldsymbol{\alpha}}_i^{\left( l+1\right)}+\frac{\mu_1}{L}\mathit{\operatorname{sgn}}\left({\boldsymbol{\alpha}}_i^{\left( l+1\right)}-{\boldsymbol{\varphi}}_i^{(l)}\right)+\frac{\mu_2}{L}\mathit{\operatorname{sgn}}\left({\boldsymbol{\alpha}}_i^{\left( l+1\right)}-{\boldsymbol{\psi}}_i^{(l)}\right)={\boldsymbol{\alpha}}_i^{(l)}-\frac{1}{L}\nabla f\left({\boldsymbol{\alpha}}_i^{(l)}\right) $$

(16)

If we let the left side part of (16) form anther new function F, that is

$$ F\left({\boldsymbol{\alpha}}_i^{\left( l+1\right)}\right)={\boldsymbol{\alpha}}_i^{\left( l+1\right)}+\frac{\mu_1}{L}\mathit{\operatorname{sgn}}\left({\boldsymbol{\alpha}}_i^{\left( l+1\right)}-{\boldsymbol{\varphi}}_i^{(l)}\right)+\frac{\mu_2}{L}\mathit{\operatorname{sgn}}\left({\boldsymbol{\alpha}}_i^{\left( l+1\right)}-{\boldsymbol{\psi}}_i^{(l)}\right) $$

(17)

Then, the function ρ can be expressed as the inverse function of F

$$ \rho \left({\boldsymbol{\alpha}}_i^{t mp}, L\right)={F}^{-1}\left({\boldsymbol{\alpha}}_i^{(l)}-\frac{1}{L}\nabla f\left({\boldsymbol{\alpha}}_i^{(l)}\right)\right)={F}^{-1}\left({\boldsymbol{D}}_{t_i}^T\left({\boldsymbol{x}}_i^{(l)}-\frac{2}{L}\left({\boldsymbol{H}}^T{\boldsymbol{H}\boldsymbol{x}}_i^{(l)}-{\boldsymbol{H}}^T{\boldsymbol{y}}_i\right)\right)\right) $$

(18)

And it can be readily formulated as

$$ \mathrm{If}\ \varphi \le \psi, $$

$$ \rho \left(\alpha, L\right)=\left\{\begin{array}{ll}\alpha +\frac{\mu_1}{L}+\frac{\mu_2}{L}\hfill & \mathrm{if}\kern0.5em \alpha <\varphi -\frac{\mu_1}{L}-\frac{\mu_2}{L}\hfill \\ {}\varphi \hfill & \mathrm{if}\kern0.5em \varphi -\frac{\mu_1}{L}-\frac{\mu_2}{L}\le \alpha <\varphi +\frac{\mu_1}{L}-\frac{\mu_2}{L}\hfill \\ {}\alpha -\frac{\mu_1}{L}+\frac{\mu_2}{L}\hfill & \mathrm{if}\kern0.5em \varphi +\frac{\mu_1}{L}-\frac{\mu_2}{L}\le \alpha <\psi +\frac{\mu_1}{L}-\frac{\mu_2}{L}\hfill \\ {}\psi \hfill & \mathrm{if}\kern0.5em \psi +\frac{\mu_1}{L}-\frac{\mu_2}{L}\le \alpha <\psi +\frac{\mu_1}{L}+\frac{\mu_2}{L}\hfill \\ {}\alpha -\frac{\mu_1}{L}-\frac{\mu_2}{L}\hfill & \mathrm{if}\kern0.5em \psi +\frac{\mu_1}{L}+\frac{\mu_2}{L}\le \alpha \hfill \end{array}\right. $$

(19)

$$ \mathrm{Else}\ \mathrm{if}\ \varphi >\psi, $$

$$ \rho \left(\alpha, L\right)=\left\{\begin{array}{ll}\alpha +\frac{\mu_1}{L}+\frac{\mu_2}{L}\hfill & \mathrm{if}\kern0.5em \alpha <\psi -\frac{\mu_1}{L}-\frac{\mu_2}{L}\hfill \\ {}\psi \hfill & \mathrm{if}\kern0.5em \psi -\frac{\mu_1}{L}-\frac{\mu_2}{L}\le \alpha <\psi -\frac{\mu_1}{L}+\frac{\mu_2}{L}\hfill \\ {}\alpha +\frac{\mu_1}{L}-\frac{\mu_2}{L}\hfill & \mathrm{if}\kern0.5em \psi -\frac{\mu_1}{L}+\frac{\mu_2}{L}\le \alpha <\varphi -\frac{\mu_1}{L}+\frac{\mu_2}{L}\hfill \\ {}\varphi \hfill & \mathrm{if}\kern0.5em \varphi -\frac{\mu_1}{L}+\frac{\mu_2}{L}\le \alpha <\varphi +\frac{\mu_1}{L}+\frac{\mu_2}{L}\hfill \\ {}\alpha -\frac{\mu_1}{L}-\frac{\mu_2}{L}\hfill & \mathrm{if}\kern0.5em \varphi +\frac{\mu_1}{L}+\frac{\mu_2}{L}\le \alpha \hfill \end{array}\right. $$

(20)

It can be noted that the above algorithm corresponds to the general process of classic iterative thresholding algorithm (ITA) [11], which can be viewed as an extension of the classic gradient-based method. Function ρ acts as the shrinkage operator and processed in a pixelwise manner. Although it is well known that the first order optimization algorithms are often the only simple and practical option to deal with large-scale problems such as the case in this paper, it still has been found that the sequence produced by the above algorithm converges quite slowly to the final minimizer [4]. To accelerate the algorithm, the temporary variable before shrinkage operation in (18) ought not to be computed by considering the result obtained in the previous iteration only, but rather to be calculated by utilizing a very special linear combination of the previous two results. Therefore, formula (18) can be modified as follows:

$$ \left\{\begin{array}{l}\boldsymbol{Z}={\widehat{\boldsymbol{X}}}^{(l)}+\frac{t^{\left( l-1\right)}-1}{t^{(l)}}\left({\boldsymbol{X}}^{(l)}-{\boldsymbol{X}}^{\left( l-1\right)}\right)\\ {}\rho \left({\boldsymbol{\alpha}}_i^{t mp}, L\right)=\rho \left({\boldsymbol{D}}_{t_i}^T{\boldsymbol{R}}_i\left(\boldsymbol{Z}-\frac{2}{L}\left({\boldsymbol{H}}^T\boldsymbol{HZ}-{\boldsymbol{H}}^T\boldsymbol{Y}\right)\right), L\right)\end{array}\right. $$

(21)

Besides, anther condition ensuring convergence is to require that the step-size controller L is set to be no less than the smallest Lipschitz constant of the gradient of function f, and this quantity can be determined by employing a backtracking step-size rule. In summary, a step-by-step description of the above implementation can be given in Algorithm 1.