# Hybrid variational model based on alternating direction method for image restoration

- 106 Downloads

## Abstract

The total variation model is widely used in image deblurring and denoising process with the features of protecting the image edge. However, this model usually causes some staircase effects. To overcome the shortcoming, combining the second-order total variation regularization and the total variation regularization, we propose a hybrid total variation model. The new improved model not only eliminates the staircase effect, but also well protects the edges of the image. The alternating direction method of multipliers (ADMM) is employed to solve the proposed model. Numerical results show that our proposed model can get more details and higher image visual quality than some current state-of-the-art methods.

## Keywords

Total variation Image restoration Staircase effect Alternating direction method of multipliers## 1 Introduction

*α*is a positive regularization parameter that controls the tradeoff between these two terms. To define the discrete TV norm, we first introduce the discrete gradient ∇

*f*:

*f*, which is the \((i,j)\)th pixel location of the image; see [7]. Then the discrete TV of

*f*is defined by

*β*is a penalty parameter. Experimental results verify the effectiveness of the FTVd method. But in the calculation, the penalty parameter

*β*needs to approach infinity, which creates numerical instability. To avoid the approach of penalty parameter to infinity, Chan et al. [28] proposed the alternating direction method of multipliers (ADMM) to solve model (1.2). By defining the augmented Lagrange function, the image restoration model (1.2) can be translated into the following form:

*λ*is a Lagrange multiplier. The experimental results show that the ADMM method is robust and fast, and has a good restoration effect.

*f*in model (1.2), Huang et al. [29] proposed a fast total variation minimization method for image restoration as follows:

Although the total variation regularization can preserve sharp edges very well, it also causes some staircase effects [31, 32]. To overcome this kind of staircase effect, some high-order total variational models [33, 34, 35, 36, 37, 38, 39] and fractional-order total variation models [40, 41, 42, 43, 44] are introduced. It has been proved that the high-order TV norm can remove the staircase effect and preserve the edges well in the process of image restoration.

*u*, and \(\|\nabla^{2}f\|_{2}\) is the second-order TV norm of

*f*. The definition of the second-order TV norm is similar to that of the TV norm. The second-order TV norm is defined by

*f*. For more detail about the second-order differences, we refer to [45]. The second-order TV regularization and TV regularization are used; the edges in the restored image can be preserved quite well, and the staircase effect is reduced simultaneously.

The rest of this paper is organized as follows. In Sect. 2, we propose our alternating iterative algorithm to solve model (1.5). In Sect. 3, we give some numerical results to demonstrate the effectiveness of the proposed algorithm. Finally, concluding remarks are given in Sect. 4.

## 2 The alternating iterative algorithm

### 2.1 The deblurring step

*ω*, the unconstrained optimization problem (2.1) can be transformed into the following equivalent constraint optimization problem:

*f*is denoted by \(\mathcal{F}(f)\), and \(\mathcal{F}^{-1}(f)\) is the inverse Fourier transform of

*f*. By using the Fourier transform the solution of

*f*can be given as follows:

*ω*can be written as

*η*is a relaxation parameter, and \(\eta\in(0,(\sqrt{5} + 1)/2)\).

### 2.2 The denoising step

Subproblem (2.2) is a classical TV regularization process for image denoising, which can be solved by the Chambolle projection algorithm. However, it is well known that the Chambolle projection algorithm has large amount of calculations in the process of experiment and causes numerical instability. To overcome the disadvantage of numerical instability and large amount of calculations of the Chambolle projection algorithm, in this paper, we adopt the alternating direction multiplier method to solve subproblem (2.2).

*v*, problem (2.2) can be transformed into the following constraint minimization problem:

The variables *u*, *f*, *v* are coupled together, so we separate this problem into two subproblems and adopt the alternating iteration minimization method. The two subproblems are given as follows.

*u*-subproblem” for

*v*fixed:

*v*-subproblem” for

*u*fixed:

*v*is equivalent to the minimization problem

*η*is a relaxation parameter.

## 3 Numerical experiments

*g*, \(f^{0}\), and

*u*are the observed image, the ideal image, and the restored image, respectively. Then, the BSNR, MSE, PSNR, and SSIM are defined as follows:

*η*is the additive noise vector, \(n^{2}\) is the number of pixels of image, \(\mathrm{MAX}_{f^{0}}\) is the maximum possible pixel value of the \(f^{0}\),

*f̄*is the mean intensity value of \(f^{0}\), \(\mu_{f^{0}}\) is the mean value of the \(f^{0}\), \(\mu_{u}\) is the mean value of

*u*, \(\sigma_{f^{0}}^{2}\) and \(\sigma_{u}^{2}\) are the variances of \(f^{0}\) and

*u*, respectively, and \(\sigma_{f^{0}u}\) is the covariance of \(f^{0}\) and

*u*, and \(c_{1}\) and \(c_{2}\) are stabilizing constants for near-zero denominator values. We will also use the SSIM index map to reveals areas of high/low similarity between two images; the whiter the SSIM index map, the closer the two images. Further details on SSIM can be founded in the pioneer work [49].

*k*th iteration of the tested method. In the following experiments, for our proposed method, we fixed the parameter \(\alpha_{2}=1.3e{-}2\) for all experiments, \(\alpha_{1}=1e{-}4\) (for Gaussian blur and average blur), \(3e{-}4\) (for motion blur), \(\alpha_{3}=1e{-}4\) (for Gaussian blur and average blur), \(2e{-}4\) (for motion blur). For the parameters of FastTV and FNDTV, we refer to [29, 30]. The parameters for every compared method are selected from many experiments until we obtain the best PSNR values.

Experimental results for different images and different blur kernels, \(\mathrm{BSNR}=35\)

Image | Blur kernels | Fast-TV [26] | FNDTV [27] | Our | |||
---|---|---|---|---|---|---|---|

PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | ||

Cameraman | Gaussian(5, 5) | 27.0656 | 0.4399 | 27.2341 | 0.4417 | | |

Gaussian(7, 7) | 26.1232 | 0.3992 | 26.8021 | 0.4155 | | | |

Gaussian(9, 9) | 24.9719 | 0.3807 | 25.6502 | 0.4024 | | | |

Couple | Gaussian(5, 5) | 31.3219 | 0.7337 | 31.6776 | 0.7595 | | |

Gaussian(7, 7) | 29.9460 | 0.6767 | 30.7103 | 0.6989 | | | |

Gaussian(9, 9) | 29.2731 | 0.6694 | 29.8778 | 0.6739 | | | |

Lenna | average(7) | 31.3460 | 0.6673 | 31.8335 | 0.6916 | | |

average(9) | 30.5242 | 0.6415 | 31.0531 | 0.6541 | | | |

average(11) | 29.5574 | 0.6134 | 30.4395 | 0.6376 | | | |

Goldhill | average(7) | 28.3139 | 0.6077 | 29.2712 | 0.6188 | | |

average(9) | 28.1314 | 0.5816 | 28.3268 | 0.5990 | | | |

average(11) | 26.8336 | 0.5244 | 27.4211 | 0.5576 | | | |

Man | motion(20, 20) | 29.8667 | 0.6258 | 30.1235 | 0.6622 | | |

motion(10, 100) | 30.5363 | 0.6839 | 31.2202 | 0.7130 | | | |

Baboon | motion(20, 20) | 27.4672 | 0.7968 | 27.8722 | 0.8334 | | |

motion(10, 100) | 28.8783 | 0.8213 | 28.9383 | 0.8621 | | |

Experimental results for different images and different blur kernels, \(\mathrm{BSNR}=40\)

Image | Blur kernels | Fast-TV [26] | FNDTV [27] | Proposed | |||
---|---|---|---|---|---|---|---|

PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | ||

cameraman | Gaussian(5, 5) | 28.6897 | 0.4841 | 29.2310 | 0.5061 | | |

Gaussian(7, 7) | 27.4559 | 0.4427 | 27.0594 | 0.4369 | | | |

Gaussian(9, 9) | 25.6219 | 0.4068 | 26.2835 | 0.4128 | | | |

couple | Gaussian(5, 5) | 32.0301 | 0.7718 | 32.8019 | 0.7942 | | |

Gaussian(7, 7) | 31.4577 | 0.7401 | 32.0764 | 0.7610 | | | |

Gaussian(9, 9) | 30.1657 | 0.6786 | 30.9071 | 0.7011 | | | |

lenna | average(7) | 32.0735 | 0.7049 | 32.5344 | 0.7279 | | |

average(9) | 31.0432 | 0.6500 | 31.8694 | 0.6853 | | | |

average(11) | 30.7127 | 0.6404 | 31.0552 | 0.6586 | | | |

goldhill | average(7) | 30.3156 | 0.6356 | 31.1560 | 0.6576 | | |

average(9) | 29.4280 | 0.6183 | 30.2251 | 0.6301 | | | |

average(11) | 28.3722 | 0.6082 | 29.5978 | 0.6202 | | | |

man | motion(20, 10) | 30.7569 | 0.6720 | 31.3522 | 0.7031 | | |

motion(11, 100) | 31.6579 | 0.7234 | 32.1598 | 0.7398 | | | |

baboon | motion(20, 10) | 30.3003 | 0.8837 | 30.6833 | 0.8993 | | |

motion(11, 100) | 31.4377 | 0.9082 | 31.9844 | 0.9245 | | |

The numerical results of three different methods in terms of PSNR and SSIM are shown in the following tables. From Tables 1 and 2 it is not difficult to see that the PSNR and SSIM of the restored image by our proposed method are higher than those obtained by FastTV and FNDTV.

## 4 Conclusion

In this paper, we propose a hybrid total variation model. In addition, we employ the alternating direction method of multipliers to solve it. Experimental results demonstrate that the proposed model can obtain better results than those restored by some existing restoration methods. It also shows that the new model can obtain a better visual resolution than the other two methods.

## Notes

### Acknowledgements

The authors would like to thank the referees for their valuable comments and suggestions.

### Authors’ contributions

All authors worked together to produce the results and read and approved the final manuscript.

### Funding

This work was supported by National Key Research and Development Program of China (No. 2017YFC1405600), by the Training Program of the Major Research Plan of National Science Foundation of China (No. 91746104), by National Science Foundation of China (Nos. 61101208, 11326186), Qindao Postdoctoral Science Foudation (No. 2016114), Project of Shandong Province Higher Educational Science and Technology Program (No. J17KA166), Joint Innovative Center for Safe and Effective Mining Technology and Equipment of Coal Resources, Shandong Province of China and SDUST Research Fund (No. 2014TDJH102).

### Competing interests

The authors declare that there is no conflict of interest regarding the publication of this paper.

## References

- 1.Hajime, T., Hayashi, T., Nishi, T.: Application of digital image analysis to pattern formation in polymer systems. J. Appl. Phys.
**59**(11), 3627–3643 (1986) CrossRefGoogle Scholar - 2.Chen, M., Xia, D., Han, J., Liu, Z.: An analytical method for reducing metal artifacts in X-ray CT images. Math. Probl. Eng.
**2019**, Article ID 2351878 (2019) Google Scholar - 3.Chen, M., Li, G.: Forming mechanism and correction of CT image artifacts caused by the errors of three system parameters. J. Appl. Math.
**2013**, Article ID 545147 (2013) zbMATHGoogle Scholar - 4.Chen, Y., Guo, Y., Wang, Y., et al.: Denoising of hyperspectral images using nonconvex low rank matrix approximation. IEEE Trans. Geosci. Remote Sens.
**55**(9), 5366–5380 (2017) CrossRefGoogle Scholar - 5.Tikhonov, A.N., Arsenn, V.Y.: Solution of Ill-Posed Problem. Winston and Sons, Washington (1977) Google Scholar
- 6.Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physics D
**60**, 259–268 (1992) MathSciNetCrossRefGoogle Scholar - 7.Vogel, C.R., Oman, M.E.: Iterative methods for total variation denoising. SIAM J. Sci. Comput.
**17**, 227–238 (1996) MathSciNetCrossRefGoogle Scholar - 8.Zhu, J.G., Hao, B.B.: A new noninterior continuation method for solving a system of equalities and inequalities. J. Appl. Math.
**2014**, Article ID 592540 (2014) MathSciNetGoogle Scholar - 9.Yu, J., Li, M., Wang, Y., He, G.: A decomposition method for large-scale box constrained optimization. Appl. Math. Comput.
**231**(12), 9–15 (2014) MathSciNetzbMATHGoogle Scholar - 10.Han, C., Feng, T., He, G., Guo, T.: Parallel variable distribution algorithm for constrained optimization with nonmonotone technique. J. Appl. Math.
**2013**, Article ID 295147 (2013) MathSciNetzbMATHGoogle Scholar - 11.Sun, L., He, G., Wang, Y.: An accurate active set newton algorithm for large scale bound constrained optimization. Appl. Math.
**56**(3), 297–314 (2011) MathSciNetCrossRefGoogle Scholar - 12.Zheng, F., Han, C., Wang, Y.: Parallel SSLE algorithm for large scale constrained optimization. Appl. Math. Comput.
**217**(12), 5277–5384 (2011) MathSciNetzbMATHGoogle Scholar - 13.Zhu, J., Hao, B.: A new smoothing method for solving nonlinear complementarity problems. Open Math.
**17**(1), 21–38 (2019) Google Scholar - 14.Tian, Z., Tian, M., Gu, C.: An accelerated jacobi gradient based iterative algorithm for solving sylvester matrix equations. Filomat
**31**(8), 2381–2390 (2017) MathSciNetCrossRefGoogle Scholar - 15.Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis.
**20**(1–2), 89–97 (2004) MathSciNetzbMATHGoogle Scholar - 16.Wang, Y., Yang, J., Yin, W., et al.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci.
**1**(3), 248–272 (2008) MathSciNetCrossRefGoogle Scholar - 17.Zhang, R.Y., Xu, F.F., Huang, J.C.: Reconstructing local volatility using total variation. Acta Math. Sin. Engl. Ser.
**33**(2), 263–277 (2017) MathSciNetCrossRefGoogle Scholar - 18.Bai, Z.B., Dong, X.Y., Yin, C.: Existence results for impulsive nonlinear fractional differential equation with mixed boundary conditions. Bound. Value Probl.
**2016**, 63 (2016) MathSciNetCrossRefGoogle Scholar - 19.Wang, Z.: A numerical method for delayed fractional-order differential equations. J. Appl. Math.
**2013**, 256071 (2013) MathSciNetzbMATHGoogle Scholar - 20.Wang, Z., Huang, X., Zhou, J.P.: A numerical method for delayed fractional-order differential equations: based on G-L definition. Appl. Math. Inf. Sci.
**7**(2), 525–529 (2013) MathSciNetCrossRefGoogle Scholar - 21.Zhang, Y.L., Lv, K.B., et al..: Modeling gene networks in Saccharomyces cerevisiae based on gene expression profiles. Comput. Math. Methods Med.
**2015**, Article ID 621264 (2015) zbMATHGoogle Scholar - 22.Lu, X., Wang, H.X., Wang, X.: On Kalman smoothing for wireless sensor networks systems with multiplicative noises. J. Appl. Math.
**2012**, 203–222 (2012) MathSciNetzbMATHGoogle Scholar - 23.Ding, S.F., Huang, H.J., Xu, X.Z., et al.: Polynomial smooth twin support vector machines. Appl. Math. Inf. Sci.
**8**(4), 2063–2071 (2014) MathSciNetCrossRefGoogle Scholar - 24.Goldfarb, D., Yin, W.: Second-order cone programming methods for total variation based image restoration. SIAM J. Sci. Comput.
**27**(2), 622–645 (2005) MathSciNetCrossRefGoogle Scholar - 25.Han, C.Y., Zheng, F.Y., Guo, T.D., He, G.P.: Parallel algorithms for large-scale linearly constrained minimization problem. Acta Math. Appl. Sin. Engl. Ser.
**30**(3), 707–720 (2014) MathSciNetCrossRefGoogle Scholar - 26.Hao, B.B., Zhu, J.G.: Fast L1 regularized iterative forward backward splitting with adaptive parameter selection for image restoration. J. Vis. Commun. Image Represent.
**44**, 139–147 (2017) CrossRefGoogle Scholar - 27.Morini, B., Porcelli, M., Chan, R.H.: A reduced Newton method for constrained linear least squares problems. J. Comput. Appl. Math.
**233**, 2200–2212 (2010) MathSciNetCrossRefGoogle Scholar - 28.Chan, R.H., Tao, M., Yuan, X.: Constrained total variation deblurring models and fast algorithms based on alternating direction method of multipliers. SIAM J. Imaging Sci.
**6**(1), 680–697 (2013) MathSciNetCrossRefGoogle Scholar - 29.Huang, Y.M., Ng, M.K., Wen, Y.W.: A fast total variation minimization method for image restoration. Multiscale Model. Simul.
**7**(2), 774–795 (2008) MathSciNetCrossRefGoogle Scholar - 30.Liu, J., Huang, T.Z., et al..: An efficient variational method for image restoration. Abstr. Appl. Anal.
**2013**, 213536 (2013) MathSciNetzbMATHGoogle Scholar - 31.Chambolle, A., Lions, P.L.: Image recovery via total variation minimization and related problems. Numer. Math.
**76**, 167–188 (1997) MathSciNetCrossRefGoogle Scholar - 32.Chan, T., Marquina, A., Mulet, P.: High-order total variation-based image restoration. SIAM J. Sci. Comput.
**22**(2), 503–516 (2000) MathSciNetCrossRefGoogle Scholar - 33.Lv, X., Song, Y., Wang, S., et al.: Image restoration with a high-order total variation minimization method. Appl. Math. Model.
**37**, 8210–8224 (2013) MathSciNetCrossRefGoogle Scholar - 34.Liu, G., Huang, T., Liu, J.: High-order TVL1-based images restoration and spatially adapted regularization parameter selection. Comput. Math. Appl.
**67**, 2015–2026 (2014) MathSciNetCrossRefGoogle Scholar - 35.Zhu, G.J., Li, K., Hao, B.B.: Image restoration by a mixed high-order total variation and l1 regularization model. Math. Probl. Eng.
**2018**, Article ID 6538610 (2018) Google Scholar - 36.Lysaker, M., Tai, X.C.: Iterative image restoration combining total variation minimization and a second-order functional. Int. J. Comput. Vis.
**66**(1), 5–18 (2006) CrossRefGoogle Scholar - 37.Zhu, J., Liu, K., Hao, B.: Restoration of remote sensing images based on nonconvex constrained high-order total variation regularization. J. Appl. Remote Sens.
**13**(2), 022006 (2019) Google Scholar - 38.You, Y.L., Kaveh, M.: Fourth-order partial differential equations for noise removal. IEEE Trans. Image Process.
**9**(10), 1723–1730 (2000) MathSciNetCrossRefGoogle Scholar - 39.Hajiaboli, M.R.: An anisotropic fourth-order diffusion filter for image noise removal. Int. J. Comput. Vis.
**92**(2), 177–191 (2011) MathSciNetCrossRefGoogle Scholar - 40.Chen, D., Chen, Y., Xue, D.: Fractional-order total variation image denoising based on proximity algorithm. Appl. Math. Comput.
**257**, 537–545 (2015) MathSciNetzbMATHGoogle Scholar - 41.Wang, Z., Xie, Y., Lu, J.,et al..: Stability and bifurcation of a delayed generalized fractional-order prey–predator model with interspecific competition. Appl. Math. Compt.
**347**, 360–369 (2019) MathSciNetCrossRefGoogle Scholar - 42.Wang, X., Wang, Z., Huang, X.,et al..: Dynamic analysis of a fractional-order delayed SIR model with saturated incidence and treatment functions. Int. J. Bifurcat. Chaos
**28**(14), 1850180 (2018) CrossRefGoogle Scholar - 43.Jiang, C., Zhang, F., Li, T.: Synchronization and antisynchronization of N-coupled fractional-order complex systems with ring connection. Math. Methods Appl. Sci.
**41**(3), 2625–2638 (2018) MathSciNetCrossRefGoogle Scholar - 44.Ren, Z., He, C., Zhang, Q.: Fractional order total variation regularization for image super-resolution. Signal Process.
**93**(9), 2408–2421 (2013) CrossRefGoogle Scholar - 45.Wu, C., Tai, X.C.: Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM J. Imaging Sci.
**3**(3), 300–339 (2010) MathSciNetCrossRefGoogle Scholar - 46.Michael, N., Chan, R.H., Tang, W.C.: A fast algorithm for deblurring models with Neumann boundary conditions. SIAM J. Sci. Comput.
**21**(3), 851–866 (1999) MathSciNetCrossRefGoogle Scholar - 47.Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice Hall, New York (2011) Google Scholar
- 48.Yang, J.F., et al.: A fast algorithm for edge-preserving variational multichannel image restoration. SIAM J. Imaging Sci.
**2**, 569–592 (2011) MathSciNetCrossRefGoogle Scholar - 49.Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process.
**13**, 600–612 (2004) CrossRefGoogle Scholar

## Copyright information

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.