Abstract
The recent advances in hardware and imaging systems made the digital cameras ubiquitous. Although the development of hardware has steadily improved the quality of images for the last several decades, image degradation is unavoidable due to the many factors affecting the image acquisition process and the subsequent post-processing. Image denoising, which aims to reconstruct a high quality image from its degraded observation, is a classical yet still very active topic in the area of low-level computer vision. It represents an important building block in real applications such as digital photography, medical image analysis, remote sensing, surveillance and digital entertainment. Also, image denoising constitutes an ideal test bed for evaluating image prior modeling methods. In this paper, we briefly review recent progresses in image denoising. We firstly present an overview of prior modeling approaches used in image denoising task. Then, we review conventional sparse representation based denoising algorithms, low-rank based denoising algorithms and recently proposed deep neural networks based approaches. At last, we discuss some emerging topics and open problems about image denoising.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
A. Abdelhamed, S. Lin, and M. S. Brown. A high-quality denoising dataset for smartphone cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1692–1700, 2018.
E. Agustsson and R. Timofte. Ntire 2017 challenge on single image super-resolution: Dataset and study. In CVPR Workshops, 2017.
M. Aharon, M. Elad, and A. Bruckstein. rmk-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on signal processing, 2006.
L. Azzari and A. Foi. Variance stabilization for noisy+ estimate combination in iterative poisson denoising. IEEE Signal Process. Lett., 23(8):1086–1090, 2016.
A. J. Bell and T. J. Sejnowski. The “independent components” of natural scenes are edge filters. Vision research, 37(23):3327–3338, 1997.
Y. Blau, R. Mechrez, R. Timofte, T. Michaeli, and L. Zelnik-Manor. 2018 pirm challenge on perceptual image super-resolution. arXiv preprint arXiv:1809.07517, 2018.
A. Buades, B. Coll, and J.-M. Morel. A non-local algorithm for image denoising. In Computer Vision and Pattern Recognition (CVPR), 2005.
A. Buades, B. Coll, and J.-M. Morel. A review of image denoising algorithms, with a new one. Multiscale Modeling & Simulation, 4(2):490–530, 2005.
H. C. Burger, C. Schuler, and S. Harmeling. Learning how to combine internal and external denoising methods. In GCPR, 2013.
E. J. Candès and D. L. Donoho. Ridgelets: A key to higher-dimensional intermittency? Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 357(1760):2495–2509, 1999.
E. J. Candès, X. Li, Y. Ma, and J. Wright. Robust principal component analysis? Journal of the ACM, 58(3):11, 2011.
E. J. Candès and B. Recht. Exact matrix completion via convex optimization. Foundations of Computational mathematics, 9(6):717–772, 2009.
E. J. Candes, M. B. Wakin, and S. P. Boyd. Enhancing sparsity by reweighted ℓ 1 minimization. Journal of Fourier analysis and applications, 14(5–6):877–905, 2008.
A. Chambolle. An algorithm for total variation minimization and applications. Journal of Mathematical imaging and vision, 20(1–2):89–97, 2004.
R. H. Chan, C.-W. Ho, and M. Nikolova. Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Transactions on image processing, 14(10):1479–1485, 2005.
T. Chan, A. Marquina, and P. Mulet. High-order total variation-based image restoration. SIAM Journal on Scientific Computing, 22(2):503–516, 2000.
T. F. Chan and S. Esedoglu. Aspects of total variation regularized l 1 function approximation. SIAM Journal on Applied Mathematics, 65(5):1817–1837, 2005.
P. Chatterjee and P. Milanfar. Patch-based near-optimal image denoising. IEEE Transactions on Image Processing, 21(4):1635–1649, 2012.
J. Chen, J. Chen, H. Chao, and M. Yang. Image blind denoising with generative adversarial network based noise modeling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3155–3164, 2018.
T. Chen, K.-K. Ma, and L.-H. Chen. Tri-state median filter for image denoising. IEEE Transactions on Image processing, 8(12):1834–1838, 1999.
Y. Chen and T. Pock. Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE transactions on pattern analysis and machine intelligence, 2017.
Y. Chen, R. Ranftl, and T. Pock. Insights into analysis operator learning: From patch-based sparse models to higher order mrfs. IEEE Transactions on Image Processing, 23(3):1060–1072, 2014.
Y. Chen, W. Yu, and T. Pock. On learning optimized reaction diffusion processes for effective image restoration. In IEEE Conference on Computer Vision and Pattern Recognition, 2015.
T. S. Cho, N. Joshi, C. L. Zitnick, S. B. Kang, R. Szeliski, and W. T. Freeman. A content-aware image prior. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010.
R. R. Coifman and D. L. Donoho. Translation-invariant de-noising. Springer, 1995.
K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on image processing, 2007.
C.-A. Deledalle, F. Tupin, and L. Denis. Poisson nl means: Unsupervised non local means for poisson noise. In Image processing (ICIP), 2010 17th IEEE international conference on, pages 801–804. IEEE, 2010.
M. N. Do and M. Vetterli. The contourlet transform: an efficient directional multiresolution image representation. IEEE Transactions on image processing, 14(12):2091–2106, 2005.
W. Dong, G. Shi, X. Li, Y. Ma, and F. Huang. Compressive sensing via nonlocal low-rank regularization. IEEE Transaction on Image Processing, 23(8):3618–3632, 2014.
D. L. Donoho. De-noising by soft-thresholding. IEEE transactions on information theory, 41(3):613–627, 1995.
B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, et al. Least angle regression. The Annals of statistics, 32(2):407–499, 2004.
M. Elad, P. Milanfar, and R. Rubinstein. Analysis versus synthesis in signal priors. Inverse problems, 23(3):947, 2007.
S. Esakkirajan, T. Veerakumar, A. N. Subramanyam, and C. PremChand. Removal of high density salt and pepper noise through modified decision based unsymmetric trimmed median filter. IEEE Signal processing letters, 18(5):287–290, 2011.
D. J. Field. Relations between the statistics of natural images and the response properties of cortical cells. JOSA A, 4(12):2379–2394, 1987.
W. T. Freeman, E. C. Pasztor, and O. T. Carmichael. Learning low-level vision. International journal of computer vision, 40(1):25–47, 2000.
R. Giryes and M. Elad. Sparsity-based poisson denoising with dictionary learning. IEEE Transactions on Image Processing, 23(12):5057–5069, 2014.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.
S. Gu, R. Timofte, and L. Van Gool. Integrating local and non-local denoiser priors for image restoration. In 2018 24th International Conference on Pattern Recognition (ICPR), pages 2923–2928. IEEE, 2018.
S. Gu, R. Timofte, and L. Van Gool. Multi-bin trainable linear unit for fast image restoration networks. arXiv preprint arXiv:1807.11389, 2018.
S. Gu, Q. Xie, D. Meng, W. Zuo, X. Feng, and L. Zhang. Weighted nuclear norm minimization and its applications to low level vision. International Journal of Computer Vision, pages 1–26, 2016.
S. Gu, Q. Xie, D. Meng, W. Zuo, X. Feng, and L. Zhang. Weighted nuclear norm minimization and its applications to low level vision. International journal of computer vision, 2017.
S. Gu, L. Zhang, W. Zuo, and X. Feng. Weighted nuclear norm minimization with application to image denoising. In CVPR, 2014.
S. Guo, Z. Yan, K. Zhang, W. Zuo, and L. Zhang. Toward convolutional blind denoising of real photographs. arXiv preprint arXiv:1807.04686, 2018.
S. Hawe, M. Kleinsteuber, and K. Diepold. Analysis operator learning and its application to image reconstruction. IEEE Transactions on Image Processing, 22(6):2138–2150, 2013.
A. Ignatov, N. Kobyshev, R. Timofte, K. Vanhoey, and L. Van Gool. Wespe: Weakly supervised photo enhancer for digital cameras. arXiv preprint arXiv:1709.01118, 2017.
A. Ignatov, R. Timofte, T. Van Vu, T. M. Luu, T. X. Pham, C. Van Nguyen, Y. Kim, J.-S. Choi, M. Kim, J. Huang, et al. Pirm challenge on perceptual image enhancement on smartphones: Report. arXiv preprint arXiv:1810.01641, 2018.
S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.
V. Jain and S. Seung. Natural image denoising with convolutional networks. In Advances in Neural Information Processing Systems, pages 769–776, 2009.
H. Ji, C. Liu, Z. Shen, and Y. Xu. Robust video denoising using low rank matrix completion. In IEEE Conference on Computer Vision and Pattern Recognition, 2010.
J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision, pages 694–711. Springer, 2016.
D. Krishnan, T. Tay, and R. Fergus. Blind deconvolution using a normalized sparsity measure. In IEEE Conference on Computer Vision and Pattern Recognition, 2011.
A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. P. Aitken, A. Tejani, J. Totz, Z. Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR, volume 2, page 4, 2017.
A. Levin and B. Nadler. Natural image denoising: Optimality and inherent bounds. In CVPR 2011, pages 2833–2840. IEEE, 2011.
A. Levin, B. Nadler, F. Durand, and W. T. Freeman. Patch complexity, finite pixel correlations and optimal denoising. In European Conference on Computer Vision, pages 73–86. Springer, 2012.
D. C. Liu and J. Nocedal. On the limited memory bfgs method for large scale optimization. Mathematical programming, 45(1–3):503–528, 1989.
J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman. Non-local sparse models for image restoration. In IEEE International Conference on Computer Vision, 2009.
S. G. Mallat. A theory for multiresolution signal decomposition: the wavelet representation. IEEE transactions on pattern analysis and machine intelligence, 11(7):674–693, 1989.
S. G. Mallat and Z. Zhang. Matching pursuits with time-frequency dictionaries. IEEE Transactions on signal processing, 41(12):3397–3415, 1993.
X. Mao, C. Shen, and Y.-B. Yang. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Advances in neural information processing systems, pages 2802–2810, 2016.
D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th Int’l Conf. Computer Vision, volume 2, pages 416–423, July 2001.
P. Milanfar. A tour of modern image filtering: New insights and methods, both practical and theoretical. IEEE Signal Processing Magazine, 30(1):106–128, 2013.
I. Mosseri, M. Zontak, and M. Irani. Combining the power of internal and external denoising. In IEEE International Conference on Computational Photography (ICCP), page 1–9, 2013.
M. C. Motwani, M. C. Gadiya, R. C. Motwani, and F. C. Harris. Survey of image denoising techniques. In Proceedings of GSPX, pages 27–30, 2004.
V. Nair and G. E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010.
T.-H. Oh, H. Kim, Y.-W. Tai, J.-C. Bazin, and I. S. Kweon. Partial sum minimization of singular values in rpca for low-level vision. In IEEE International Conference on Computer Vision, 2013.
N. Parikh, S. P. Boyd, et al. Proximal algorithms.
Y. C. Pati, R. Rezaiifar, and P. Krishnaprasad. Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In Conference on Signals, Systems and Computers, 1993.
P. Perona and J. Malik. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on pattern analysis and machine intelligence, 12(7):629–639, 1990.
T. Plötz and S. Roth. Benchmarking denoising algorithms with real photographs. In Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pages 2750–2759. IEEE, 2017.
S. Ravishankar and Y. Bresler. Learning sparsifying transforms. IEEE Transactions on Signal Processing, 61(5):1072–1086.
S. Roth and M. J. Black. Fields of experts: A framework for learning image priors. In IEEE Conference on Computer Vision and Pattern Recognition, 2005.
R. Rubinstein, A. M. Bruckstein, and M. Elad. Dictionaries for sparse representation modeling. Proceedings of the IEEE, 98(6):1045–1057, 2010.
R. Rubinstein, T. Peleg, and M. Elad. Analysis k-svd: a dictionary-learning algorithm for the analysis sparse model. IEEE Transactions on Signal Processing, 61(3):661–677, 2013.
L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1):259–268, 1992.
S. Ruslan and N. Srebro. Collaborative filtering in a non-uniform world: Learning with the weighted trace norm. In Conference on Neural Information Processing Systems, 2010.
U. Schmidt and S. Roth. Shrinkage fields for effective image restoration. In IEEE Conference on Computer Vision and Pattern Recognition, 2014.
C. J. Schuler, H. Christopher Burger, S. Harmeling, and B. Scholkopf. A machine learning approach for non-blind image deconvolution. In CVPR, 2013.
Y. Tai, J. Yang, X. Liu, and C. Xu. Memnet: A persistent memory network for image restoration. In CVPR, 2017.
H. Talebi and P. Milanfar. Global image denoising. IEEE Transactions on Image Processing, 23(2):755–768, 2014.
M. F. Tappen, C. Liu, E. H. Adelson, and W. T. Freeman. Learning gaussian conditional random fields for low-level vision. In IEEE Conference on Computer Vision and Pattern Recognition, 2007.
R. Timofte. Anchored fusion for image restoration. In Pattern Recognition (ICPR), 2016 23rd International Conference on, pages 1412–1417. IEEE, 2016.
R. Timofte, S. Gu, J. Wu, and L. Van Gool. Ntire 2018 challenge on single image super-resolution: Methods and results. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2018.
C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In IEEE International Conference on Computer Vision, 1998.
J. S. Turek, I. Yavneh, and M. Elad. On mmse and map denoising under sparse representation modeling over a unitary dictionary. IEEE Transactions on Signal Processing, 59(8):3526–3535, 2011.
S. Wang, L. Zhang, and Y. Liang. Nonlocal spectral prior model for low-level vision. In Asian Conference on Computer Vision, 2012.
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.
Z. Wang and D. Zhang. Progressive switching median filter for the removal of impulse noise from highly corrupted images. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 46(1):78–80, 1999.
J. Weickert. Anisotropic diffusion in image processing, volume 1. 1998.
J. Xie, L. Xu, and E. Chen. Image denoising and inpainting with deep neural networks. In NIPS, 2012.
Q. Xie, Q. Zhao, D. Meng, Z. Xu, S. Gu, W. Zuo, and L. Zhang. Multispectral images denoising by intrinsic tensor sparsity regularization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1692–1700, 2016.
Y. Xie, S. Gu, Y. Liu, W. Zuo, W. Zhang, and L. Zhang. Weighted schatten p-norm minimization for image denoising and background subtraction. arXiv preprint arXiv:1512.01003, 2015.
J. Xu, L. Zhang, D. Zhang, and X. Feng. Multi-channel weighted nuclear norm minimization for real color image denoising. In Proceedings of the IEEE International Conference on Computer Vision, pages 1096–1104, 2017.
G. Yu, G. Sapiro, and S. Mallat. Solving inverse problems with piecewise linear estimators: From gaussian mixture models to structured sparsity. IEEE Transactions on Image Processing, 21(5):2481–2499, 2012.
B. Zhang, J. M. Fadili, and J.-L. Starck. Wavelets, ridgelets, and curvelets for poisson noise removal. IEEE Transactions on Image Processing, 17(7):1093–1108, 2008.
D. Zhang, Y. Hu, J. Ye, X. Li, and X. He. Matrix completion by truncated nuclear norm regularization. In IEEE Conference on Computer Vision and Pattern Recognition, 2012.
K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Transactions on Image Processing, 2017.
L. Zhang, L. Zhang, X. Mou, D. Zhang, et al. Fsim: a feature similarity index for image quality assessment. IEEE transactions on Image Processing, 20(8):2378–2386, 2011.
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.
Z. Zhang, A. Ganesh, X. Liang, and Y. Ma. Tilt: transform invariant low-rank textures. International Journal of Computer Vision, 99(1):1–24, 2012.
S. C. Zhu, Y. Wu, and D. Mumford. Filters, random fields and maximum entropy (frame): Towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2):107–126, 1998.
D. Zoran and Y. Weiss. From learning models of natural image patches to whole image restoration. In IEEE International Conference on Computer Vision (ICCV), 2011.
W. Zuo, D. Ren, S. Gu, L. Lin, and L. Zhang. Discriminative learning of iteration-wise priors for blind deconvolution. In IEEE Conference on Computer Vision and Pattern Recognition, 2015.
W. Zuo, L. Zhang, C. Song, and D. Zhang. Texture enhanced image denoising via gradient histogram preservation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1203–1210, 2013.
Acknowledgements
This work was partly supported by the ETH General Fund, Huawei, and Nvidia through a hardware grant.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Gu, S., Timofte, R. (2019). A Brief Review of Image Denoising Algorithms and Beyond. In: Escalera, S., Ayache, S., Wan, J., Madadi, M., Güçlü, U., Baró, X. (eds) Inpainting and Denoising Challenges. The Springer Series on Challenges in Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-25614-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-25614-2_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25613-5
Online ISBN: 978-3-030-25614-2
eBook Packages: Computer ScienceComputer Science (R0)