Skip to main content
SpringerLink
Log in

Sparsity-Based Denoising of Photographic Images: From Model-Based to Data-Driven

  • Chapter
  • First Online:
Denoising of Photographic Images and Video

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

What makes photographic images different from random noise? It has been hypothesized that sparsity is a key factor separating the class of photographic images from noise observations. Accordingly, sparse representations have been widely studied in the literature of image denoising in the past decades. In this chapter, we present a critical review of the most important ideas/insights behind sparsity-based image denoising algorithms. In the first-generation (model-based) approaches, we will highlight the evolution from local wavelet-based image denoising in 1990s–2000s to nonlocal and patch-based image denoising from 2006 to 2015. In the second-generation (data-driven) approaches, we have opted to review several latest advances in the field of image denoising since 2016 such as learning parametric sparse models (for heavy noise removal) and deep learning -based approaches (including deep residue learning). The overarching theme of our review is to provide a unified conceptual understanding of why and how sparsity-based image denoising works—in particular, the evolving role played by models and data. Based on our critical review, we will discuss a few open issues and promising directions for future research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.robots.ox.ac.uk/~vgg/data/oxbuildings/.

  2. 2.

    http://www.robots.ox.ac.uk/~vgg/data/parisbuildings/.

References

  1. Rudin L, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D 60:259–268

    Article  MathSciNet  Google Scholar 

  2. Perona P, Malik J (1990) Scale space and edge detection using anisotropic diffusion. IEEE Trans Patt Anal Mach Intell. 12(7):629–639

    Article  Google Scholar 

  3. Steidl G, Weickert J, Brox T, Mrázek P, Welk M (2004) On the equivalence of soft wavelet shrinkage, total variation diffusion, total variation regularization, and sides. SIAM J Numer Anal 42(2):686–713

    Article  MathSciNet  Google Scholar 

  4. Portilla J, Strela V, Wainwright M, Simoncelli E (2003) Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Trans Image Process 12:1338–1351

    Article  MathSciNet  Google Scholar 

  5. Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans Image Process 16:2080–2095

    Article  MathSciNet  Google Scholar 

  6. Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2009) Non-local sparse models for image restoration. In: 2009 IEEE 12th international conference on computer vision, pp 2272–2279

    Google Scholar 

  7. Dong W, Shi G, Li X (2013) Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans Image Process 22(2):700–711

    Article  MathSciNet  Google Scholar 

  8. Gu S, Zhang L, Zuo W, Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2862–2869

    Google Scholar 

  9. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  10. Li Y, Dong W, Xie X, Shi G, Li X (2018 to appear) Learning parametric sparse models for heavy noisy removal from images. IEEE Access

    Google Scholar 

  11. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a Gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155

    Article  MathSciNet  Google Scholar 

  12. Chen Y, Pock T (2017) Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans Pattern Anal Mach Intelligen 39(6):1256–1272

    Article  Google Scholar 

  13. Bae W, Yoo J, Ye JC (2017) Beyond deep residual learning for image restoration: Persistent homology-guided manifold simplification. In: IEEE conference on computer vision and pattern recognition (CVPR) workshops, pp 145–153

    Google Scholar 

  14. Pennebaker WB, Mitchell JL (1992) JPEG: still image data compression standard. Kluwer

    Google Scholar 

  15. Foi A, Katkovnik V, Egiazarian K (2007) Pointwise shape-adaptive dct for high-quality denoising and deblocking of grayscale and color images. IEEE Trans Image Process 16:1395–1411

    Article  MathSciNet  Google Scholar 

  16. Mallat S (1989) Multiresolution approximations and wavelet orthonormal bases of \(l^2({\vec{r}})\). Trans Amer Math Soc 315:69–87

    MathSciNet  MATH  Google Scholar 

  17. Daubechies I (1992) Ten lectures on wavelets, vol 61 of CBMS conference series in applied mathematics, Philadelphia, SIAM

    Google Scholar 

  18. Candes E (1998) Ridgelets: theory and applications. PhD thesis, Department of Statistics, Stanford University

    Google Scholar 

  19. Starck JL, Candes EJ, Donoho DL (2002) The curvelet transform for image denoising. IEEE Trans Image Process 670–684

    Google Scholar 

  20. Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14:2091–2106

    Article  Google Scholar 

  21. Mallat S (1999) A wavelet tour of signal processing, 2nd ed. Academic Press

    Google Scholar 

  22. Chen S, Donoho D, Saunders M (1998) Atomic decomposition by basis pursuit. SIAM J Scientif Comput 20:33–61

    Article  MathSciNet  Google Scholar 

  23. Donoho D (2006) Compressed sensing. IEEE Trans Infor Theor 52(4):1289–1306

    Article  MathSciNet  Google Scholar 

  24. Jolliffe IT (1986) Principal component analysis. Springer, Springer Series in Statistics, Berlin

    Book  Google Scholar 

  25. Duda R, Hart P, Stork D (2001) Pattern ccassification, New York, Wiley, 2nd ed.

    Google Scholar 

  26. Aharon M, Elad M, Bruckstein A (2006) \( rm K \)-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    Article  Google Scholar 

  27. Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Proc 15:3736–3745

    Article  MathSciNet  Google Scholar 

  28. Dabov K, Foi A, Katkovnik V, Egiazarian K (2006) Image denoising with block-matching and 3D filtering. In: Proceedings of SPIE electronic imaging: algorithms and systems, vol 6064, San Jose, CA, USA

    Google Scholar 

  29. Dong W, Li X, Zhang L, Shi G (2011) Sparsity-based image denoising via dictionary learning and structural clustering. In: IEEE conference on computer vision and pattern recognition

    Google Scholar 

  30. Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. J Royal Statistic Soc Ser B (Statist Methodol) 68(1):49–67

    Article  MathSciNet  Google Scholar 

  31. Cotter S, Rao B, Engan K, Kreutz-Delgado K (2005) Sparse solutions to linear inverse problems with multiple measurement vectors. IEEE Trans Signal Process 53(7):2477–2488

    Article  MathSciNet  Google Scholar 

  32. Cai J, Candes E, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optimiz 20:1956

    Article  MathSciNet  Google Scholar 

  33. Andrews DF, Mallows CL (1974) Scale mixtures of normal distributions. J Royal Statistic Soc Ser B (Statist Methodol) 36(1):99–102

    MathSciNet  MATH  Google Scholar 

  34. Dong W, Shi G, Ma Y, Li X (2015) Image restoration via simultaneous sparse coding: where structured sparsity meets Gaussian scale mixture. Int J Comput Vis 1–16

    Google Scholar 

  35. Lyu S, Simoncelli E (2009) Modeling multiscale subbands of photographic images with fields of Gaussian scale mixtures. IEEE Trans Pattern Anal Mach Intelligen 31(4):693–706

    Article  Google Scholar 

  36. Box GE, Tiao GC (2011) Bayesian inference in statistical analysis, vol 40. Wiley

    Google Scholar 

  37. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. In: IEEE conference on computer vision and pattern recognition, CVPR’07, pp 1–8. IEEE

    Google Scholar 

  38. Dai L, Sun L, Wu E, Yu N (2013) Large scale image retrieval with visual groups. In: IEEE international conference on image processing (ICIP), pp 2582–2586

    Google Scholar 

  39. Babenko A, Lempitsky V (2015) Aggregating local deep features for image retrieval. In: Proceedings of the IEEE international conference on computer vision, pp 1269–1277

    Google Scholar 

  40. Gordo A, Almazán J, Revaud J, Larlus D (2016) Deep image retrieval: learning global representations for image search. In: European Conference on Computer Vision, pp 241–257. Springer

    Google Scholar 

  41. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comp Vis 60:91–110

    Article  Google Scholar 

  42. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell, pp 971–987

    Google Scholar 

  43. Yue H, Sun X, Yang J, Wu F (2013) Landmark image super-resolution by retrieving web images. IEEE Trans Image Process 22(12):4865–4878

    Article  MathSciNet  Google Scholar 

  44. Li Y, Dong W, Shi G, Xie X (2015) Learning parametric distributions for image super-resolution: where patch matching meets sparse coding. In: Proceedings of the IEEE international conference on computer vision, pp 450–458

    Google Scholar 

  45. Fischler MA, Bolles RC (1987) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Read Comput Vis, pp 726–740. Elsevier

    Google Scholar 

  46. Yue H, Sun X, Yang J, Wu E (2014) Cid: Combined image denoising in spatial and frequency domains using web images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2933–2940

    Google Scholar 

  47. Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630

    Article  MathSciNet  Google Scholar 

  48. Zoran D, Weiss Y (2011) From learning models of natural image patches to whole image restoration. In: Proceedings of the ICCV

    Google Scholar 

  49. Burger H, Schuler C, Harmeling S (2012) Image denoising: can plain neural networks compete with bm3d? In: 2012 IEEE conference on computer vision and pattern recognition (CVPR), pp 2392–2399

    Google Scholar 

  50. Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166

    Article  Google Scholar 

  51. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

    Google Scholar 

  52. Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22:56–65

    Article  Google Scholar 

  53. Timofte R, De Smet V, Van Gool L (2014) A+: adjusted anchored neighborhood regression for fast super-resolution. Comput Vis–ACCV 2014, pp 111–126. Springer

    Google Scholar 

  54. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. Comput Vis-ECCV 2014:184–199

    Google Scholar 

  55. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst, pp 1097–1105

    Google Scholar 

  56. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision, pp 391–407. Springer

    Google Scholar 

  57. Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1637–1645

    Google Scholar 

  58. Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A et al (2017) Mastering the game of go without human knowledge. Nature 550(7676):354–359

    Article  Google Scholar 

  59. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Process Syst, pp 2672–2680

    Google Scholar 

  60. Pinheiro PO, Lin TY, Collobert R, Dollár P (2016) Learning to refine object segments. In: European conference on computer vision, pp 75–91. Springer

    Google Scholar 

  61. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241. Springer

    Google Scholar 

  62. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. In: International conference on learning representations (ICLR)

    Google Scholar 

  63. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2016) Photo-realistic single image super-resolution using a generative adversarial network arXiv:1609.04802

  64. Yang Q, Yan P, Zhang Y, Yu H, Shi Y, Mou X, Kalra MK, Wang G (2017) Low dose ct image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. arXiv:1708.00961

  65. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USA

    Xin Li

  2. School of Electronic Engineering, Xidian University, Xi’an, China

    Weisheng Dong & Guangming Shi

Authors
  1. Xin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weisheng Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guangming Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Li .

Editor information

Editors and Affiliations

  1. Pompeu Fabra University, Barcelona, Spain

    Marcelo Bertalmío

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Li, X., Dong, W., Shi, G. (2018). Sparsity-Based Denoising of Photographic Images: From Model-Based to Data-Driven. In: Bertalmío, M. (eds) Denoising of Photographic Images and Video. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-96029-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-96029-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-96028-9

  • Online ISBN: 978-3-319-96029-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation