Deep Image Demosaicking Using a Cascade of Convolutional Residual Denoising Networks

  • Filippos KokkinosEmail author
  • Stamatios Lefkimmiatis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11218)


Demosaicking and denoising are among the most crucial steps of modern digital camera pipelines and their joint treatment is a highly ill-posed inverse problem where at-least two-thirds of the information are missing and the rest are corrupted by noise. This poses a great challenge in obtaining meaningful reconstructions and a special care for the efficient treatment of the problem is required. While there are several machine learning approaches that have been recently introduced to deal with joint image demosaicking-denoising, in this work we propose a novel deep learning architecture which is inspired by powerful classical image regularization methods and large-scale convex optimization techniques. Consequently, our derived network is more transparent and has a clear interpretation compared to alternative competitive deep learning approaches. Our extensive experiments demonstrate that our network outperforms any previous approaches on both noisy and noise-free data. This improvement in reconstruction quality is attributed to the principled way we design our network architecture, which also requires fewer trainable parameters than the current state-of-the-art deep network solution. Finally, we show that our network has the ability to generalize well even when it is trained on small datasets, while keeping the overall number of trainable parameters low.


Deep learning Denoising Demosaicking Proximal method Residual denoising 


  1. 1.
    Li, X., Gunturk, B., Zhang, L.: Image demosaicing: a systematic survey (2008)Google Scholar
  2. 2.
    Zhang, L., Wu, X., Buades, A., Li, X.: Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J. Electron. Imaging 20(2), 023016 (2011)CrossRefGoogle Scholar
  3. 3.
    Duran, J., Buades, A.: Self-similarity and spectral correlation adaptive algorithm for color demosaicking. IEEE Trans. Image Process. 23(9), 4031–4040 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Buades, A., Coll, B., Morel, J.M., Sbert, C.: Self-similarity driven color demosaicking. IEEE Trans. Image Process. 18(6), 1192–1202 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Heide, F., et al.: Flexisp: a flexible camera image processing framework. ACM Trans. Graph. (TOG) 33(6), 231 (2014)CrossRefGoogle Scholar
  6. 6.
    Chang, K., Ding, P.L.K., Li, B.: Color image demosaicking using inter-channel correlation and nonlocal self-similarity. Signal Process. Image Commun. 39, 264–279 (2015)CrossRefGoogle Scholar
  7. 7.
    Hirakawa, K., Parks, T.W.: Adaptive homogeneity-directed demosaicing algorithm. IEEE Trans. Image Process. 14(3), 360–369 (2005)CrossRefGoogle Scholar
  8. 8.
    Alleysson, D., Susstrunk, S., Herault, J.: Linear demosaicing inspired by the human visual system. IEEE Trans. Image Process. 14(4), 439–449 (2005)CrossRefGoogle Scholar
  9. 9.
    Dubois, E.: Frequency-domain methods for demosaicking of bayer-sampled color images. IEEE Signal Process. Lett. 12(12), 847–850 (2005)CrossRefGoogle Scholar
  10. 10.
    Dubois, E.: Filter design for adaptive frequency-domain bayer demosaicking. In: 2006 International Conference on Image Processing, pp. 2705–2708, October 2006Google Scholar
  11. 11.
    Dubois, E.: Color filter array sampling of color images: Frequency-domain analysis and associated demosaicking algorithms, pp. 183–212, January 2009CrossRefGoogle Scholar
  12. 12.
    Sun, J., Tappen, M.F.: Separable markov random field model and its applications in low level vision. IEEE Trans. Image Process. 22(1), 402–407 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    He, F.L., Wang, Y.C.F., Hua, K.L.: Self-learning approach to color demosaicking via support vector regression. In: 19th IEEE International Conference on Image Processing (ICIP), pp. 2765–2768. IEEE (2012)Google Scholar
  14. 14.
    Khashabi, D., Nowozin, S., Jancsary, J., Fitzgibbon, A.W.: Joint demosaicing and denoising via learned nonparametric random fields. IEEE Trans. Image Process. 23(12), 4968–4981 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Foi, A., Trimeche, M., Katkovnik, V., Egiazarian, K.: Practical poissonian-gaussian noise modeling and fitting for single-image raw-data. IEEE Trans. Image Process. 17(10), 1737–1754 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Ossi Kalevo, H.R.: Noise reduction techniques for bayer-matrix images (2002)Google Scholar
  17. 17.
    Menon, D., Calvagno, G.: Joint demosaicking and denoisingwith space-varying filters. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 477–480, November 2009Google Scholar
  18. 18.
    Zhang, L., Lukac, R., Wu, X., Zhang, D.: PCA-based spatially adaptive denoising of CFA images for single-sensor digital cameras. IEEE Trans. Image Process. 18(4), 797–812 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Klatzer, T., Hammernik, K., Knobelreiter, P., Pock, T.: Learning joint demosaicing and denoising based on sequential energy minimization. In: 2016 IEEE International Conference on Computational Photography (ICCP), pp. 1–11, May 2016Google Scholar
  20. 20.
    Gharbi, M., Chaurasia, G., Paris, S., Durand, F.: Deep joint demosaicking and denoising. ACM Trans. Graph. 35(6), 191:1–191:12 (2016)CrossRefGoogle Scholar
  21. 21.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends® Mach. Learn. 3(1), 1–122 (2011)zbMATHCrossRefGoogle Scholar
  22. 22.
    Goldstein, T., Osher, S.: The split bregman method for l1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Hunter, D.R., Lange, K.: A tutorial on MM algorithms. Am. Stat. 58(1), 30–37 (2004)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Figueiredo, M.A., Bioucas-Dias, J.M., Nowak, R.D.: Majorization-minimization algorithms for wavelet-based image restoration. IEEE Trans. Image Process. 16(12), 2980–2991 (2007)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Romano, Y., Elad, M., Milanfar, P.: The little engine that could: Regularization by denoising (red). SIAM J. Imaging Sci. 10(4), 1804–1844 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Venkatakrishnan, S.V., Bouman, C.A., Wohlberg, B.: Plug-and-play priors for model based reconstruction. In: 2013 IEEE Global Conference on Signal and Information Processing, pp. 945–948, December 2013Google Scholar
  27. 27.
    Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Lefkimmiatis, S.: Universal denoising networks: a novel CNN architecture for image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3204–3213 (2018)Google Scholar
  29. 29.
    Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. arXiv preprint (2017)Google Scholar
  30. 30.
    Foi, A.: Clipped noisy images: Heteroskedastic modeling and practical denoising. Signal Process. 89(12), 2609–2629 (2009)zbMATHCrossRefGoogle Scholar
  31. 31.
    Liu, X., Tanaka, M., Okutomi, M.: Single-image noise level estimation for blind denoising. IEEE Trans. Image Process. 22(12), 5226–5237 (2013)CrossRefGoogle Scholar
  32. 32.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  33. 33.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  34. 34.
    Lin, Q., Xiao, L.: An adaptive accelerated proximal gradient method and its homotopy continuation for sparse optimization. Comput. Optim. Appl. 60(3), 633–674 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 60–65. IEEE (2005)Google Scholar
  36. 36.
    Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision, ICCV 2001, vol. 2, pp. 416–423 (2001)Google Scholar
  38. 38.
    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)Google Scholar
  39. 39.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  40. 40.
    Robinson, A.J., Fallside, F.: The utility driven dynamic error propagation network. Technical report CUED/F-INFENG/TR.1, Engineering Department, Cambridge University, Cambridge, UK (1987)Google Scholar
  41. 41.
    Getreuer, P.: Color demosaicing with contour stencils. In: 2011 17th International Conference on Digital Signal Processing (DSP), pp. 1–6, July 2011Google Scholar
  42. 42.
    Bigdeli, S.A., Zwicker, M., Favaro, P., Jin, M.: Deep mean-shift priors for image restoration. In: Advances in Neural Information Processing Systems, pp. 763–772 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Skolkovo Institute of Science and Technology (Skoltech)MoscowRussia

Personalised recommendations