Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 19, pp 25905–25918 | Cite as

Single image deraining using deep convolutional networks

  • Meihua Wang
  • Jiaming Mai
  • Ruichu Cai
  • Yun Liang
  • Hua Wan
Article

Abstract

A deep learning-based single image deraining algorithm is proposed in this work. Instead of modeling a rain layer as a linear function between the rain image and its clear version as previous works do, we directly formulate the clear image as the result of a non-linear mapping of thrain image. We construct a coarse deraining convolutional network and a refinement convolutional network to learn this non-linear mapping function. The coarse deraining network is trained to detect the rain streaks with different directions, and restore a raw derained result. The refinement network aims at refining the result according to the raw derained image and the original rain image. By combining the two networks, we are able to well-restore the rain-free image. Experimental results demonstrate that the proposed deraining method can produce high-quality clear images from both synthetic and real-world rain images, outperforming the state-of-the-art methods qualitatively and quantitatively.

Keywords

Image deraining image restoration image enhancement deep learning 

Notes

Acknowledgments

This work is financially supported by National Natural Science Foundation of China (61202269, 61472089, 61202293, 31600591), Science and Technology Plan Project of Guangdong Province (2014A050503057, 2015A020209124, 2016A020210087).

Supplementary material

11042_2018_5825_MOESM1_ESM.pdf (779 kb)
(PDF 778 KB)

References

  1. 1.
    Adding rain to a photo with Photoshop. http://www.photoshopessentials.com/photo-effects/rain/
  2. 2.
    Bossu J, Hautiere N, Tarel JP (2011) Rain or snow detection in image sequences through use of a histogram of orientation of streaks. Int J Comput Vis 93(3):348–367CrossRefGoogle Scholar
  3. 3.
    Cai B, Xu X, Jia K, Qing C, Tao D (2016) DehazeNet: an end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198MathSciNetCrossRefGoogle Scholar
  4. 4.
    Chen LC, Barron JT, Papandreou G, Murphy K, Yuille AL (2016) Semantic image segmentation with task-specific edge detection using CNNs and a discriminatively trained domain transform. In: IEEE conference on computer vision and pattern recognitionGoogle Scholar
  5. 5.
    Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307CrossRefGoogle Scholar
  6. 6.
    Garg K, Nayar SK (2004) Detection and removal of rain from videos. In: IEEE conference on computer vision and pattern recognitionGoogle Scholar
  7. 7.
    Garg K, Nayar SK (2007) Vision and rain. Int J Comput Vis 75(1):3–27CrossRefGoogle Scholar
  8. 8.
    Garg K, Nayar SK (2005) When does a camera see rain?. In: IEEE conference on computer visionGoogle Scholar
  9. 9.
    He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: European conference on computer vision, pp 346–361Google Scholar
  10. 10.
    Hong S, You T, Han B (2015) Online tracking by learning discriminative saliency map with convolutional neural network. In: International conference on machine learningGoogle Scholar
  11. 11.
    Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: Convolutional architecture for fast feature embedding. In: ACM international conference on multimedia, pp 675–678Google Scholar
  12. 12.
    Kang LW, Lin CW, Fu YH (2012) Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans Image Process 21(4):1742–1755MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Kim JH, Lee C, Sim JY, Kim CS (2013) Single image deraining using an adaptive nonlocal means filter. In: IEEE international conference on image processingGoogle Scholar
  14. 14.
    Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  15. 15.
    Li Y, Tan RT, Guo X, Lu J, Brown MS (2016) Rain streak removal using layer priors. In: IEEE conference computer vision and pattern recognitionGoogle Scholar
  16. 16.
    Luo Y, Xu Y, Ji H (2015) Removing rain from a single image via discriminative sparse coding. In: IEEE international conference on computer visionGoogle Scholar
  17. 17.
    Ren W, Liu S, Zhang H, Pan J, Cao X, Yang MH (2004) Single image dehazing via multi-scale convolutional neural networks. In: IEEE conference on computer vision and pattern recognitionGoogle Scholar
  18. 18.
    Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15(2):430–444CrossRefGoogle Scholar
  19. 19.
    Thu QH, Ghanbari M (2008) Scope of validity of psnr in image/video quality assessment. Electron Lett 44(13):800–801CrossRefGoogle Scholar
  20. 20.
    Tripathi AK, Mukhopadhyay S (2014) Removal of rain from videos: a review. Signal, Image and Video Processing, Springer 8(8):1421–1430CrossRefGoogle Scholar
  21. 21.
    Vincent P, Larochelle H, Lajoie I, Bengio Y, Manazagol PA (2010) Stacked denoising autoencoders: learning useful representations in a deep neural network with a local denoising criterion. J Mach Learn Res 11(12):3371–3408MathSciNetzbMATHGoogle Scholar
  22. 22.
    Wang Z, Bovik AC (2002) A universal image quality index. IEEE Trans Image Signal Process Lett 19(3):81–84CrossRefGoogle Scholar
  23. 23.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612CrossRefGoogle Scholar
  24. 24.
    Zhang X, Li H, Qi Y, Leow WK, Ng TK (2006) Rain removal in video by combining temporal and chromatic properties. In: IEEE international conference on multimedia and expoGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.College of Mathematics and InformaticsSouth China Agricultural UniversityGuangzhouChina
  2. 2.School of Computer Science and TechnologyGuangdong University of TechnologyGuangzhouChina

Personalised recommendations