Fast Single Image De-raining via a Weighted Residual Network
Deep learning based methods for single image de-raining have shown great success in recent literatures. However, it is still a challenge to reduce the computation time while maintaining the de-raining performance. In this paper, we introduce a weighted residual network (WRN) to address above challenge. Inspired by the image processing knowledge that a rainy image can be decomposed into a base (low-pass) layer and a detail (high-pass) layer, we train the network on a weighted residual between the weighted detail layer of rainy image and the detail layer of clean image, which can significantly reduce the mapping range from input to output and easily employ the image enhancement operation on the base layer and the detail layer separately to handle the heavy rain with hazy looking. We also introduce a weighted convolution-deconvolution network structure to make the training easier. The first layer of network is a multi-scale convolution to expand the receptive field of the network. Our WRN requires less computation time for processing a test image because we set the stride of intermediate layers to 2 without zero-padding. Experiment results on both synthetic and real-world images demonstrate our WRN achieves high-quality recovery compared to several advanced methods of single image de-raining.
KeywordsRain removal Deep learning WRN
This work is supported by the National Natural Science Foundation of China (No.61762014, No.61462026 and No.61762012), the Opening Project of Guangxi Colleges and Universities Key Laboratory of robot & welding (Guilin University of Aerospace Technology), the Opening Project of Shaanxi Key Laboratory of Complex Control System and Intelligent Information Processing, and the Research Fund of Guangxi Key Lab of intelligent integrated automation.
- 2.Eigen, D., Krishnan, D., Fergus, R.: Restoring an image taken through a window covered with dirt or rain. In: IEEE International Conference on Computer Vision, pp. 633–640 (2014)Google Scholar
- 4.Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1715–1723 (2017)Google Scholar
- 5.Garg, K., Nayar, S.K.: Detection and removal of rain from videos. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, pp. I-528-I-535 (2004)Google Scholar
- 7.He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
- 9.Kim, J.H., Lee, C., Sim, J.Y., Kim, C.S.: Single-image deraining using an adaptive nonlocal means filter. In: IEEE International Conference on Image Processing, pp. 914–917 (2014)Google Scholar
- 11.Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: Computer Vision and Pattern Recognition, pp. 2736–2744 (2016)Google Scholar
- 12.Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: IEEE International Conference on Computer Vision, pp. 3397–3405 (2015)Google Scholar
- 13.Mao, X.J., Shen, C., Yang, Y.B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Advances in Neural Information Processing Systems, pp. 2802–2810 (2016)Google Scholar
- 16.Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1357–1366 (2017)Google Scholar
- 18.Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
- 19.Zhang, X., Li, H., Qi, Y., Leow, W.K.: Rain removal in video by combining temporal and chromatic properties. In: IEEE International Conference on Multimedia and Expo, pp. 461–464 (2006)Google Scholar