Abstract
The current outdoor surveillance equipment and cameras are vulnerable to be influenced by rain, snow, and other inclement weather, reducing the performance of the surveillance systems. In this paper, we propose a method to detect and remove rain streaks even snow artifacts from video sequences, using motion compensation and low-rank matrix completion method. First, we adopt the optical flow estimation method between consecutive frames to get a warped frame and obtain an initial binary rain map. We further use morphological component analysis method to dilate the tiny rain streaks. Then we employ the online dictionary learning for sparse representation technique and SVM classifier to refine the rain map by getting rid of parts which are not rain streaks. Finally, we reconstruct the video sequence by using low-rank matrix completion techniques. The experimental results demonstrate the proposed algorithm and perform qualitatively as well as quantitatively better in terms of PSNR/SSIM.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Abdel-Hakim, A.E.: A novel approach for rain removal from videos using low-rank recovery. In: 2014 5th International Conference on Intelligent Systems, Modelling and Simulation, pp. 351–356. IEEE (2014)
Barnum, P.C., Narasimhan, S.G., Kanade, T.: Analysis of rain and snow in frequency space. Int. J. Comput. Vision 86, 256–274 (2008)
Bossu, J., Hautière, N., Tarel, J.P.: Rain or snow detection in image sequences through use of a histogram of orientation of streaks. Int. J. Comput. Vision 93(3), 348–367 (2011)
Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)
Fadili, M.J., Starck, J.L., Bobin, J., Moudden, Y.: Image decomposition and separation using sparse representations: an overview. Proc. IEEE 98(6), 983–994 (2009)
Garg, K., Nayar, S.K.: Detection and removal of rain from videos. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, p. I (2004)
Garg, K., Nayar, S.K.: When does a camera see rain? In: Tenth IEEE International Conference on Computer Vision (ICCV 2005) Volume 1, vol. 2, pp. 1067–1074 (2005)
Garg, K., Nayar, S.K.: Vision and rain. Int. J. Comput. Vision 75, 3–27 (2006)
George, J., Bhavani, S., Jaya, J.: Certain explorations on removal of rain streaks using morphological component analysis. Int. J. Eng. Res. Technol. (IJERT) 2 (2013). ISSN: 2278-0181
Jiang, T.X., Huang, T.Z., Zhao, X.L., Deng, L.J., Wang, Y.: FastDeRain: a novel video rain streak removal method using directional gradient priors. IEEE Trans. Image Process. 28, 2089–2102 (2018)
Kim, J.H., Sim, J.Y., Kim, C.S.: Video deraining and desnowing using temporal correlation and low-rank matrix completion. IEEE Trans. Image Process. 24, 2658–2670 (2015)
Li, M., et al.: Video rain streak removal by multiscale convolutional sparse coding. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6644–6653 (2018)
Li, R., Cheong, L.F., Tan, R.T.: Heavy rain image restoration: integrating physics model and conditional adversarial learning. arXiv abs/1904.05050 (2019)
Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 262–277. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_16
Liu, C., et al.: Beyond pixels: exploring new representations and applications for motion analysis. Ph.D. thesis, Massachusetts Institute of Technology (2009)
Liu, P., Xu, J., Liu, J., Tang, X.: Pixel based temporal analysis using chromatic property for removing rain from videos. Comput. Inf. Sci. 2, 53–60 (2009)
Mairal, J., Bach, F.R., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 11, 19–60 (2010)
Mukhopadhyay, S., Tripathi, A.K.: Combating bad weather part i: rain removal from video. Synth. Lect. Image Video Multimedia Process. 7(2), 1–93 (2014)
Qian, R., Tan, R.T., Yang, W., Su, J., Liu, J.: Attentive generative adversarial network for raindrop removal from a single image. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2482–2491 (2017)
Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: a better and simpler baseline. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
Tripathi, A.K., Mukhopadhyay, S.: Removal of rain from videos: a review. SIViP 8, 1421–1430 (2014)
Tripathi, A., Mukhopadhyay, S.: Video post processing: low-latency spatiotemporal approach for detection and removal of rain. IET Image Proc. 6(2), 181–196 (2012)
Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., Lau, R.W.: Spatial attentive single-image deraining with a high quality real rain dataset. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
Wei, W., Yi, L., Xie, Q., Zhao, Q., Meng, D., Xu, Z.: Should we encode rain streaks in video as deterministic or stochastic? In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2535–2544 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, Y., Shimada, N. (2020). Using Motion Compensation and Matrix Completion Algorithm to Remove Rain Streaks and Snow for Video Sequences. In: Palaiahnakote, S., Sanniti di Baja, G., Wang, L., Yan, W. (eds) Pattern Recognition. ACPR 2019. Lecture Notes in Computer Science(), vol 12046. Springer, Cham. https://doi.org/10.1007/978-3-030-41404-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-41404-7_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-41403-0
Online ISBN: 978-3-030-41404-7
eBook Packages: Computer ScienceComputer Science (R0)