Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Benchmark for the robustness of image features in rainy conditions

  • 236 Accesses

  • 1 Citations

Abstract

Computer vision systems are increasingly present on roadways, both on the roadside and on board vehicles. Image features are an essential building block for computer vision algorithms in a road environment. Eight of the most representative image features in a road environment are selected on the basis of a literature review, and their robustness in rainy conditions is evaluated. In order to do this, a new evaluation method is proposed, which is then applied to a new weather-image database. This database contains rain typical of latitudes with a temperate climate (\(0{-}30\, \mathrm{mm}\,\mathrm{h}^{-1} \)), various camera settings and images with natural rain and images with digitally simulated rain. Image features based on pixel intensity and those that use vertical edges are sensitive to rainy conditions. Conversely, the Harris feature and features that combine different edge orientations remain robust for rainfall rates of \(0{-}30\, \mathrm{mm}\,\mathrm{h}^{-1} \). The robustness of image features in rainy conditions decreases as the rainfall rate increases. Finally, the image features most sensitive to rain have potential for use in a camera-based rain classification application.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Notes

  1. 1.

    Sensor inserted into the road surface that detects vehicles via an electromagnetic field.

References

  1. 1.

    Allamano, P., Croci, A., Laio, F.: Toward the camera rain gauge. Water Resour. Res. 51(3), 1744–1757 (2015). https://doi.org/10.1002/2014WR016298

  2. 2.

    Ballard, D.: Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognit. 13(2), 111–122 (1981). https://doi.org/10.1016/0031-3203(81)90009-1

  3. 3.

    Bay, H., Tuytelaars, T., Gool, L.V.: Surf: Speeded up robust features. In: European Conference on Computer Vision (2006). http://link.springer.com/chapter/10.1007/11744023_32

  4. 4.

    Bertozzi, M., Broggi, A., Caraffi, C., Del Rose, M., Felisa, M., Vezzoni, G.: Pedestrian detection by means of far-infrared stereo vision. Comput. Vis. Image Underst. 106, 194–204 (2007). https://doi.org/10.1016/j.cviu.2006.07.016

  5. 5.

    Bossu, J., Hautière, N., Tarel, J.P.: Rain or snow detection in image sequences through use of a histogram of orientation of streaks. Int. J. Comput. Vis. 93(3), 348–367 (2011). https://doi.org/10.1007/s11263-011-0421-7

  6. 6.

    Chen, Y.L., Hsu, C.T.: A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: International Conference on Computer Vision, pp. 1968–1975 (2013). https://doi.org/10.1109/ICCV.2013.247. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6751355

  7. 7.

    Chen, J., Chau, L.P.: A rain pixel recovery algorithm for videos with highly dynamic scenes. IEEE Trans. Image Process 23(3), 1097–1104 (2014). https://doi.org/10.1109/TIP.2013.2290595

  8. 8.

    Colomb, M., Hirech, K., André, P., Boreux, J., Lacôte, P., Dufour, J.: An innovative artificial fog production device improved in the European project FOG. Atmos. Res. 87(3–4), 242–251 (2008). https://doi.org/10.1016/j.atmosres.2007.11.021

  9. 9.

    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005). http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1467360

  10. 10.

    Duthon, P., Bernardin, F., Chausse, F., Colomb, M.: Methodology used to evaluate computer vision algorithms in adverse weather conditions. In: Proceedings of 6th Transport Research Arena. Warsaw, Poland (2016)

  11. 11.

    Duthon, P., Chausse, F., Bernardin, F., Colomb, M.: Descripteurs d’images en contexte routier: une revue bibliographique. In: Reconnaissance des Formes et l’Intelligence Artificielle (RFIA). Clermont-Fd, France (2016)

  12. 12.

    Foucher, P., Sebsadji, Y., Tarel, J.p., Charbonnier, P., Nicolle, P.: Detection and recognition of urban road markings using images. In: IEEE Conference on Intelligent Transportation Systems, pp. 1747–1752 (2011)

  13. 13.

    Franke, U., Pfeiffer, D., Rabe, C., Knoeppel, C., Enzweiler, M., Stein, F., Herrtwich, R.G.: Making bertha see. In: IEEE International Conference on Computer Vision, pp. 214–221 (2013). https://doi.org/10.1109/ICCVW.2013.36

  14. 14.

    Fukushima, K.: Neocognitron: a hierarchical neural network capable of visual pattern recognition. Neural Netw. 1(2), 119–130 (1988). https://doi.org/10.1016/0893-6080(88)90014-7

  15. 15.

    Garg, K., Nayar, S.: When does a camera see rain? In: International Conference on Computer Vision, vol. 2, pp. 1067–1074 IEEE (2005). https://doi.org/10.1109/ICCV.2005.253. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1544839

  16. 16.

    Garg, K., Nayar, S.K.: Vision and rain. Int. J. Comput. Vis. 75(1), 3–27 (2007). https://doi.org/10.1007/s11263-006-0028-6

  17. 17.

    Habib, E., Krajewski, W.F., Kruger, A.: Sampling errors of tipping-bucket rain gauge measurements. J. Hydrol. Eng. 6(2), 159–166 (2001). https://doi.org/10.1061/(ASCE)1084-0699(2001)6:2(159)

  18. 18.

    Hanwell, D., Mirmehdi, M.: Detection of lane departure on high-speed roads. Icpram (2) (2012). http://www.cs.bris.ac.uk/home/dh3173/DetectionOfLaneDepartureOnHighSpeedRoads.pdf

  19. 19.

    Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey Vision Conference (1988). http://courses.daiict.ac.in/pluginfile.php/13002/mod_resource/content/0/References/harris1988.pdf

  20. 20.

    Huiying, D., Xue, L., Xuejing, Z.: Method of detection and removal rain from image based on the HSV color space. J. Chem. Pharm. Res. 6(6), 1481–1487 (2014)

  21. 21.

    Kim, J.H., Sim, J.Y., Kim, C.S., Member, S.: Video deraining and desnowing using temporal correlation and low-rank matrix completion. IEEE Trans. Image Process 24, 2658–2670 (2015)

  22. 22.

    Lemaire, T., Berger, C., Jung, I.K., Lacroix, S.: Vision-based SLAM: stereo and monocular approaches. Int. J. Comput. Vis. 74(3), 343–364 (2007). https://doi.org/10.1007/s11263-007-0042-3

  23. 23.

    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

  24. 24.

    Oliveira, L., Monteiro, G., Peixoto, P., Nunes, U.: Towards a robust vision-based obstacle perception with classifier fusion in cybercars. Eurocast 4739, 1089–1096 (2007). https://doi.org/10.1007/978-3-540-75867-9

  25. 25.

    Reisfeld, D., Wolfson, H., Yeshurun, Y.: Context-free attentional operators: the generalized symmetry transform. Int. J. Comput. Vis. 14, 119–130 (1995). https://doi.org/10.1007/BF01418978

  26. 26.

    Ren, W., Tian, J., Han, Z., Chan, A., Tang, Y.: Video desnowing and deraining based on matrix decomposition. In: CVPR, pp. 4210–4219 (2017)

  27. 27.

    Royer, E., Bom, J., Dhome, M., Thuilot, B., Lhuillier, M., Marmoiton, F.: Outdoor autonomous navigation using monocular vision. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (2005). https://doi.org/10.1109/IROS.2005.1545495

  28. 28.

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

  29. 29.

    Ruta, A., Porikli, F., Watanabe, S., Li, Y.: In-vehicle camera traffic sign detection and recognition. Mach. Vis. Appl. 22(2), 359–375 (2011). https://doi.org/10.1007/s00138-009-0231-x

  30. 30.

    Satpute, P., Pawar, S.: A survey on rain pixel recovery techniques for videos. Int. J. Eng. Trends Technol. 10(5), 264–266 (2014)

  31. 31.

    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. ICLR 2015, 1–14 (2015)

  32. 32.

    Toulminet, G., Bertozzi, M., Mousset, S., Bensrhair, A., Broggi, A.: Vehicle detection by means of stereo vision-based obstacles features extraction and monocular pattern analysis. IEEE Trans. Image Process 15(8), 2364–2375 (2006). https://doi.org/10.1109/TIP.2006.875174

  33. 33.

    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2001). http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=990517

  34. 34.

    World Meteorological Organization, (WMO): The guide to hydrological practices (WMO No.168). Tech. rep., World Meteorological Organization (2009)

  35. 35.

    Yao, C., Wang, C., Hong, L., Cheng, Y.: A bayesian probabilistic framework for rain detection. Entropy 16(6), 3302–3314 (2014)

  36. 36.

    Zhou, J., Gao, D., Zhang, D.: Moving vehicle detection for automatic traffic monitoring. In: IEEE Vehicular Technology (2007). http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4067123

Download references

Acknowledgements

The authors would like to thank LabEx ImobS3 and ViaMeca innovation cluster thanks to which relations between the Clermont-Ferrand laboratories are even stronger. They would also like to thank the whole team in charge of the Cerema R&D Fog and Rain platform and the weather-image station.

Author information

Correspondence to Pierre Duthon.

Additional information

This work has been sponsored by the French government research program "Investissements d’Avenir" through the IMobS3 Laboratory of Excellence (ANR-10-LABX-16-01) and the RobotEx Equipment of Excellence (ANR-10-EQPX-44), by the European union through the Regional Competitiveness and Employment program 2014–2020 (ERDF - AURA region) and by the AURA region.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Duthon, P., Bernardin, F., Chausse, F. et al. Benchmark for the robustness of image features in rainy conditions. Machine Vision and Applications 29, 915–927 (2018). https://doi.org/10.1007/s00138-018-0945-8

Download citation

Keywords

  • Intelligent transportation systems
  • Image processing
  • Image analysis
  • Image feature extraction
  • Machine vision
  • Cameras
  • Advanced driver assistance system
  • Rainfall rate
  • Rainfall