Advertisement

Estimating ambient visibility in the presence of fog: a deep convolutional neural network approach

  • Fatma Outay
  • Bilal Taha
  • Hazar Chaabani
  • Faouzi KamounEmail author
  • Naoufel Werghi
  • Ansar -Ul-Haque Yasar
Original Article

Abstract

Next-generation intelligent transportation systems are based on the acquisition of ambient data that influence traffic flow and safety. Among these, is the ambient visibility range whose estimation, in the presence of fog, is extremely useful for next-generation intelligent transportation systems. However, existing camera-based approaches are based on “engineered features” extraction methods that use computer algorithms and procedures from the image processing field. In this contribution, a novel approach to estimate visibility range under foggy weather conditions is proposed which is based on “learned features” instead. More precisely, we use AlexNet deep convolutional neural network (DCNN), trained with raw image data, for feature extraction and a support vector machine (SVM) for visibility range estimation. Our quantitative analysis showed that the proposed approach is very promising in estimating the visibility range with very good accuracy. The proposed solution can pave the way towards intelligent driveway assistance systems to enhance awareness of driving weather conditions and hence mitigate the safety risks emanating from fog-induced low visibility conditions.

Keywords

Intelligent transportation systems Ubiquitous technologies Atmospheric visibility Road safety Deep convolutional neural networks Ambient intelligence 

Notes

Funding information

This research was financially supported by Zayed University Cluster Research Grant No. R17075.

References

  1. 1.
    Christina M (2019) Heavy fog causes 20-plus vehicle collision in Texas. https://edition.cnn.com/2019/01/01/us/austin-20-vehicle-crash/index.html . Accessed 14 March 2019
  2. 2.
    Napoli, DS (2018) Heavy early-morning fog causes fatal accident in north Harris County. https://www.click2houston.com/news/heavy-early-morning-fog-causes-fatal-accident . Accessed March 14 2019
  3. 3.
    Heavy fog a factor in fatal crash involving up to 29 vehicles in Verona, https://fox6now.com/2018/02/12/fatal-dane-county-highway-crash-involves-about-20-vehicles/ . Accessed March 14 2019
  4. 4.
    Heavy fog leads to fatal crash in Wisconsin. https://www.kcrg.com/content/news/Heavy-fog-leads-to-fatal-crash-in-Wisconsin-473867243.html . Accessed March 14 2019
  5. 5.
    Dubai police record 564 accidents due to fog on Thursday, https://www.thenational.ae/uae/dubai-police-record-564-accidents-due-to-fog-on-thursday-1.703044 . Accessed March 14, 2019
  6. 6.
    Hamilon B, Tefft B, Arnold L, Grabowski J (2014) Hidden highways: fog and traffic crashes on America’s roads, 2014 AAA Foundation for Traffic Safety Report. July 2014:1–24Google Scholar
  7. 7.
    Chaabani H, Kamoun F, Bargaoui H, Outay F, Yasar A (2017) A neural network approach to visibility range estimation under foggy weather conditions. Procedia Comput Sci 113:466–471.  https://doi.org/10.1016/j.procs.2017.08.304 CrossRefGoogle Scholar
  8. 8.
    Hautière N, Bossu J, Bigorgne E, Hiblot N, Boubezoul, Lusett B, Aubert D (2009) Sensing visibility range at low cost in the SAFESPOT roadside unit. ITS World Congress (ITS’09), Stockholm, pp 1–8Google Scholar
  9. 9.
    Meteorological office: observers’ handbook. Third edition, London: HMSO, 1969Google Scholar
  10. 10.
    Grosshans H, Kristensson E, Szász RZ, Berrocal E (2015) Prediction and measurement of the local extinction coefficient in sprays for 3D simulation/experiment data comparison. Int J Multiphase Flow 72:218–232.  https://doi.org/10.1016/j.ijmultiphaseflow.2015.01.009 CrossRefGoogle Scholar
  11. 11.
    Middleton W (1952) Vision through the atmosphere. University of Toronto Press, TorontoCrossRefGoogle Scholar
  12. 12.
    IEC (1987) International lighting vocabulary. CIE 17:4–1987 The International Commission on IlluminationGoogle Scholar
  13. 13.
    Perrin J, Martin P, Cottrell W (2000) Effects of variable speed limit signs on driver behavior during inclement weather. Compendium for the Institute of Transportation Engineers (ITE). 70th Annual Meeting, Nashville, TennesseeGoogle Scholar
  14. 14.
    Kwon T (1998) An automatic visibility measurement system based on video cameras. Publication MN/RC-1998-25. Minnesota Department of Transportation: 1-62Google Scholar
  15. 15.
    Baumer D, Versick S, Vogel B (2008) Determination of the visibility using a digital panorama camera. Atmos Environ 42(11):2593–2602.  https://doi.org/10.1016/j.atmosenv.2007.06.024 CrossRefGoogle Scholar
  16. 16.
    Abdel-Aty M, Ahmed MM, Lee JSQ, Abuzwidah M (2012) Synthesis of visibility detection systems. University of Central Florida Report, BDK78 977-11: 1-130, http://www.fdot.gov/research/Completed_Proj/Summary_TE/FDOT-BDK78-977-11-rpt.pdf . Accessed 14 January 2018
  17. 17.
    Hautiere N, Labayrade R, Aubert D (2006) Estimation of the visibility distance by stereovision: a generic approach. IEICE Trans on Information and Systems 89(7):2084–2091CrossRefGoogle Scholar
  18. 18.
    Bush C, Debes E (1998) Wavelet transform for analyzing fog visibility. IEEE Intell Syst 13(6):66–71CrossRefGoogle Scholar
  19. 19.
    Guo F, Peng H, Tang J, Zou B, Tang C (2016) Visibility detection approach to road scene foggy images. KSII Trans on Internet and Information Systems 10(9):4419–4441.  https://doi.org/10.3837/tiis.2016.09.022 CrossRefGoogle Scholar
  20. 20.
    Hautiere N, Tarel JP, Lavenant J, Aubert D (2006) Automatic fog detection and estimation of visibility distance through the use of onboard camera. Mach Vis Appl 17(1):8–20.  https://doi.org/10.1007/s00138-005-0011-1 CrossRefGoogle Scholar
  21. 21.
    Bronte S, Bergasa LM., Alcantarilla PF (2009) Fog detection system based on computer vision techniques. 12th International IEEE Conference on Intelligent Transportation Systems: 1–6  https://doi.org/10.1109/ITSC.2009.5309842.
  22. 22.
    Hautière N, Bigorgne E, Aubert D (2008) Daytime visibility range monitoring through use of a roadside camera. IEEE Intelligent Vehicles Symposium, Eindhoven, pp 470–475.  https://doi.org/10.1109/IVS.2008.4621200 CrossRefGoogle Scholar
  23. 23.
    Hautière N, Bigorgne E, Bossu J, Aubert D (2008) Meteorological conditions processing for vision-based traffic monitoring. In: The Eighth International Workshop on Visual Surveillance -VS2008. Marseille, France, pp 1–8Google Scholar
  24. 24.
    Boussard C, Hautière N, D’andréa-Novel B (2008) Vehicle dynamics estimation for camera-based visibility distance estimation. IEEE/RSJ International Conference on Intelligent Robotics and Systems, Nice-France, pp 600–605Google Scholar
  25. 25.
    Hautière N, Labayrade R, Aubert D (2006) Real-time disparity contrast combination for onboard estimation of the visibility distances. IEEE Trans Intell Transp Syst 7(2):201–212.  https://doi.org/10.1109/TITS.2006.874682 CrossRefGoogle Scholar
  26. 26.
    Gallen R, Cord A, Hautiére N, Aubert D (2011) Towards night fog detection through use of in-vehicle multipurpose cameras. IEEE Intelligent Vehicle Symposium, Baden-Baden, Germany, pp 399–404.  https://doi.org/10.1109/IVS.2011.5940486 CrossRefGoogle Scholar
  27. 27.
    Wauben W, Roth M (2016) Exploration of fog detection and visibility estimation from camera images. WMO Technical Conference on Meteorological and Environmental Instruments and Methods of Observation, CIMO TECO, Madrid, Spain: 1-14Google Scholar
  28. 28.
    Hautiére N, Babari R, Dumont É, Brémond R, Paparoditis N (2010) Estimating meteorological visibility using cameras: a probabilistic model-driven approach. In: Kimmel R, Klette R, Sugimoto A (eds) Computer Vision – ACCV 2010, Lecture Notes in Computer Science, vol 6495. Springer, Berlin, Heidelberg, pp 243–254.  https://doi.org/10.1007/978-3-642-19282-1_20 CrossRefGoogle Scholar
  29. 29.
    Budnik, M, Gutierrez-Gomez, EL, Safadi B, Quénot G (2015) Learned features versus engineered features for semantic video indexing. Proceedings of the 13th International Workshop on Content-Based Multimedia Indexing (CBMI), Prague, Czech Republic: 1-6Google Scholar
  30. 30.
    Chow TWS, Rahman MKM (2007) A new image classification technique using tree-structured regional features. Neurocomputing 70(4-6):1040–1050.  https://doi.org/10.1016/j.neucom.2006.01.033 CrossRefGoogle Scholar
  31. 31.
    Zhang H, Fritts JE, Goldman SA( 2005) A fast texture feature extraction method for region-based image segmentation. Proc. SPIE 5685, Image and Video Communications and Processing 2005 Conference:1-12Google Scholar
  32. 32.
    Xu X, Shafin SH, Li Y, Hao HW (2014) A prototype system for atmospheric visibility estimation based on image analysis and learning. Journal of Information and Computational Science 11(3):4577–4585CrossRefGoogle Scholar
  33. 33.
    Fan B, Kong Q, Wang X, Wang Z, Xiang S, Pan C, Fua P (2019) A performance evaluation of local features for image-based 3d reconstruction. IEEE Transactions on Image Processing:1-16Google Scholar
  34. 34.
    Ojala, T., Pietikäinen, M., Harwood,D., 1996. A comparative study of texture measures with classification based on featured distribution. Pattern Recogn 29 (1): 51–59.  https://doi.org/10.1016/0031-3203(95)00067-4
  35. 35.
    Huang D, Shan C, Ardabilian M, Wang Y, Chen L (2011) Local binary patterns and its application to facial image analysis: a survey. IEEE Trans Syst Man Cybern Part C Appl Rev 46(1):765–781.  https://doi.org/10.1109/TSMCC.2011.2118750 CrossRefGoogle Scholar
  36. 36.
    Luo CH, Wen CY, Yuan CS, Liaw JL, Lo CC, Chiu SH (2005) Investigation of urban atmospheric visibility by high-frequency extraction: model development and field test. Atmos Environ 39:2545–2552.  https://doi.org/10.1016/j.atmosenv.2005.01.023 CrossRefGoogle Scholar
  37. 37.
    Pavlic M, Rigoll G, Ilic S (2013) Classification of images in fog and fog-free scenes for use in vehicles. 2013 IEEE Intelligent Vehicles Symposium (IV), Gold Coast, QLD, Australia: 1-6Google Scholar
  38. 38.
    Chen Z, Li J, Chen Q (2009) Real-time video detection of road visibility conditions. WRI World Congress on Computer Science and Information Engineering, Los Angeles, pp 472–476Google Scholar
  39. 39.
    Babari R, Hautière N, Dumont É, Paparoditis N, Misener J (2012) Visibility monitoring using conventional roadside cameras, emerging applications. Trans Res, part C : Emerg technol 22:17–28CrossRefGoogle Scholar
  40. 40.
    Babari R, Hautière N, Dumont É, Brémond R, Paparoditis N (2011) A model-driven approach to estimate atmospheric visibility with ordinary cameras. Atmos Environ 45(30):5316–5324.  https://doi.org/10.1016/j.atmosenv.2011.06.053 CrossRefGoogle Scholar
  41. 41.
    Hallowell R, Matthews M, Pisano P (2007) An automated visibility detection algorithm utilizing camera imagery. 23rd Conference on Interactive Information and Processing Systems for Meteorology, Oceanography and Hydrology, San Antonio, TX, Amer. Meteor. Soc: 1-15Google Scholar
  42. 42.
    Tian DP (2013) A review on image feature extraction and representation techniques. Int J Multimed Ubiquitous Eng 8(4):385–396Google Scholar
  43. 43.
    Nithin K, Sivakumar B (2015) Generic feature learning in computer vision. Proceedings of the second international symposium on computer vision and Internet. Procedia Computer Science 58:202–209CrossRefGoogle Scholar
  44. 44.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Processing Syst:1097–1105Google Scholar
  45. 45.
    He K,Gkioxari G, Dollar P, Girshick R (2018) Mask r-cnn. arXivpreprint arXiv:1703.06870. https://arxiv.org/pdf/1703.06870.pdf . Accessed 24 July 2018
  46. 46.
    Redmon J, Divvala S, Girshick R, Farhadi R (2016) You only look once: unified, real-time object detection. IEEE Conf. on ComputerVision and Pattern Recognition (CVPR):779–788  https://doi.org/10.1109/CVPR.2016.91
  47. 47.
    Deng J, Dong W, Socher R, Li-Jia L,Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR09):248-255.  https://doi.org/10.1109/CVPR.2009.5206848
  48. 48.
    Razavian A, Azizpour H, Sullivan J, Carlsson S (2014) CNN features off-the-shelf: an astounding baseline for recognition. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW): 512–519. https://doi.org/10.1109/CVPRW.2014.131
  49. 49.
    Liaw JJ, Lian SB, Chen RC (2009) Atmospheric visibility monitoring using digital image analysis techniques. In: Jiang X, Petkov N (eds) International Conference on Computer Analysis of Images and Patterns, Lecture Notes in Computer Science (LNCS), vol 5702, pp 1204–1211.  https://doi.org/10.1007/978-3-642-03767-2_146 CrossRefGoogle Scholar
  50. 50.
    Xie L, Chiu A, Newsam S (2008) Estimating atmospheric visibility using general-purpose cameras. In Bebis G (ed) International Symposium on Visual Computing, Part II. Lecture Notes in Computer Science (LNCS) 5359: 356–367.  https://doi.org/10.1007/978-3-540-89646-3_35
  51. 51.
    Ming-wei A, Zong-Liang G., Jibin L, Tao Z. (2010) Visibility detection based on traffic camera imagery. Proceedings of the Third International Conference on Information Sciences and Interaction Sciences (ICIS). Chengdu, China: 411-414Google Scholar
  52. 52.
    Campbell FW, Robson JG (1968) Application of Fourier analysis to the visibility of gratings. J Physiol 197(3):551–566CrossRefGoogle Scholar
  53. 53.
    Hagiwara T, Ota Y, Kaneda Y, Nagata Y, Araki K (2006) A method of processing CCTV digital images for poor visibility identification. Transp Res Rec 1973:95–104.  https://doi.org/10.1177/0361198106197300112 CrossRefGoogle Scholar
  54. 54.
    Krizhevsky A, Sutskever I., Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS), Lake Tahoe, Nevada 1: 1097-1105Google Scholar
  55. 55.
    Cortes C, Vapnik V (1995) Support-vector networks, machine learning 20 (3): 273–297  https://doi.org/10.1007/BF00994018
  56. 56.
  57. 57.
    Shannon CE, Weaver W (1949) The mathematical theory of communication. University of Illinois Press, UrbanazbMATHGoogle Scholar
  58. 58.
    Heikkilä M, Pietikäinen M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recogn 42(3):425–436.  https://doi.org/10.1016/j.patcog.2008.08.014 CrossRefzbMATHGoogle Scholar
  59. 59.
    Seewald AK, Fürnkranz J (2001) An evaluation of grading classifiers. In: Hoffmann F, Hand DJ, Adams N, Fisher D, Guimaraes G (eds) Advances in intelligent data analysis. IDA 2001. Lecture Notes in Computer Science, 2189. Springer, Berlin, Heidelberg.  https://doi.org/10.1007/3-540-44816-0_12 CrossRefGoogle Scholar
  60. 60.
    Bianco S, Cadene R, Celona L, Napoletano P (2018) Benchmark analysis of representative deep neural network architectures. IEEE Access 6(1):64270–64277.  https://doi.org/10.1109/ACCESS.2018.2877890 CrossRefGoogle Scholar
  61. 61.
    Kang H (2018) Accelerator-aware pruning for convolutional neural networks, CoRR, vol. abs/1804.09862. https://arxiv.org/pdf/1804.09862.pdf.
  62. 62.
    He Y, Dong X, Kang G, Fu Y, Yang Y (2019) Progressive deep neural networks acceleration via soft filter pruning”, arXiv:1808.07471v3, March 2019. https://arxiv.org/pdf/1808.07471.pdf. Accessed September 29 2019
  63. 63.
    Kamoun F, Chaabani H, Outay F, Yasar A (2020) A survey of approaches for estimating meteorological visibility distance under foggy weather conditions. In: Outay F, Yasar A, Shakshuki E (eds) Global advancements in connected and intelligent mobility: emerging research and opportunities, IGI Global, Chapter, vol 2, pp 65–92.  https://doi.org/10.4018/978-1-5225-9019-4.ch002 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Technological InnovationZayed UniversityDubaiUAE
  2. 2.Electrical and Computer Engineering DepartmentUniversity of TorontoTorontoCanada
  3. 3.ESPRIT School of EngineeringTunisTunisia
  4. 4.Department of Electrical Engineering and Computer SciencesKhalifa UniversityAbu DhabiUAE
  5. 5.Hasselt UniversityDiepenbeekBelgium

Personalised recommendations