Dual Deep Learning Model for Image Based Smoke Detection

  • Arun Singh PundirEmail author
  • Balasubramanian Raman


Image-based smoke detection could help in faster and robust detection and monitoring of wildfires. It is becoming the best alternate of sensor based detectors for early detection of wildfire. The limitations of sensor based detector is that, they need close vicinity to fire for raising the alarm which make them vulnerable in case of detecting far-distant wild fire. Hence, vision based detection system which utilizes the surveillance cameras which shows more fastness and robustness as compared to sensor based detectors. These cameras when installed on hill top or mobile tower can raise the early alarm for any possibility of smoke present in the frames of videos whether near-by or far-away smoke. The proposed work presents a robust method for smoke detection which, utilizes a dual deep learning framework. The proposed architecture makes use of framework based on Deep Convolutional Neural Networks, which has proven their supremacy in object recognition tasks. The first deep learning framework is employed for extracting the image-based features from smoke patches, which are being extracted using superpixel algorithm. We have employed total of 20,000 frames with equally distribution of non smoke and smoke classes, out of which 6000 frames are utilized for testing purpose and 14,000 are used for fine tuning purpose. These features are comprised of smoke-color, smoke-texture, sharp edge detection and perimeter disorder analysis. The second deep learning framework is used for extracting motion-based features such as moving region of smoke, growing region and rising region detection. Optical flow method is employed, in order to capture the random motion of smoke. These extracted optical flow are then feed into Deep CNN for extracting motion based features. Features from both the framework are combined to train the Support Vector Machine and end to end classification which is CNN classifier. Accuracy on the nearby smoke and faraway smoke is \(98.29\%\) and \(91.96\%\) respectively. Testing on different varieties of non-smoke videos such as clouds, fog, sandstorm and images of cloud on water, method proves its precision and robustness. The average accuracy in all the scenarios is \(97.49\%\) which outperforms the state of the art method for these scenarios. Contribution of this work lies in the fact that we have given 20,000 frames based smoke and non-smoke dataset and secondly our method outperform the existing method on challenging imaging conditions.


Smoke detection Deep learning Deep CNN Superpixel segmentation Optical flow 



  1. 1.
    Achanta, R, Shaji, A, Smith, K, Lucchi, A, Fua, P, Süsstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282CrossRefGoogle Scholar
  2. 2.
    Amiaz T, Fazekas S, Chetverikov D, Kiryati N (2007) Detecting regions of dynamic texture. In: International conference on scale space and variational methods in computer vision, SSVMCV, pp 848–859. SpringerGoogle Scholar
  3. 3.
    Añazco EV, Lopez PR, Lee S, Byun K, Kim TS (2018) Smoking activity recognition using a single wrist IMU and deep learning light. In: Proceedings of the 2nd international conference on digital signal processing, pp 48–51. ACMGoogle Scholar
  4. 4.
    Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828CrossRefGoogle Scholar
  5. 5.
    Calderara S, Piccinini P, Cucchiara R (2008) Smoke detection in video surveillance: a MoG model in the wavelet domain. In: Computer vision systems, pp 119–128Google Scholar
  6. 6.
    Celik T (2010) Fast and efficient method for fire detection using image processing. ETRI J 32(6):881–890CrossRefGoogle Scholar
  7. 7.
    Celik T, Demirel H, Ozkaramanli H, Uyguroglu M (2007) Fire detection using statistical color model in video sequences. J Vis Commun Image Represent 18(2):176–185CrossRefGoogle Scholar
  8. 8.
    Çelik T, Özkaramanlı H, Demirel H (2007) Fire and smoke detection without sensors: image processing based approach. In: 15th European signal processing conference, pp 1794–1798. IEEEGoogle Scholar
  9. 9.
    Chen J, He Y, Wang J (2010) Multi-feature fusion based fast video flame detection. Build Environ 45(5):1113–1122CrossRefGoogle Scholar
  10. 10.
    Chen TH, Wu PH, Chiou YC (2004) An early fire-detection method based on image processing. In: International conference on image processing, ICIP, vol 3, pp 1707–1710. IEEEGoogle Scholar
  11. 11.
    Chetverikov D, Péteri R (2005) A brief survey of dynamic texture description and recognition. In: Computer recognition systems, pp 17–26. Springer, Berlin, HeidelbergGoogle Scholar
  12. 12.
    Chino DY, Avalhais LP, Rodrigues JF, Traina AJ (2015) Bowfire: detection of fire in still images by integrating pixel color and texture analysis. In: 28th SIBGRAPI conference on graphics, patterns and images, SIBGRAPI, pp 95–102. IEEEGoogle Scholar
  13. 13.
    Cui Y, Dong H, Zhou E (2008) An early fire detection method based on smoke texture analysis and discrimination. In: Congress on image and signal processing, CISP, vol 3, pp 95–99. IEEEGoogle Scholar
  14. 14.
    Dimitropoulos K, Barmpoutis P, Grammalidis N (2015) Spatio-temporal flame modeling and dynamic texture analysis for automatic video-based fire detection. IEEE Trans Circuits Syst Video Technol 25(2):339–351CrossRefGoogle Scholar
  15. 15.
    Dimitropoulos K, Barmpoutis P, Grammalidis N (2017) Higher order linear dynamical systems for smoke detection in video surveillance applications. IEEE Trans Circuits Syst Video Technol 27(5):1143–1154CrossRefGoogle Scholar
  16. 16.
    Doretto G, Chiuso A, Wu YN, Soatto S (2003) Dynamic textures. Int J Comput Vis 51(2):91–109CrossRefzbMATHGoogle Scholar
  17. 17.
    Enis Cetin A, Porikli F (2011) Special issue on dynamic textures in video. Mach Vis Appl 22(5):739–740CrossRefGoogle Scholar
  18. 18.
    Günay O, Taşdemir K, Töreyin BU, Çetin AE (2010) Fire detection in video using lMS based active learning. Fire Technol 46(3):551–577CrossRefGoogle Scholar
  19. 19.
    Horn BK, Schunck BG (1981) Determining optical flow. Artif Intell 17(1–3):185–203CrossRefGoogle Scholar
  20. 20.
    Hu Y, Lu X (2018) Real-time video fire smoke detection by utilizing spatial-temporal ConvNet features. Multimed Tools Appl 77:1–19CrossRefGoogle Scholar
  21. 21.
    Huang FJ, Boureau YL, LeCun Y et al (2007) Unsupervised learning of invariant feature hierarchies with applications to object recognition. In: IEEE conference on computer vision and pattern recognition, CVPR, pp 1–8. IEEEGoogle Scholar
  22. 22.
    Huang FJ, LeCun Y (2006) Large-scale learning with SVM and convolutional for generic object categorization. In: Computer Society conference on computer vision and pattern recognition, CVPR, vol 1, pp 284–291. IEEEGoogle Scholar
  23. 23.
    Jarrett K, Kavukcuoglu K, LeCun Y et al (2009) What is the best multi-stage architecture for object recognition? In: 12th International conference on computer vision, ICCV, pp 2146–2153. IEEEGoogle Scholar
  24. 24.
    Kaabi R, Sayadi M, Bouchouicha M, Fnaiech F, Moreau E, Ginoux JM (2018) Early smoke detection of forest wildfire video using deep belief network. In: 4th International conference on advanced technologies for signal and image processing (ATSIP), pp 1–6. IEEEGoogle Scholar
  25. 25.
    Ko B, Cheong KH, Nam JY (2010) Early fire detection algorithm based on irregular patterns of flames and hierarchical Bayesian networks. Fire Saf J 45(4):262–270CrossRefGoogle Scholar
  26. 26.
    Kolesov I, Karasev P, Tannenbaum A, Haber E (2010) Fire and smoke detection in video with optimal mass transport based optical flow and neural networks. In: 17th IEEE international conference on image processing, ICIP, pp 761–764. IEEEGoogle Scholar
  27. 27.
    Kopilovic I, Vagvolgyi B, Szirányi T (2000) Application of panoramic annular lens for motion analysis tasks: surveillance and smoke detection. In: 15th International conference on pattern recognition, ICPR, vol 4, pp 714–717. IEEEGoogle Scholar
  28. 28.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105. Lake Tahoe, NevadaGoogle Scholar
  29. 29.
    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444CrossRefGoogle Scholar
  30. 30.
    LeCun Y, Boser BE, Denker JS, Henderson D, Howard RE, Hubbard WE, Jackel LD (1990) Handwritten digit recognition with a back-propagation network. In: Advances in neural information processing systems, pp 396–404Google Scholar
  31. 31.
    LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRefGoogle Scholar
  32. 32.
    LeCun Y, Kavukcuoglu K, Farabet C (2010) Convolutional networks and applications in vision. In: International symposium on circuits and systems, ISCAS, pp 253–256. IEEEGoogle Scholar
  33. 33.
    Owrutsky JC, Steinhurst DA, Minor CP, Rose-Pehrsson SL, Williams FW, Gottuk DT (2006) Long wavelength video detection of fire in ship compartments. Fire Saf J 41(4):315–320CrossRefGoogle Scholar
  34. 34.
    Phillips Iii W, Shah M, da Vitoria Lobo N (2002) Flame recognition in video. Pattern Recognit Lett 23(1):319–327CrossRefzbMATHGoogle Scholar
  35. 35.
    Piccinini P, Calderara S, Cucchiara R (2008) Reliable smoke detection in the domains of image energy and color. In: 15th IEEE international conference on image processing, ICIP, pp 1376–1379. IEEEGoogle Scholar
  36. 36.
    Prema CE, Vinsley S, Suresh S (2018) Efficient flame detection based on static and dynamic texture analysis in forest fire detection. Fire Technol 54(1):255–288CrossRefGoogle Scholar
  37. 37.
    Pundir AS, Raman B (2017) Deep belief network for smoke detection. Fire Technol 53(6):1943–1960CrossRefGoogle Scholar
  38. 38.
    Qi X, Ebert J (2009) A computer vision based method for fire detection in color videos. Int J Imaging 2(S09):22–34Google Scholar
  39. 39.
    Serre T, Wolf L, Poggio T (2005) Object recognition with features inspired by visual cortex. In: Computer Society conference on computer vision and pattern recognition, CVPR, vol 2, pp 994–1000. IEEEGoogle Scholar
  40. 40.
    Toreyin B, Dedeoglu Y, Cetin AE, Fazekas S, Chetverikov D, Amiaz T, Kiryati, N (2007) Dynamic texture detection, segmentation and analysis. In: 6th ACM international conference on image and video retrieval, CIVR, pp 131–134. ACMGoogle Scholar
  41. 41.
    Töreyin BU, Dedeoğlu Y, Güdükbay U, Cetin AE (2006) Computer vision based method for real-time fire and flame detection. Pattern Recognit Lett 27(1):49–58CrossRefGoogle Scholar
  42. 42.
    Vicente J, Guillemant P (2002) An image processing technique for automatically detecting forest fire. Int J Therm Sci 41(12):1113–1120CrossRefGoogle Scholar
  43. 43.
    Xiong Z, Caballero R, Wang H, Finn AM, Lelic MA, Peng PY (2007) Video-based smoke detection: possibilities, techniques, and challenges. In: IFPA, fire suppression and detection research and applications—a technical working conference (SUPDET), Orlando, FLGoogle Scholar
  44. 44.
    Xu Z, Xu J (2007) Automatic fire smoke detection based on image visual features. In: International conference on computational intelligence and security workshops, CISW, pp 316–319. IEEEGoogle Scholar
  45. 45.
    Ye W, Zhao J, Wang S, Wang Y, Zhang D, Yuan Z (2015) Dynamic texture based smoke detection using surfacelet transform and HMT model. Fire Saf J 73:91–101CrossRefGoogle Scholar
  46. 46.
    Yin M, Lang C, Li Z, Feng S, Wang T (2018) Recurrent convolutional network for video-based smoke detection. Multimed Tools Appl 78:1–20Google Scholar
  47. 47.
    Yuan F (2008) A fast accumulative motion orientation model based on integral image for video smoke detection. Pattern Recognit Lett 29(7):925–932CrossRefGoogle Scholar
  48. 48.
    Yuan F (2011) Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf J 46(3):132–139CrossRefGoogle Scholar
  49. 49.
    Zhang QX, Lin GH, Zhang YM, Xu G, Wang JJ (2018) Wildland forest fire smoke detection based on faster R-CNN using synthetic smoke images. Procedia Eng 211:441–446CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringIIT RoorkeeRoorkeeIndia

Personalised recommendations