Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Encoding features from multi-layer Gabor filtering for visual smoke recognition

  • 2 Accesses

Abstract

It is a challenging task to accurately recognize smoke from visual scenes due to large variations in smoke shape, color and texture. To improve recognition accuracy, we propose a framework mainly with a robust local feature extraction module based on Gabor convolutional networks. We first propose a Gabor convolutional network, each layer of which mainly consists of Gabor convolution and feature fusion. To fuse feature maps generated by Gabor convolution, we present three Basic Grouping Methods, which divide the feature maps into several groups along orientation axis, scale axis and both of them. To avoid exponential growth of feature maps and preserve discriminative information simultaneously, we propose three element-wise aggregation functions, which are mean, min and max, to combine feature maps in each group. To further improve efficiency, we use local binary pattern to encode hidden and output maps of Gabor convolutional layers. In addition, we use a weight vector to optimize concatenation of histograms for further improvement. Experiments show that our method achieves very outstanding results on smoke, texture and material image datasets. Although the feature extraction step of our method is training-free, our framework consistently outperforms state-of-the-art methods on small smoke datasets, even including deep learning-based methods.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. 1.

    Hwang U, Jeong J (2016) Computer vision based smoke detection method by using colour and object tracking. In: Proceedings of 5th international conference on information technology convergence and services (ITCS), Sydney, pp 53–59

  2. 2.

    Jia Y, Yuan J, Wang J, Fang J, Zhang Q, Zhang Y (2016) A saliency-based method for early smoke detection in video sequences. Fire Technol 52(5):1–22

  3. 3.

    Yu C, Fang J, Wang J, Zhang Y (2012) Video fire smoke detection using motion and color features. Fire Technol 46(3):651–663

  4. 4.

    Hu Y, Lu X (2018) Real-time video fire smoke detection by utilizing spatial-temporal ConvNet features. Multimed Tools Appl 77(8):1–19

  5. 5.

    Yuan F (2008) A fast accumulative motion orientation model based on integral image for video smoke detection. Pattern Recognit Lett 29(7):925–932

  6. 6.

    Yuan F (2012) A double mapping framework for extraction of shape-invariant features based on multi-scale partitions with AdaBoost for video smoke detection. Pattern Recognit 45(12):4326–4336

  7. 7.

    Maruta H, Nakamura A, Kurokawa F (2010) A new approach for smoke detection with texture analysis and support vector machine. In: Proceedings of IEEE international symposium industrial electronics, Bari, Italy, pp 1550–1555

  8. 8.

    Nguyen T, Kim J (2013) Multistage optical smoke detection approach for smoke alarm systems. Opt Eng 52(5):1–12

  9. 9.

    Yuan F (2011) Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf J 46(3):132–139

  10. 10.

    Yuan F, Shi J, Xia X, Fang Y, Fang Z, Mei T (2016) High-order local ternary patterns with locality preserving projection for smoke detection and image classification. Inf Sci 372:225–240

  11. 11.

    Yuan F, Shi J, Xia X, Yang Y, Fang Y, Wang R (2016) Sub oriented histograms of local binary patterns for smoke detection and texture classification. KSII Trans Internet Inf Syst 10(4):1807–1823

  12. 12.

    Yuan F, Xia X, Shi J, Li H, Li G (2017) Non-linear dimensionality reduction and Gaussian process based classification method for smoke detection. IEEE Access 5:6833–6841

  13. 13.

    Zhao Y, Zhou Z, Xu M (2015) Forest fire smoke video detection using spatiotemporal and dynamic texture features. J Electr Comput Eng 2015:1–7

  14. 14.

    Calderara S, Piccinini P, Cucchiara R (2011) Vision based smoke detection system using image energy and color information. Mach Vis Appl 22:705–719

  15. 15.

    Gonzalez-Gonzalez R, Alarcon-Aquino V, Rosas-Romero R, Starostenko O, Rodriguez-Asomoza J, Ramirez-Cortes J (2010) Wavelet-based smoke detection in outdoor video sequences. In: Proceedings of 53rd international midwest symposium on circuits and systems, midwest symposium on circuits and systems conference proceedings, IEEE, pp 383–387

  16. 16.

    Gubbi J, Marusica S, Palaniswamia M (2009) Smoke detection in video using wavelets and support vector machines. Fire Saf J 44(8):1110–1115

  17. 17.

    Maruta H, Kato Y, Nakamura A, Kurokawa F (2009) Smoke detection in open areas using its texture features and time series properties. In: Proceedings of IEEE international symposium on industrial electronics (ISIE), IEEE, pp 1904–1908

  18. 18.

    Toreyin BU, Dedeoglu Y, Cetin A (2005) Wavelet based real-time smoke detection in video. In: Proceedings of European signal processing conference, Antalya, pp 4–8

  19. 19.

    Ye S, Bai Z, Chen H, Bohush R, Ablameyko S (2017) An effective algorithm to detect both smoke and flame using color and wavelet analysis. Pattern Recognit Image Anal 27(1):131–138

  20. 20.

    Tung T, Kim J (2011) An effective four-stage smoke-detection algorithm using video images for early fire-alarm systems. Fire Saf J 46(5):276–282

  21. 21.

    Yin Z, Wan B, Yuan F, Xia X, Shi J (2017) A deep normalization and convolutional neural network for image smoke detection. IEEE Access 5:18429–18438

  22. 22.

    Yuan F, Fang Z, Wu S, Yang Z, Fang Y (2015) A real-time video smoke detection using staircase searching based dual threshold AdaBoost and dynamic analysis. IET Image Proc 9(10):849–856

  23. 23.

    Dora L, Agrawal S, Panda R, Abraham A (2017) An evolutionary single Gabor kernel based filter approach to face recognition. Eng Appl Artif Intell 62:286–301

  24. 24.

    Liu C, Wechsler H (2002) Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Trans Image Process 11:467–476

  25. 25.

    Su Y, Shan S, Chen X, Gao W (2009) Hierarchical ensemble of global and local classifiers for face recognition. IEEE Trans Image Process 18:1885–1896

  26. 26.

    Hwang W, Huang X, Li S, Kim J (2015) Face recognition using extended curvature Gabor classifier bunch. Pattern Recognit 48(4):1247–1260

  27. 27.

    Li C, Duan G, Zhong F (2015) Rotation invariant texture retrieval considering the scale dependence of Gabor wavelet. IEEE Trans Image Process 24(8):2344–2354

  28. 28.

    Jie Z, Qiang J, Nagy G (2007) A comparative study of local matching approach for face recognition. IEEE Trans Image Process 16:2617–2628

  29. 29.

    Cament L, Castillo L, Perez J, Galdames F, Perez C (2014) Fusion of local normalization and Gabor entropy weighted features for face identification. Pattern Recognit 47(2):568–577

  30. 30.

    Shen L, Bai L, Auer D (2008) 3D Gabor wavelets for evaluating SPM normalization algorithm. Med Image Anal 12(3):375–383

  31. 31.

    Zhan W, Shan S, Gao W, Chen X, Zhang H (2005) Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: Proceedings of 10th IEEE international conference on computer vision (ICCV), pp 786–791

  32. 32.

    Elbakary M, Sundareshan M (2005) Accurate representation of local frequency using a computationally efficient Gabor filter fusion approach with application to image registration. Pattern Recognit Lett 26(14):2164–2173

  33. 33.

    Wang H, Qi H, Wang X (2013) A new Gabor based approach for wood recognition. Neurocomputing 116:192–200

  34. 34.

    Zhang B, Shan S, Chen X, Gao W (2007) Histogram of Gabor phase patterns (HGPP): a novel object representation approach for face recognition. IEEE Trans Image Process 16:57–68

  35. 35.

    Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Proceedings of advances in neural information processing systems, vol 25, pp 1097–1105

  36. 36.

    Zeiler M, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proceedings of European conference on computer vision (ECCV), pp 818–833

  37. 37.

    Balntas V, Riba E, Ponsa D, Mikolajczyk K (2016) Learning local feature descriptors with triplets and shallow convolutional neural networks. In: Proceedings of British machine vision conference (BMVC), vol 1, No. 2. pp 119.1–119.11

  38. 38.

    Hu Y, Li C, Hu D, Yu W (2016) Gabor feature based convolutional neural network for object recognition in natural scene. In: Proceedings of international conference on information science and control engineering, IEEE, pp 386–390

  39. 39.

    Wiskott L, Fellous JM, Kuiger N, Malsburg CVD (1997) Face recognition by elastic bunch graph matching. IEEE Trans Pattern Anal Mach Intell 19:775–779

  40. 40.

    Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary pattern. Eur Conf Comput Vis 24(7):404–420

  41. 41.

    Caputo B, Hayman E, Mallikarjuna P (2005) Class-specific material categorisation. In: Proceedings of 10th IEEE international conference on computer vision, IEEE, vol 2, pp 1597–1604

  42. 42.

    Guo Z, Zhang L, Zhang D (2010) A completed modeling of local binary pattern operator for texture classification. IEEE Trans Image Process 19(6):1657–1663

  43. 43.

    Murala S, Maheshwari R, Balasubramanian R (2012) Local tetra patterns: A new feature descriptor for content-based image retrieval. IEEE Trans Image Process 21(5):2874–2886

  44. 44.

    Qian X, Hua X, Chen P, Ke L (2011) PLBP: An effective local binary patterns texture descriptor with pyramid representation. Pattern Recognit 44(10–11):2502–2515

  45. 45.

    Zhang B, Gao Y, Zhao S, Liu J (2013) Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE Trans Image Process 19(2):533–544

  46. 46.

    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. https://arxiv.org/abs/1409.1556

  47. 47.

    Qi X, Xiao R, Li C, Qiao Y, Guo J, Tang X (2014) Pairwise rotation invariant co-occurrence local binary pattern. IEEE Trans Pattern Anal Mach Intell 36(11):2199–2211

  48. 48.

    Ahonen T, Matas J, He C, Pietikäinen M (2009) Rotation invariant image description with local binary pattern histogram Fourier features. In: Proceedings of 16th image analysis, Scandinavian conference, Oslo, Norway, pp 61–70

  49. 49.

    Zhang J, Liang J, Zhao H (2013) Local energy pattern for texture classification using self-adaptive quantization thresholds. IEEE Trans Image Process 22(1):31–42

  50. 50.

    Crosier M, Griffin LD (2010) Using basic image features for texture classification. Int J Comput Vis 88(3):447–460

  51. 51.

    Sharma G, Hussain SU, Frédéric J (2012) Local higher-order statistics (LHS) for texture categorization and facial analysis. In: Proceedings of computer vision—European conference on computer vision. Springer, Berlin, pp 1–12

  52. 52.

    Hong X, Zhao G, Pietikäinen M, Chen X (2014) Combining LBP difference and feature correlation for texture description. IEEE Trans Image Process 23(6):2557–2568

  53. 53.

    Ryu J, Hong S, Yang H (2015) Sorted consecutive local binary pattern for texture classification. IEEE Trans Image Process 24(7):2254–2265

  54. 54.

    Mehta R, Egiazarian K (2016) Texture classification using dense micro-block difference. IEEE Trans Image Process 25(4):1604–1616

  55. 55.

    Chen J, Shan S, He C, Zhao G, Pietikainen M, Chen X (2010) WLD: a robust local image descriptor. IEEE Trans Pattern Anal Mach Intell 32(9):1705–1720

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (61862029), the Open Project of Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering (2018SDSJ01), the Natural Science Foundation of Jiangxi Province (20192BAB207011) and the Science Technology Projects of Jiangxi Education Department (GJJ170317, GJJ170892, GJJ180206).

Author information

Correspondence to Feiniu Yuan or Gang Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yuan, F., Li, G., Xia, X. et al. Encoding features from multi-layer Gabor filtering for visual smoke recognition. Pattern Anal Applic (2020). https://doi.org/10.1007/s10044-020-00864-x

Download citation

Keywords

  • Smoke recognition
  • Gabor transform
  • Gabor convolutional network
  • Local binary pattern (LBP)
  • Basic grouping method
  • Aggregation function