Smoke vehicle detection based on multi-feature fusion and hidden Markov model

  • Huanjie Tao
  • Xiaobo LuEmail author
Original Research Paper


Existing smoke vehicle detection methods and vision-based smoke detection methods are vulnerable to false alarms. This paper presents an automatic smoke vehicle detection method based on multi-feature fusion and hidden Markov model (HMM). In this method, we first detect moving objects using an improved visual background extractor (ViBe) algorithm and obtain smoke-colored blocks using color histogram features in the HSI (hue, saturation, and intensity) color space. The adaptive scale local binary pattern (AS-LBP) and the discriminative edge orientation histogram (disEOH) are proposed and combined to characterize the smoke-colored blocks. More specifically, the proposed AS-LBP, a texture feature descriptor, is based on the quadratic fitting of our labelled data to obtain the best scale. The proposed disEOH, a gradient-based feature descriptor, is robust to noise by extracting discriminative edge information using Gaussian filters and principal component analysis (PCA). The discrete cosine transform (DCT) is employed to extract frequency domain information from the region fused by smoke blocks. To utilize the dynamic features, the HMMs are employed to analyze and classify the smoke-colored block sequences and region sequences in continuous frames. The experimental results show that the proposed method achieves better performances than existing smoke detection methods, especially achieves lower false alarms.


Smoke vehicle detection Local binary pattern Visual background extractor algorithm Edge orientation histogram Hidden Markov model Discrete cosine transform 



This work was supported by the National Natural Science Foundation of China (No. 61871123), Key Research and Development Program in Jiangsu Province (No. BE2016739), a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions, the Postgraduate Research and Practice Innovation Program of Jiangsu Province (No. KYCX18_0101), the Scientific Research Foundation of Graduate School of Southeast University (No. YBPY1871), and the State Scholarship Fund from China Scholarship Council.


  1. 1.
    Liu, Y.H., Liao, W.Y., Li, L., et al.: Vehicle emission trends in China’s Guangdong Province from 1994 to 2014. Sci. Total Environ. 3(15), 512–521 (2017)Google Scholar
  2. 2.
    Asano, I., Shinohara, M., Hamada, K.: Exhaust gas analysis system and exhaust gas analysis program, U.S. Patent 9 568 411 B2, Feb. 14, (2017)Google Scholar
  3. 3.
    Liu, H., Chen, S., Kubota, N.: Intelligent video systems and analytics: a survey. IEEE Trans. Ind. Inf. 9(3), 1222–1233 (2013)Google Scholar
  4. 4.
    Pyykonen, P., Peussa, P., Kutila, M., et al.: Multi-camera-based smoke detection and traffic pollution analysis system. Proc. Int. Conf. Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, 2016, pp. 233–238Google Scholar
  5. 5.
    Tao, H., Lu, X.: Smoke vehicle detection based on multi-scale block Tamura features. Signal Image Video Process. 12(6), 1061–1068 (2018)Google Scholar
  6. 6.
    Tao, H., Lu, X.: Smoke vehicle detection based on multi-feature fusion and ensemble neural networks. Multimed. Tools Appl. 77(24), 32153–32177 (2018)Google Scholar
  7. 7.
    Tao, H., Lu, X.: Smoke vehicle detection in surveillance video based on gray level co-occurrence matrix. in Proc. Int. Conf. on Digital Image Processing, Shanghai, SPIE, vol. 10806, id.1080642, pp. 1–7, Aug. 2018Google Scholar
  8. 8.
    Tao, H., Lu, X.: Automatic smoky vehicle detection from traffic surveillance video based on vehicle rear detection and multi-feature fusion” IET Intel. Transport Syst. (2018). Google Scholar
  9. 9.
    Tao, H., Lu, X.: Contour-based smoke vehicle detection from surveillance video for alarm systems. SIViP. (2018). Google Scholar
  10. 10.
    Tao, H., Lu, X.: Smoky vehicle detection based on range filtering on three orthogonal planes and motion orientation histogram. IEEE Access. 6(1), 57180–57190, (2018)Google Scholar
  11. 11.
    Saponara, S., Pilato, L., Fanucci, L.: Early video smoke detection system to improve fire protection in rolling stocks. Proc. SPIE 9139(913903), 9 (2014)Google Scholar
  12. 12.
    Saponara, S., Pilato, L., Fanucci, L.: Exploiting CCTV camera system for advanced passenger services on-board trains. IEEE Int. Smart Cities Conf. pp. 1–6 (2016)Google Scholar
  13. 13.
    Gunay, O., Toreyin, B.U., Kose, K., et al.: ‘Entropy-functional-based online adaptive decision fusion framework with application to wildfire detection in video. IEEE Trans. Image Process. 21(5), 2853–2865 (2012)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Kolesov, I., Karasev, P., Tannenbaum, A., et al.: ‘Fire and smoke detection in video with optimal mass transport based optical flow and neural networks,” in Proc. IEEE International Conference on Image Processing, 2010, pp. 761–764Google Scholar
  15. 15.
    Wang, S., He, Y., Yang, H., et al.: Video smoke detection using shape, color and dynamic features. J. Intell. Fuzzy Syst. 33(1), 305–313 (Feb. 2017)Google Scholar
  16. 16.
    Calderara, S., Piccinini, P., Cucchiara, R.: Vision based smoke detection system using image energy and color information. Mach. Vis Appl. 22(4), 705–719, (2011)Google Scholar
  17. 17.
    Jakovcevic, T., Stipanicev, D., Krstinic, D.: Visual spatial-context based wildfire smoke sensor. Mach. Vis. Appl. 24(4), 707–719 (2013)Google Scholar
  18. 18.
    Millan-Garcia, L., Sanchez-Perez, G., Nakano, M., et al.: An early fire detection algorithm using IP cameras. Sensors 12(5), 5670–5686 (2012)Google Scholar
  19. 19.
    Prema, C.E., Vinsley, S.S., Suresh, S.: Multi feature analysis of smoke in YUV color space for early forest fire detection. Fire Technol 52(5), 1319–1342 (2016)Google Scholar
  20. 20.
    Ugur-Töreyin, B., Dedeoglu, Y., Enis-Çetin, A.: Contour Based smoke detection in video using wavelets. Proceedings of European Signal Processing Conference; Florence, Italy. 4–8 September 2006Google Scholar
  21. 21.
    Yu, C., Faon, J., Wang, J., et al.: Video fire smoke detection using motion and color features. Fire Technol 46, 651–663 (2010)Google Scholar
  22. 22.
    Ko, B., Park, J., Nam, J.Y.: Spatiotemporal bag-of-features for early wildfire smoke detection. Image Vis. Comput. 31(10), 786–795, (2013)Google Scholar
  23. 23.
    Toreyin, B.U., Dedeoglu, Y., Cetin, A.E.: Wavelet based real-time smoke detection in video. in Proc. European Signal Processing Conference, (2005)Google Scholar
  24. 24.
    Wang, Y., Chua, T.W., Chang, R., et al.: Real-time smoke detection using texture and color features. In Proc. International Conference on Pattern Recognition, pp. 1727–1730 (2012)Google Scholar
  25. 25.
    Tian, H., Li, W., Ogunbona, P., et al.: Smoke detection in videos using non-redundant local binary pattern-based features. In Proc. IEEE International Workshop on Multimedia Signal Processing, pp. 1–4 (2011)Google Scholar
  26. 26.
    Yuan, F.: Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf. J. 46(3), 132–139, (2011)Google Scholar
  27. 27.
    Lin, G., Zhang, Y., Zhang, Q., et al.: Smoke detection in video sequences based on dynamic texture using volume local binary patterns. Ksii Trans. Internet Inf. Syst. 11(11), 5522–5536 (2017)MathSciNetGoogle Scholar
  28. 28.
    Favorskaya, M., Pyataeva, A., Popov, A.: Verification of smoke detection in video sequences based on spatio-temporal local binary patterns. Proc. Comput. Sci. 60(1), 671–680 (2015)Google Scholar
  29. 29.
    Yuan, F., Shi, J., Xia, X., et al.: High-order local ternary patterns with locality preserving projection for smoke detection and image classification. Inf. Sci. 372(C), 225–240 (2016)Google Scholar
  30. 30.
    Datondji, S.R.E., Dupuis, Y., Subirats, P.: A survey of vision-based traffic monitoring of road intersections. IEEE Trans. Intell. Transp. Syst. 17(10), 2681–2698 (2016)Google Scholar
  31. 31.
    Barnich, O., Droogenbroeck, M.V.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Ojala, T., Pietikainen, M., Maenpaa, T.T.: Multiresolution gray-scale and rotation invariant texture classification with local binary pattern. IEEE Trans. Pattern Analysis Mach. Intell. 7(24), 971–987 (2002)zbMATHGoogle Scholar
  33. 33.
    Li, Z., Liu, G., Yang, Y., et al.: Scale- and rotation-invariant local binary pattern using scale-adaptive texton and subuniform-based circular shift. IEEE Trans. Image Process 21(4), 2874–2886 (2012)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Guo, Z.H., Zhang, L., Zhang, D.: Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recognit 43(3), 706–719 (2010)zbMATHGoogle Scholar
  35. 35.
    Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process. 19(6), 1657 (2010)MathSciNetzbMATHGoogle Scholar
  36. 36.
    Thanh, N.D., Ogunbona, P.O., Li, W.: A novel shape-based non-redundant local binary pattern descriptor for object detection. Pattern Recogn. 46(5), 1485–1500 (2013)Google Scholar
  37. 37.
    Zhao, G., Ahonen, T., Matas, J., et al.: Rotation-invariant image and video description with local binary pattern features. IEEE Trans. Image Process. 21(4), 1465–1477 (2012)MathSciNetzbMATHGoogle Scholar
  38. 38.
    Zhu, C., Wang, R.: Local multiple patterns based multiresolution gray-scale and rotation invariant texture classification. Inf. Sci. 187(1), 93–108 (2012)Google Scholar
  39. 39.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)MathSciNetGoogle Scholar
  40. 40.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings IEEE International Conference Computer Vision and Pattern Recognition, 2005, Vol. 1, pp. 886–893Google Scholar
  41. 41.
    Levi, K., Weiss Y.: Learning object detection from a small number of examples: the importance of good features. Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit. 2, 53–60Google Scholar
  42. 42.
    Baum, E., Petrie, T., Soules, G., Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Stat. 41, 164–171 (1970)MathSciNetzbMATHGoogle Scholar
  43. 43.
    Ronao, C., Ann, Cho, S.B.: Human activity recognition using smartphone sensors with two-stage continuous hidden Markov models. Int. Conf. Nat. Comput. IEEE. 681–686 (2014)Google Scholar
  44. 44.
    Hu, J., Brown, M.K., Turin, W.: HMM based on-line hand-writing recognition. IEEE Trans. Pattern Anal. Mach. Intell. 18(10), 1039–1045 (1996)Google Scholar
  45. 45.
    Lee, L.M., Jean, F.R.: High-order hidden Markov model for piecewise linear processes and applications to speech recognition. J. Acoust. Soc. Am. 140(2), EL204 (2016)Google Scholar
  46. 46.
    Lawrence, R., Rabiner, A.: Tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)Google Scholar
  47. 47.
    Yuan, F.: A double mapping framework for extraction of shape-invariant features based on multi-scale partitions with AdaBoost for video smoke detection. Pattern Recogn. 45(12), 4326–4336 (2012)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of AutomationSoutheast UniversityNanjingChina
  2. 2.Key Laboratory of Measurement and Control of Complex Systems of EngineeringMinistry of Education, Southeast UniversityNanjingChina

Personalised recommendations