Improved background modeling of video sequences using spatio-temporal extension of fuzzy local binary pattern

  • Akram Norouzi Sefidmazgi
  • Manoochehr NahviEmail author


Background subtraction is a method of motion detection in video sequences captured by static camera based on construction of a background model and its progressive comparison with each frame of the video. Sometimes the changes in the background objects are not permanent and appear at a rate faster than that of the background update, and this leads to emergence of dynamic textures in the background. As a result, high-quality background modeling plays a major role in motion detection performance, especially in videos with dynamic backgrounds and adverse environmental conditions such as noise. Although Local Binary Pattern (LBP) is a successful methodology for background subtraction, but it cannot properly extract textures from uniform areas of the foreground. In recent years, Fuzzy Local Binary Pattern (FLBP) has been developed to improve the performance of LBP operator in texture extraction from images with additive noise. The use of FLBP texture descriptor for background subtraction has led to improved robustness against noise and low sensitivity of the background model to slight changes in the texture gray scale values, and therefore better texture extraction from uniform areas, even in the presence of dynamic backgrounds and adverse environmental conditions. But despite this improvement, this operator is still sensitive to time-variations of pixels in dynamic backgrounds. To avoid this issue and incorporate correlation of pixel values over successive frames, this study proposes the use of spatio-temporal extension of FLBP with symmetry about central pixel in combination with the Local Color Histogram (LCH) in the Improved Hue Luminance and Saturation (IHLS) color space for describing the pixel color features. The results of tests conducted with standard databases show that for dynamic backgrounds, the use of proposed Spatio-Temporal Fuzzy Center Symmetric Local Binary Pattern (STFCS-LBP) operator with the spatio-temporal neighborhood texture patterns and the local color histogram yields better results than the existing methods.


Background subtraction Video analysis Background modeling Fuzzy local binary pattern Dynamic texture IHLS color space 



  1. 1.
    Ayub M (2009) Choquet and Sugeno integrals. Thesis for the degree Master of Science in Mathematical Modeling and SimulationGoogle Scholar
  2. 2.
    Barnich O, Droogenbroeck MV (2011) ViBe : a universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20(6):1709–1724MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bianco S, Ciocca G, Schettini R (2017) Combination of video change detection algorithms by genetic programming. IEEE Trans Evol Comput 21(6):914–928CrossRefGoogle Scholar
  4. 4.
    Bilodeau G-A, Jodoin J-P, Saunier N (2013) Change detection in feature space using local binary similarity patterns. Proc Int Conf Comput Robot Vision (CRV): 106–112Google Scholar
  5. 5.
    Bouwmans T, Porikli F, Horferlin B, Vacavant A (2014) Handbook on background modeling and foreground detection for video surveillance. CRC press, Taylor and Francis groupGoogle Scholar
  6. 6.
    Brutzer S, Hӧferlin B, Heidemann G (2011) Evaluation of background subtraction techniques for video surveillance. Proc IEEE Conf Comput Vision Pattern Recogn (CVPR): 1937–1944Google Scholar
  7. 7.
    Candès EJ, Li X, Ma Y, Wright J (2011) Robust principal component analysis? J ACM 58(3):11MathSciNetCrossRefGoogle Scholar
  8. 8.
    Cheng F, Huang S, Ruan S (2011) Illumination-sensitive background modeling approach for accurate moving object detection. IEEE Trans Broadcast 57(4):794–801CrossRefGoogle Scholar
  9. 9.
    Chua T W, Leman K, Wang Y (2012) Fuzzy rule-based system for dynamic texture and color based background subtraction. 2012 IEEE Int Conf Fuzzy Syst (FUZZ-IEEE), 1559–1565Google Scholar
  10. 10.
    El Baf F, Bouwmans T, Vachon B (2008) Fuzzy integral for moving object detection. Proc IEEE Int Conf Fuzzy Syst: 1729–1736Google Scholar
  11. 11.
    Elgammal A, Harwood D, Davis L (2000) Non-parametric model for background subtraction. Proc 5th Eur Conf Comput Vision 1843Google Scholar
  12. 12.
    Gemignani G, Rozza A (2016) A robust approach for the background subtraction based on multi-layered self-organizing maps. IEEE Trans Image Process 25(11):5239–5251MathSciNetCrossRefGoogle Scholar
  13. 13.
    Grabisch M, Nguyen HT, Walker EA (1995) Fundamentals of university calculi with applications to fuzzy inference. Kluwer AcademicGoogle Scholar
  14. 14.
    Grabisch M, Murofushi T, Sugeno M (2000) Fuzzy Measure and Integrals. Theory and Applications (editedvolume). Studies in Fuzziness, Physica VerlagGoogle Scholar
  15. 15.
    Guo L, Xu D, Qiang Z (2016) Background subtraction using local SVD binary pattern. Proc IEEE Int Conf Comput Vision Pattern Recogn Workshops: 1159–1167Google Scholar
  16. 16.
    Han B, Davis LS (2012) Density-based multifeature background subtraction with support vector machine. IEEE Trans Pattern Anal Mach Intell 34(5):1017–1023CrossRefGoogle Scholar
  17. 17.
    Hanbury A (2003) A 3D-polar coordinate colour representation well adapted to image analysis. Proc Scandinav Conf Image Anal (SCIA) 2749Google Scholar
  18. 18.
    Hasan S, Samson Cheung S-C (2017) Universal multimode background subtraction. IEEE Trans Image Process 26(7):3249–3260MathSciNetCrossRefGoogle Scholar
  19. 19.
    Heikkila M, Pietikäinen M (2006) A texture-based method for modeling the background and detecting moving objects. IEEE Trans Pattern Anal Mach Intell 28(4):657–662CrossRefGoogle Scholar
  20. 20.
    Heikkila M, Pietikainen M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recogn 42(3):425–436CrossRefGoogle Scholar
  21. 21.
    Hu Q, Li Sh, He K, Lin H (2010) A robust fusion method for vehicle detection in road traffic surveillance. Proc 6th Int Conf Intell Comput (ICIC) 6216Google Scholar
  22. 22.
    Iakovidis DK, Keramidas EG, Maroulis D (2008) Fuzzy local binary patterns for ultrasound texture characterization. 2008 Springer Int Conf Image Anal Recogn 5112:750–759Google Scholar
  23. 23.
    Jiang S, Lu X (2017) WeSamBE : a weight-sample-based method for background subtraction. IEEE Trans Circ Syst Video TechnolGoogle Scholar
  24. 24.
    Kim K, Chalidabhongse T, Harwood D (2004) Background modeling and subtraction by codebook construction. 2004 IEEE Int Conf Image Process (ICIP): 3061–3064Google Scholar
  25. 25.
    Lai A, Yung N (1998) A fast and accurate scoreboard algorithm for estimating stationary backgrounds in an image sequence. IEEE Int Sym Circ Syst (ISCAS): 241–244Google Scholar
  26. 26.
    Lee D–S (2005) Effective Gaussian mixture learning for video background subtraction. IEEE Trans Pattern Anal Mach Intell 27(5):827–832CrossRefGoogle Scholar
  27. 27.
    Liao WH (2012) Incorporating fuzziness in extended local ternary patterns. IEEE Int Sym Multimed (ISM)Google Scholar
  28. 28.
    Liao S, Zhao G, Kellokumpu V, Pietikäinen M, Li S Z (2010) Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. Proc IEEE Conf Comput Vision Pattern Recogn (CVPR): 1301–1306Google Scholar
  29. 29.
    Lipton A J, Fujiyoshi H, Patil RS (1998) Moving target classification and tracking from Real-time video. 1998 Proc IEEE Workshop Appl Comput Vision: 8–44Google Scholar
  30. 30.
    Maddalena L, Petrosino A (2008) A self-organizing approach to background subtraction for visual surveillance applications. IEEE Trans Image Process 17(7):1168–1177MathSciNetCrossRefGoogle Scholar
  31. 31.
    Martins I, Carvalho P, Corte-Real L, Alba-Castro JL (2017) BMOG : boosted Gaussian mixture model with controlled complexity. 2017 IbPRIA Pattern Recogn Image Anal: 50–57Google Scholar
  32. 32.
    Mason M, Duric Z (2001) Using histograms to detect and track objects in color video. Proc IEEE 30th Appl Imag Pattern Recogn Workshop (AIPR)Google Scholar
  33. 33.
    Mateos G, Giannakis GB (2010) Sparsity control for robust principal component analysis. 2010 Conf Rec Forty Fourth Asilomar Conf Sign Syst Comput (ASILOMAR): 1925–1929Google Scholar
  34. 34.
    Ojala T, Pietikäinen M, Harwood D (1996) A camparative study of texture measure with classification based on feature distributions. Pattern Recogn 29(1):51–59CrossRefGoogle Scholar
  35. 35.
    Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Analy Mach Intell 24(7):971–987CrossRefGoogle Scholar
  36. 36.
    Oliver NM, Rosario B, Pentland AP (2000) A Bayesian computer vision system for modeling human interactions. IEEE Trans Pattern Anal Mach Intell (TPAMI) 22(8):831–843CrossRefGoogle Scholar
  37. 37.
    Pietikäinen M, Ahonen T (2008) Soft histograms for local binary patterns. Proc Finnish Sign Process Symp (FINSIG)Google Scholar
  38. 38.
    Pietikäinen M, Hadid A, Zhao G, Ahnen T (2011) Local binary patterns for still images, Chapter 2. Springer, Computer Vision Using Local Binary Patterns, XVI, p. 212, HardcoverGoogle Scholar
  39. 39.
    Randen T, Husøy JH (1999) Filtering for texture classification: a comparative study. IEEE Trans Pattern Anal Mach Intell 21(4):291–310CrossRefGoogle Scholar
  40. 40.
    Ren J, Jiang X, Yuan J (2013) Noise-resistant local binary pattern with an embedded error-correction mechanism. IEEE Trans Image Process 22(10):4049–4060MathSciNetCrossRefGoogle Scholar
  41. 41.
    Sigari M, Mozayani N, Pourreza H (2008) Fuzzy running average and fuzzy background subtraction: concepts and application. Int J Comput Sci Netw Sec (IJCSNS) 8 (2) : 138–143Google Scholar
  42. 42.
    Sobral A, Vacavant A (2014) A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Comput Vis Image Underst 122:4–21CrossRefGoogle Scholar
  43. 43.
    Stauffer C, Grimson W (1999) Adaptive background mixture models for real-time tracking. Proc IEEE Conf Comput Vision Pattern Recogn (CVPR)Google Scholar
  44. 44.
    St-Charles P-L, Bilodeau G-A, Bergevin R (2014) SuBSENSE : a universal change detection method with local adaptive sensitivity. IEEE Trans Image Process 24(1):359–373MathSciNetCrossRefGoogle Scholar
  45. 45.
    St-Charles PL, Bilodeau GA, Bergevin R (2015) SuBSENSE : a universal change detection method with local adaptive sensitivity. IEEE Trans Image Process 24(1):359–373MathSciNetCrossRefGoogle Scholar
  46. 46.
    Stefano L, Tombari F, Mattoccia S, Lisi E (2007) Robust and accurate change detection under sudden illumination variations. 2007 ACCV Workshop Multi-Dimensional Multi-view Image ProcessGoogle Scholar
  47. 47.
    Szummer M, Picard RW (1996) Temporal texture modeling. 1996 IEEE Int Conf Image Process: 823–826Google Scholar
  48. 48.
    Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Processing 19(6):1635–1650MathSciNetCrossRefGoogle Scholar
  49. 49.
    Tian Y, Wang Y, Hu Z, Huang T (2013) Selective Eigenbackground for background modeling and subtraction in crowded scenes. IEEE Trans Circ Syst Video Technol 23(11):1849–1864CrossRefGoogle Scholar
  50. 50.
    Tombari F, Stefano L, Mattoccia S (2007) A robust measure for visual correspondence. 2007 Proc IEEE 14th Int Conf Image Anal Process (ICIAP): 376–381Google Scholar
  51. 51.
    Wang Z, Klir GJ (1992) Fuzzy measure theory. PlenumGoogle Scholar
  52. 52.
    Wang K, Liu Y, Guo C, Wang F-Y (2016) A multi-view learning approach to foreground detection for traffic surveillance applications. IEEE Trans Vehic Technol 65(6):4144–4158CrossRefGoogle Scholar
  53. 53.
    Wang Y, Luo ZM, Jodoin PM (2016) Interactive deep learning method for segmenting moving objects. Pattern Recogn Lett 96:66–75CrossRefGoogle Scholar
  54. 54.
    Wang K, Gou C, Wang F-Y (2018) M4CD: A Robust Change Detection Method for Intelligent Visual Surveillance arXiv:1802.04979Google Scholar
  55. 55.
    Zeng D, Zhu M, Zhou T, Xu F, Yang H (2017) Robust background subtraction via the local similarity statistical descriptor. Appl Sci 7(10):989CrossRefGoogle Scholar
  56. 56.
    Zeng D, Zhu M, Zhou T, Xu F (2018) An extended scale invariant local binary pattern for background subtraction. IET Image ProcessGoogle Scholar
  57. 57.
    Zhang H, xu D (2006) Fusing color and texture features for background model. Proc 3th Int Conf Fuzzy Syst Knowl Discov (ICFSKD) 4223(7):887–893CrossRefGoogle Scholar
  58. 58.
    Zhang B, Gao Y, Zhao S, Zhong B (2011) Kernel similarity modeling of texture pattern flow for motion detection in complex background. IEEE Trans Circ Syst Video Technol 21(1):29–38CrossRefGoogle Scholar
  59. 59.
    Zhao X, Satoh Y, Takauji H, Kaneko S, Iwata K, Ozaki R (2011) Object detection based on a robust and accurate statistical multi-point-pair model. Pattern Recogn 44:1296–1311CrossRefGoogle Scholar
  60. 60.
    Zheng W, Wang K, Wang F-Y (2017) A novel background subtraction algorithm based on parallel vision and Bayesian GANs. Adv Neural Inform Process Syst 30 (NIPS)Google Scholar
  61. 61.
    Zivkovic Z (2004) Improved adaptive Gaussian mixture model for background subtraction. In: proceedings of the IEEE 17th. Int Conf Pattern Recog (ICPR) 2:28–31Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Electrical EngineeringUniversity of GuilanGuilanIran

Personalised recommendations