Multimedia Tools and Applications

, Volume 78, Issue 22, pp 31415–31439 | Cite as

Spatiotemporal local compact binary pattern for background subtraction in complex scenes

  • Wei He
  • Hak-Lim Ko
  • Yong Kwan KimEmail author
  • Jianhui Wu
  • Guoyun Zhang
  • Qi Qi
  • Bing Tu
  • Xianfeng OuEmail author


A variety of binary feature descriptors such as local binary pattern (LBP) and its variations have recently attracted considerable attention for modelling backgrounds, due to their robustness and strong discriminatory power. However, most existing binary feature descriptors fail to model complex scenes due to their sensitivity to noise. In this paper, we propose an effective local compact binary descriptor for background modelling. For each image, local compact binary patterns (LCBPs) are first extracted by computing a number of low-dimensional pixel difference vectors (PDVs). Then, the LCBP is extended to the spatiotemporal domain taking into account the temporal persistence of pixels, and a novel local compact binary descriptor, STLCBP, is proposed. Multiple color spaces are also considered in order to separate foreground from background pixels accurately. Finally, a joint domain-range adaptive kernel density estimate (KDE) model is used to estimate the background and foreground scores by combining texture features with color features. Experimental results on two well-known datasets, I2R and CDnet2014, demonstrate that the proposed approach significantly outperforms many state-of-the-art methods and works effectively on a wide range of complex videos.


Background modeling binary feature learning local compact binary pattern kernel density estimation multi-color spaces 



This work has been supported in part by Hunan Provincial Natural Science Foundation of China (2019JJ40104, 2019JJ50211, 2019JJ50212), the Open Fund of Education Department of Hunan Province (18K086), the Science and Technology Program of Hunan Province (2016TP1021), the Development of Distributed Underwater Monitoring and Control Networks funded by the Ministry of Ocean and Fisheries, South Korea.


  1. 1.
    Amraee S, Vafaei A, Jamshidi K, Adibi P (2018) Anomaly detection and localization in crowed scenes using connected component analysis. Multimed Tools Appl 77(12):14767–14782CrossRefGoogle Scholar
  2. 2.
    Balcilar M, Amasyali M, Sonmez A (2014) Moving object detection using lab2000HL color space with spatial and temporal smoothing. Applied Math Infor Sci 8(4):1755–1766CrossRefGoogle Scholar
  3. 3.
    Balcilar M, Karabiber F, Sonmez A (2013) Performance analysis of Lab2000HL color space for background subtraction. IEEE Inter Symon Innov in Intell Syst Appl:1–6Google Scholar
  4. 4.
    Barnich O, Droogenbroeck M (2011) ViBe: A universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20(6):1709–1724MathSciNetCrossRefGoogle Scholar
  5. 5.
    Bilodeau G, Jodoin J, Saunier N (2013) Change detection in feature space using local binary similarity patterns. Int Conf Comput Robot Vis:106–112Google Scholar
  6. 6.
    Wu J, Huang F, Hu W, He W (2019) Study of multiple moving detection in fisheye video based on the moving blob model. Tools Appl 78:877-896Google Scholar
  7. 7.
    Chen M, Wei X, Yang Q, Li Q, Wang G, Yang M (2018) Spatiotemporal GMM for background subtraction with superpixel hierarchy. IEEE Trans Pattern Anal Mach Intell 40(6):1518–1525CrossRefGoogle Scholar
  8. 8.
    Chengjun J, Guiran C, Wei C, Huiyan J (2011) Background extraction and update method based on histogram in ycbcr color space. Inter Conf E-Business E-Government:1–4Google Scholar
  9. 9.
    Comaniciu D, Zhu Y, Davis L (2008) Sequential kernel density approximation and its application to real-time visual tracking. IEEE Trans Pattern Anal Mach Intell 30(7):1186–1197CrossRefGoogle Scholar
  10. 10.
    Cuevas C, Mohedano R, Garcia N (2012) Versatile Bayesian classifier for moving object detection by non-parametric background-foreground modeling. IEEE Conf on Image Process:313–316Google Scholar
  11. 11.
    Duan Y, Lu J, Feng J, Zhou J (2018) Context-aware local binary feature learning for face recognition. IEEE Trans Pattern Anal Mach Intell 40(5):1139–1152CrossRefGoogle Scholar
  12. 12.
    Elgammal A, Duraiswami R, Harvood D, Davis L (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90(7):1151–1163CrossRefGoogle Scholar
  13. 13.
    Guo L, Xu D, Qiang Z (2016) Background subtraction using local SVD binary pattern. Proc IEEE Conf Comput Vis Patt Recogn:86–94Google Scholar
  14. 14.
    He W, King Y, Wu J et al (2018) Local compact binary patterns for background subtraction in complex scenes. Proc Int Conf Pattern Recognit:1518–1523Google Scholar
  15. 15.
    Heikkila M, Pietikainen M (2006) A texture-based method for modeling the background and detecting moving objects. IEEE Trans Pattern Anal Mach Intell 28(4):657–662CrossRefGoogle Scholar
  16. 16.
    Heikkila M, Pietikainen M, Heikkila J (2004) A texture-based method for detecting moving objects. IEEE Trans in British Mach Vis Conf:187–196Google Scholar
  17. 17.
    Hofmann M, Tiefenbacher P, Rigoll G (2012) Background segmentation with feedback: The pixel-based adaptive segmenter. IEEE Conf on Comput Vis Pattern Recogn:38–43Google Scholar
  18. 18.
    Johnson G, Song X, Montag E, Fairchild M (2010) Derivation of a color space for image color difference measurement. Color Res Appl 35(6):387–400CrossRefGoogle Scholar
  19. 19.
    Karpagavalli P, Ramprasad AV (2017) An adaptive hybrid GMM for multiple human detection in crowd scenario. Multimed Tools Appl 76(122):14129–14149CrossRefGoogle Scholar
  20. 20.
    Li L, Huang W, Gu I, Tian Q (2003) Foreground object detection from videos containing complex background. Proc ACM Conf on Multi:2–10Google Scholar
  21. 21.
    Liao S, Zhao G, Kellokumpu V, Pietikainen M, Li S (2010) Modeling pixel processing with scale invariant local patterns for background subtraction in complex scenes. IEEE Conf on Comput Vis Pattern Recogn:1301–1306Google Scholar
  22. 22.
    Lin L, Xu Y, Liang X, Lai J (2014) Complex background subtraction by pursuing dynamic spatio-temporal models. IEEE Trans Image Process 23(7):3191–3202MathSciNetCrossRefGoogle Scholar
  23. 23.
    Lissner I, Preiss J, Urban P, Lichtenauer M, Zolliker P (2013) Image-Difference prediction: from grayscale to color. IEEE Trans Image Process 22(2):435–446MathSciNetCrossRefGoogle Scholar
  24. 24.
    Lissner I, Urban P (2012) Toward a unified color space for perception-based image processing. IEEE Trans Image Process 21(3):1153–1168MathSciNetCrossRefGoogle Scholar
  25. 25.
    Liu L, Lao S, Guo Y, Wang X, Pietikainen M (2016) Median robust extended local binary pattern for texture classification. IEEE Trans Image Process 25(3):1368–1381MathSciNetCrossRefGoogle Scholar
  26. 26.
    Liu W, Lin S, Yang M, Huang C (2013) Real-time binary descriptor based background modeling. Asian Conf on Pattern Recogn:722–726Google Scholar
  27. 27.
    Liu X, Zhao G, Yao J, Qi C (2015) Background subtraction based on low-rank and structured sparse decomposition. IEEE Trans Image Process 24(8):2502–2514MathSciNetCrossRefGoogle Scholar
  28. 28.
    Lu J, Liong V, Zhou X, Zhou J (2015) Learning compact binary face descriptor for face recognition. IEEE Trans Pattern Anal Mach Intell 37(10):2041–2056CrossRefGoogle Scholar
  29. 29.
    Maddalena L, Petrosino A (2012) The SOBS algorithm: what are the limits? IEEE Conf on Comput Vis Pattern Recogn Workshops:21–26Google Scholar
  30. 30.
    Maddalena L, Petrosino A (2014) The 3dSOBS+ algorithm for moving object detection. Comput Vis Image Und 122:65–73CrossRefGoogle Scholar
  31. 31.
    Narayana M, Hanson A, Learned-Miller E (2012) Background modeling using adaptive pixelwise kernel variances in a hybrid feature space. IEEE Conf on Comput Vis Pattern Recogn:2104–2111Google Scholar
  32. 32.
    Narayana M, Hanson A, Learned-Miller E (2012) Improvements in joint domain-range modeling for background subtraction. Proceed of the British Mach Vis Conf:1–11Google Scholar
  33. 33.
    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987CrossRefGoogle Scholar
  34. 34.
    Pan Z, Liu S, Fu W (2017) A review of visual moving target tracking. Multimed Tools Appl 76(16):16989–17018CrossRefGoogle Scholar
  35. 35.
    Roy S, Ghosh A (2018) Real-time adaptive histogram min-max bucket (HMMB) model for background subtraction. IEEE Trans Pattern Anal Mach Intell 28(7):1513–1525Google Scholar
  36. 36.
    Sajid H, Cheung S (2017) Universal multimode background subtraction. IEEE Trans Image Process 26(7):3249–3260MathSciNetCrossRefGoogle Scholar
  37. 37.
    Seki M, Wada T, Fujiwara H, Sumi K (2003) Background subtraction based on co-occurrence of image variations. IEEE Conf on Comput Vis Pattern Recogn II:65–72Google Scholar
  38. 38.
    Sheikh Y, Shah M (2005) Bayesian modeling of dynamic scenes for object detection. IEEE Trans Pattern Anal Mach Intell 27(11):1178–1792CrossRefGoogle Scholar
  39. 39.
    Shu Y, Zhang H (2018) Multimodal information fusion based human movement recognition. Multimed Tools Appl.
  40. 40.
    Stauffer C, Grimson W (1999) Adaptive background mixture models for real-time tracking. IEEE Conf on Comput Vis Pattern Recogn:246–252Google Scholar
  41. 41.
    Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650MathSciNetCrossRefGoogle Scholar
  42. 42.
    Wang Y, Jodoin P, Porikli F, Konrad J, Benezeth Y, Ishwar P (2014) CDnet 2014: An expanded change detection benchmark dataset. IEEE Workshop Comput Vis Pattern Recogn:387–394Google Scholar
  43. 43.
    Wen Z, Yin W (2013) A feasible method for optimization with orthogonality constraints. Math Program 142(1-2):397–434MathSciNetCrossRefGoogle Scholar
  44. 44.
    Wu J, Huang F, Hu W et al (2018) Study of multiple moving targets’ detection in fisheye video based on the moving blob model. Multimed Tools Appl. CrossRefGoogle Scholar
  45. 45.
    Xue G, Song L, Sun J, Wu M (2011) Hybrid center-symmetric local pattern for dynamic background subtraction. IEEE Inter Conf on Multimed Expo:1–6Google Scholar
  46. 46.
    Xue G, Sun J, Song L (2010) Dynamic background subtraction based on spatial extended center-symmetric local binary pattern. IEEE Inter Conf on Multimed Expo:1050–1054Google Scholar
  47. 47.
    Zhang S, Yao H, Liu S (2008) Dynamic background Modeling and subtraction using spatio-temporal local binary patterns. IEEE Conf on Image Process:1556–1559Google Scholar
  48. 48.
    Zhao G, Ahonen T, Matas J, Pietikainen M (2012) Rotation-invariant image and video description with local binary pattern features. IEEE Trans Image Process 21(4):1465–1477MathSciNetCrossRefGoogle Scholar
  49. 49.
    Zhao G, Pietikaninen M (2007) Dynamic Texture Recognition using volume local binary patterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 29(6):915–928CrossRefGoogle Scholar
  50. 50.
    Zivkovic Z (2004) Improved adaptive Gaussian mixture model for background subtraction. Inter Conf on Pattern Recogn II:28–31Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Information and Communication EngineeringHunan Institute of Science and TechnologyYueyangChina
  2. 2.Department of Information and Communication EngineeringHoseo UniversityAsanSouth Korea

Personalised recommendations