Journal of Real-Time Image Processing

, Volume 16, Issue 6, pp 2319–2335 | Cite as

SuperBE: computationally light background estimation with superpixels

  • Andrew Tzer-Yeu ChenEmail author
  • Morteza Biglari-Abhari
  • Kevin I-Kai Wang
Original Research Paper


This paper presents a motion-based superpixel-level background estimation algorithm that aims to be competitively accurate while requiring less computation time for background modelling and updating. Superpixels are chosen for their spatial and colour coherency and can be grouped together to better define the shapes of objects in an image. RGB mean and colour covariance matrices are used as the discriminative features for comparing superpixels to their background model samples. The background model initialisation and update procedures are inspired by existing approaches, with the key aim of minimising computational complexity and therefore processing time. Experiments carried out with a widely used dataset show that SuperBE can achieve a high level of accuracy and is competitive against other state-of-the-art background estimation algorithms. The main contribution of this paper is the computationally efficient use of superpixels in background estimation while maintaining high accuracy, reaching 135 fps on 320 × 240 resolution images.


Computer vision Background estimation Object segmentation Image motion analysis Embedded vision Real-time systems 



This research was conducted with support from the New Zealand eScience Infrastructure (NeSI) Group and their high-performance computing facilities.


  1. 1.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  2. 2.
    Babaee, M., Dinh, D.T., Rigoll, G.: A deep convolutional neural network for background subtraction. CoRR (2017).
  3. 3.
    Bak, S., Corvée, E., Brémond, F., Thonnat, M.: Person re-identification using spatial covariance regions of human body parts. In: International Conference on Advanced Video and Signal Based Surveillance, pp. 435–440 (2010)Google Scholar
  4. 4.
    Barnich, O., van Droogenbroeck, M.: Vibe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Bouwmans, T.: Traditional and recent approaches in background modeling for foreground detection: an overview. Comput. Sci. Rev. 11(12), 31–66 (2014)CrossRefGoogle Scholar
  6. 6.
    Bradski, G.: The OpenCV Library. Dr Dobb’s Journal of Software Tools (2000)Google Scholar
  7. 7.
    Brutzer, S., Höferlin, B., Heidemann, G.: Evaluation of background subtraction techniques for video surveillance. In: International Conference on Computer Vision and Pattern Recognition, pp. 1937–1944 (2011)Google Scholar
  8. 8.
    Cherian, A., Sra, S., Banerjee, A., Papanikolopoulos, N.: Jensen–Bregman logdet divergence with application to efficient similarity search for covariance matrices. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2161–2174 (2013)CrossRefGoogle Scholar
  9. 9.
    Devi, S.K., Malmurugan, N., Poornima, S.: Improving the efficiency of background subtraction using superpixel extraction and midpoint for centroid. Int. J. Comput. Appl. 43(10), 1–5 (2012)Google Scholar
  10. 10.
    El Baf, F., Bouwmans, T., Vachon, B.: Type-2 fuzzy mixture of Gaussians model: application to background modeling. In: International Symposium on Visual Computing, pp. 772–781 (2008)Google Scholar
  11. 11.
    Figueira, D., Taiana, M., Nambiar, A., Nascimento, J., Bernardino, A.: The HDA+ data set for research on fully automated re-identification systems. In: European Conference on Computer Vision Workshops, pp. 241–255 (2015)CrossRefGoogle Scholar
  12. 12.
    Förstner, W., Moonen, B.: A metric for covariance matrices. In: Grafarend E.W., Krumm F.W., Schwarze V.S. (eds) Geodesy-The Challenge of the 3rd Millennium, pp. 299–309, Springer, Berlin, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Giordano, D., Murabito, F., Palazzo, S., Spampinato, C.: Superpixel-based video object segmentation using perceptual organization and location prior. In: Conference on Computer Vision and Pattern Recognition, pp. 4814–4822 (2015)Google Scholar
  14. 14.
    Gu, X., Deng, J.D., Purvis, M.K.: Improving superpixel-based image segmentation by incorporating color covariance matrix manifolds. In: International Conference on Image Processing, pp. 4403–4406 (2014)Google Scholar
  15. 15.
    Hofmann, M., Tiefenbacher, P., Rigoll, G.: Background segmentation with feedback: the pixel-based adaptive segmenter. In: Conference on Computer Vision and Pattern Recognition Workshops, pp. 38–43 (2012)Google Scholar
  16. 16.
    Ivanov, Y., Bobick, A., Liu, J.: Fast lighting independent background subtraction. In: IEEE Workshop on Visual Surveillance (1998)Google Scholar
  17. 17.
    Javed, S., Oh, S.H., Sobral, A., Bouwmans, T., Jung, S.K. Background subtraction via superpixel-based online matrix decomposition with structured foreground constraints. In: International Conference on Computer Vision Workshop, pp. 930–938 (2015)Google Scholar
  18. 18.
    Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.S.: Real-time foreground-background segmentation using codebook model. Real Time Imaging 11(3), 172–185 (2005)CrossRefGoogle Scholar
  19. 19.
    Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M., Shafer, S.: Multi-camera multi-person tracking for easyliving. In: International Workshop on Visual Surveillance, pp. 3–10 (2000)Google Scholar
  20. 20.
    Li, F., Kim, T., Humayun, A., Tsai, D., Rehg, J.M.: Video segmentation by tracking many figure-ground segments. In: International Conference on Computer Vision, pp. 2192–2199 (2013)Google Scholar
  21. 21.
    Liem, M.C., Gavrila, D.M.: A comparative study on multi-person tracking using overlapping cameras. In: International Conference on Computer Vision Systems, pp. 203–212 (2013)Google Scholar
  22. 22.
    Liem, M.C., Gavrila, D.M.: Joint multi-person detection and tracking from overlapping cameras. Comput. Vis. Image Underst. 128, 36–50 (2014)CrossRefGoogle Scholar
  23. 23.
    Lim, J., Han, B.: Generalized background subtraction using superpixels with label integrated motion estimation. In: European Conference on Computer Vision, pp. 173–187 (2014)CrossRefGoogle Scholar
  24. 24.
    Lu, W., Tan, Y.P.: A color histogram based people tracking system. Int. Symp. Circuits Syst. 2, 137–140 (2001)Google Scholar
  25. 25.
    Luque, R.M., Domínguez, E., Palomo, E.J., Muñoz, J.: A neural network approach for video object segmentation in traffic surveillance. In: International Conference on Image Analysis and Recognition, pp. 151–158 (2008)Google Scholar
  26. 26.
    Mittal, A., Davis, L.S.: M2tracker: a multi-view approach to segmenting and tracking people in a cluttered scene. Int. J. Comput. Vis. 51(3), 189–203 (2003)CrossRefGoogle Scholar
  27. 27.
    Papazoglou, A., Ferrari, V.: Fast object segmentation in unconstrained video. In: International Conference on Computer Vision, pp. 1777–1784 (2013)Google Scholar
  28. 28.
    Piccardi, M.: Background subtraction techniques: a review. In: International Conference on Systems, Man, and Cybernetics, pp. 3099–3104 (2004)Google Scholar
  29. 29.
    Schick, A., Bäuml, M., Stiefelhagen, R.: Improving foreground segmentations with probabilistic superpixel markov random fields. In: Conference on Computer Vision and Pattern Recognition Workshops, pp. 27–31 (2012)Google Scholar
  30. 30.
    Shoushtarian, B., Bez, H.E.: A practical adaptive approach for dynamic background subtraction using an invariant colour model and object tracking. Pattern Recognit. Lett. 26(1), 5–26 (2005)CrossRefGoogle Scholar
  31. 31.
    Shu, G., Dehghan, A., Shah, M.: Improving an object detector and extracting regions using superpixels. In: Conference on Computer Vision and Pattern Recognition, pp. 3721–3727 (2013)Google Scholar
  32. 32.
    Sobral, A., Vacavant, A.: A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Comput. Vis. Image Underst. 122, 4–21 (2014)CrossRefGoogle Scholar
  33. 33.
    Spampinato, C., Palazzo, S., Kavasidis, I.: A texton-based kernel density estimation approach for background modeling under extreme conditions. Comput. Vis. Image Underst. 122, 74–83 (2014)CrossRefGoogle Scholar
  34. 34.
    St-Charles, P., Bilodeau, G., Bergevin, R. A self-adjusting approach to change detection based on background word consensus. In: Winter Conference on Applications of Computer Vision, pp. 990–997 (2015)Google Scholar
  35. 35.
    St-Charles, P., Bilodeau, G., Bergevin, R.: Subsense: a universal change detection method with local adaptive sensitivity. IEEE Trans. Image Process. 24(1), 359–373 (2015)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Tsai, D., Lai, S.: Independent component analysis-based background subtraction for indoor surveillance. IEEE Trans. Image Process. 18(1), 158–167 (2009)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Wang, H., Suter, D.: Background subtraction based on a robust consensus method. In: International Conference on Pattern Recognition, pp. 223–226 (2006)Google Scholar
  38. 38.
    Wang, R., Bunyak, F., Seetharaman, G., Palaniappan, K.: Static and moving object detection using flux tensor with split gaussian models. In: Conference on Computer Vision and Pattern Recognition, pp. 420–424 (2014)Google Scholar
  39. 39.
    Wang, Y., Jodoin, P., Porikli, F.M., Konrad, J., Benezeth, Y., Ishwar, P.: Cdnet 2014: an expanded change detection benchmark dataset. In: Conference on Computer Vision and Pattern Recognition, pp. 393–400 (2014)Google Scholar
  40. 40.
    Xin, B., Tian, Y., Wang, Y., Gao, W.: Background subtraction via generalized fused lasso foreground modeling. In: Conference on Computer Vision and Pattern Recognition, pp. 4676–4684 (2015)Google Scholar
  41. 41.
    Zivkovic, Z.: Improved adaptive gaussian mixture model for background subtraction. Int. Conf. on Pattern Recognit. 2, 28–31 (2004)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Andrew Tzer-Yeu Chen
    • 1
    Email author
  • Morteza Biglari-Abhari
    • 1
  • Kevin I-Kai Wang
    • 1
  1. 1.Embedded Systems Research Group, Department of Electrical and Computer EngineeringThe University of AucklandAucklandNew Zealand

Personalised recommendations