Background Modeling Using Deep-Variational Autoencoder

  • Midhula VijayanEmail author
  • R. Mohan
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 940)


A foreground segmentation algorithm using the variational autoencoder with skip architecture is presented in this paper. The deep-variational autoencoder network is trained using the target frame and its ground truth image. The variational encoder network constructs non-handcrafted feature sets, and the decoder-network transforms the feature sets into the segmented binary map. Moreover, the variational autoencoder with skip architecture accurately segment the moving objects. The skip architecture used to combine the fine and the coarse scale feature information. Experiments conducted on ‘ (CDnet-2014)’ dataset show that the variational autoencoder based algorithm produces significant results when compared with the classical background modeling methods.


Foreground segmentation Convolutional Neural Network (CNN) Background-modeling Skip architecture Variational autoencoder (VAE) 


  1. 1.
    Allebosch, G., Deboeverie, F., Veelaert, P., Philips, W.: EFIC: edge based foreground background segmentation and interior classification for dynamic camera viewpoints. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 130–141. Springer, Cham (2015)CrossRefGoogle Scholar
  2. 2.
    Babaee, M., Dinh, D.T., Rigoll, G.: A deep convolutional neural network for video sequence background subtraction. Pattern Recognit. 76, 635–649 (2018)CrossRefGoogle Scholar
  3. 3.
    Barnich, O., Van Droogenbroeck, M.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Chen, Y., Wang, J., Lu, H.: Learning sharable models for robust background subtraction. In: 2015 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2015)Google Scholar
  5. 5.
    Culibrk, D., Marques, O., Socek, D., Kalva, H., Furht, B.: Neural network approach to background modeling for video object segmentation. IEEE Trans. Neural Netw. 18(6), 1614–1627 (2007)CrossRefGoogle Scholar
  6. 6.
    De Gregorio, M., Giordano, M.: WiSARDrp for change detection in video sequences. Submitted to (CVPR 2016) Google Scholar (2016)Google Scholar
  7. 7.
    Doersch, C.: Tutorial on variational autoencoders (2016). arXiv preprint: arXiv:1606.05908
  8. 8.
    Goyette, N., Jodoin, P.M., Porikli, F., Konrad, J., Ishwar, P., et al.: a new change detection benchmark dataset. In: CVPR Workshops, pp. 1–8 (2012)Google Scholar
  9. 9.
    Heikkila, M., Pietikainen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 657–662 (2006)CrossRefGoogle Scholar
  10. 10.
    Hofmann, M., Tiefenbacher, P., Rigoll, G.: Background segmentation with feedback: the pixel-based adaptive segmenter. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 38–43. IEEE (2012)Google Scholar
  11. 11.
    Jain, S.D., Xiong, B., Grauman, K.: Fusionseg: learning to combine motion and appearance for fully automatic segmention of generic objects in videos. In: Proceedings of CVPR, vol. 1 (2017)Google Scholar
  12. 12.
    Jiang, S., Lu, X.: WeSamBE: a weight-sample-based method for background subtraction. IEEE Trans. Circ. Syst. Video Technol. 28(9), 2105–2115 (2018)CrossRefGoogle Scholar
  13. 13.
    Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.: Real-time foreground-background segmentation using codebook model. Real-time Imaging 11(3), 172–185 (2005)CrossRefGoogle Scholar
  14. 14.
    Liu, W.C., Lin, S.Z., Yang, M.H., Huang, C.R.: Real-time binary descriptor based background modeling. In: 2013 2nd IAPR Asian Conference on Pattern Recognition (ACPR), pp. 722–726. IEEE (2013)Google Scholar
  15. 15.
    Martins, I., Carvalho, P., Corte-Real, L., Alba-Castro, J.L.: BMOG: boosted Gaussian mixture model with controlled complexity. In: Iberian Conference on Pattern Recognition and Image Analysis, pp. 50–57. Springer, Cham (2017)CrossRefGoogle Scholar
  16. 16.
    Mayer, B.A., Mundy, J.L.: Duration dependent codebooks for change detection. In: BMVC (2014)Google Scholar
  17. 17.
    Pathak, D., Girshick, R.B., Dollár, P., Darrell, T., Hariharan, B.: Learning features by watching objects move. In: CVPR, vol. 1, p. 7 (2017)Google Scholar
  18. 18.
    Ramírez-Alonso, G., Chacón-Murguía, M.I.: Auto-adaptive parallel som architecture with a modular analysis for dynamic object segmentation in videos. Neurocomputing 175, 990–1000 (2016)CrossRefGoogle Scholar
  19. 19.
    Sajid, H., Cheung, S.C.S.: Background subtraction for static & moving camera. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4530–4534. IEEE (2015)Google Scholar
  20. 20.
    Schick, A., Bäuml, M., Stiefelhagen, R.: Improving foreground segmentations with probabilistic superpixel markov random fields. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 27–31. IEEE (2012)Google Scholar
  21. 21.
    St-Charles, P.L., Bilodeau, G.A., Bergevin, R.: SuBSENSE: a universal change detection method with local adaptive sensitivity. IEEE Trans. Image Process. 24(1), 359–373 (2015)MathSciNetCrossRefGoogle Scholar
  22. 22.
    St-Charles, P.L., Bilodeau, G.A., Bergevin, R.: Universal background subtraction using word consensus models. IEEE Trans. Image Process. 25(10), 4768–4781 (2016)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: CVPR, p. 2246. IEEE (1999)Google Scholar
  24. 24.
    Van Droogenbroeck, M., Paquot, O.: Background subtraction: experiments and improvements for vibe. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 32–37. IEEE (2012)Google Scholar
  25. 25.
    Varghese, A., Sreelekha, G.: Sample-based integrated background subtraction and shadow detection. IPSJ Trans. Comput. Vis. Appl. 9(1), 25 (2017)CrossRefGoogle Scholar
  26. 26.
    Wang, K., Gou, C., Wang, F.Y.: M4CD: a robust change detection method for intelligent visual surveillance. IEEE Access 6, 15505–15520 (2018)CrossRefGoogle Scholar
  27. 27.
    Wang, Y., Luo, Z., Jodoin, P.M.: Interactive deep learning method for segmenting moving objects. Pattern Recognit. Lett. 96, 66–75 (2017)CrossRefGoogle Scholar
  28. 28.
    Yang, L., Li, J., Luo, Y., Zhao, Y., Cheng, H., Li, J.: Deep background modeling using fully convolutional network. IEEE Trans. Intell. Transp. Syst. 19(1), 254–262 (2018)CrossRefGoogle Scholar
  29. 29.
    Zhu, Q., Avidan, S., Cheng, K.T.: Learning a sparse, corner-based representation for time-varying background modelling. In: Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 1, pp. 678–685. IEEE (2005)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.National Institute of TechnologyTiruchirappalliIndia

Personalised recommendations