Signal, Image and Video Processing

, Volume 13, Issue 1, pp 9–16 | Cite as

Unsupervised video object segmentation using conditional random fields

  • Asma Hamza Bhatti
  • Anis Ur RahmanEmail author
  • Asad Anwar Butt
Original Paper


In this work, we propose a graph-based superpixel segmentation technique to perform spatiotemporal oversegmentation of videos. The generated superpixels are post-processed by applying a straightforward threshold-based foreground separation model. These superpixels are used in a conditional random field, and a potential function is defined, which is solved using energy minimization techniques to produce a final segmentation. Experiments on two datasets containing over 24 videos demonstrate that our method produces competitive or better results for the video object segmentation task over the state-of-the-art algorithms.


Segmentation Video Superpixels 


  1. 1.
    Bai, X., Wang, W.: Saliency-svm: an automatic approach for image segmentation. Neurocomputing 136, 243–255 (2014)CrossRefGoogle Scholar
  2. 2.
    Banica, D., Agape, A., Ion, A., Sminchisescu, C.: Video object segmentation by salient segment chain composition. In: IEEE International Conference on Computer Vision Workshops, pp. 283–290 (2013)Google Scholar
  3. 3.
    Bhatti, A.H., Rahman, A., Butt, A.A.: Video segmentation using spectral clustering on superpixels. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 869–873 (2016)Google Scholar
  4. 4.
    Callet, P., Niebur, E.: Visual attention and applications in multimedia technologies. In: Proceedings of the IEEE, pp. 2058–2067 (2013)Google Scholar
  5. 5.
    Carreira, J., Sminchisescu, C.: Cpmc: automatic object segmentation using constrained parametric min-cuts. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1312–1328 (2012)CrossRefGoogle Scholar
  6. 6.
    Chang, J., Donglai, W., Fisher, J.W.: A video representation using temporal superpixels. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2051–2058 (2013)Google Scholar
  7. 7.
    Fukuchi, K., Miyazato, K., Kimura, A., Takagi, S., Yamato, J.: Saliency-based video segmentation with graph cuts and sequentially updated priors. In: IEEE International Conference on Multimedia and Expo, pp. 638–641 (2009)Google Scholar
  8. 8.
    Galasso, F., Cipolla, R., Schiele, B.: Video segmentation with superpixels. In: Asian Conference on Computer Vision, pp. 760–774. Springer, Berlin (2013)Google Scholar
  9. 9.
    Haller, E., Leordeanu, M.: Unsupervised object segmentation in video by efficient selection of highly probable positive features. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 5095–5103 (2017)Google Scholar
  10. 10.
    Lee, Y.J., Kim, J., Grauman, K.: Key-segments for video object segmentation. In: IEEE International Conference on Computer Vision, pp. 1995–2002. IEEE Computer Society, Washington, DC, USA (2011)Google Scholar
  11. 11.
    Li, F., Kim, T., Humayun, A., Tsai, D., Rehg, J.M.: Video segmentation by tracking many figure-ground segments. In: IEEE International Conference on Computer Vision, pp. 2192–2199 (2013)Google Scholar
  12. 12.
    Li, W.T., Chang, H.S., Lien, K.C., Chang, H.T., Wang, Y.C.: Exploring visual and motion saliency for automatic video object extraction. IEEE Trans. Image Process. 22(7), 2600–2610 (2013)CrossRefGoogle Scholar
  13. 13.
    Li, Z., Chen, J.: Superpixel segmentation using linear spectral clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1356–1363 (2015)Google Scholar
  14. 14.
    Li, Z., Wu, X.-M., Chang, S.-F.: Segmentation using superpixels: a bipartite graph partitioning approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 789–796 (2012)Google Scholar
  15. 15.
    Liu, Z., Zhang, X., Luo, S., Le Meur, O.: Superpixel-based spatiotemporal saliency detection. IEEE Trans. Circuits Syst. Video Technol. 24(9), 1522–1540 (2014)CrossRefGoogle Scholar
  16. 16.
    Luque, R.M., Domínguez, E., Palomo, E.J., Muñoz, J.: A neural network approach for video object segmentation in traffic surveillance. In: Image Analysis and Recognition, 5th International Conference, ICIAR 2008, Póvoa de Varzim, Portugal, June 25–27, 2008. Proceedings, pp. 151–158 (2008)Google Scholar
  17. 17.
    Masuda, M., Mochizuki, Y., Ishikawa, H.: Unsupervised video object segmentation by supertrajectory labeling. In: 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), pp. 448–451. IEEE (2017)Google Scholar
  18. 18.
    Milan, A., Leal-Taixe, L., Schindler, K., Reid, I.: Joint tracking and segmentation of multiple targets. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5397–5406 (2015)Google Scholar
  19. 19.
    Shi, J.: Video segmentation by tracing discontinuities in a trajectory embedding. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’12, pp. 1846–1853 (2012)Google Scholar
  20. 20.
    Tsai, D., Flagg, M., Nakazawa, A., Rehg, J.: Motion coherent tracking using multi-label MRF optimization. Int. J. Comput. Vis. 100, 190–202 (2012)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Vasconcelos, M.J.M., Tavares, J.M.R.S.: Human motion segmentation using active shape models. In: Computational and Experimental Biomedical Sciences: Methods and Applications, pp. 237–246. Springer, Berlin (2015)Google Scholar
  22. 22.
    Wang, W., Shen, J., Porikli, F.: Saliency-aware geodesic video object segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3395–3402 (2015)Google Scholar
  23. 23.
    Xu, L., Li, H., Zeng, L., Ngan, K.N.: Saliency detection using joint spatial-color constraint and multi-scale segmentation. J. Vis. Comun. Image Represent. 24(4), 465–476 (2013)CrossRefGoogle Scholar
  24. 24.
    Zhang, D., Javed, O., Shah, M.: Video object segmentation through spatially accurate and temporally dense extraction of primary object regions. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 628–635 (2013)Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Electrical Engineering and Computer SciencesNational University of Sciences and TechnologyIslamabadPakistan
  2. 2.National Institute of Standards and Technology (NIST)GaithersburgUSA

Personalised recommendations