Soft Computing

, Volume 23, Issue 24, pp 13055–13065 | Cite as

The low-rank decomposition of correlation-enhanced superpixels for video segmentation

  • Haixia XuEmail author
  • Edwin R. Hancock
  • Wei Zhou
Methodologies and Application


Low-rank decomposition (LRD) is an effective scheme to explore the affinity among superpixels in the image and video segmentation. However, the superpixel feature collected based on colour, shape, and texture may be rough, incompatible, and even conflicting if multiple features extracted in various manners are vectored and stacked straight together. It poses poor correlation, inconsistence on intra-category superpixels, and similarities on inter-category superpixels. This paper proposes a correlation-enhanced superpixel for video segmentation in the framework of LRD. Our algorithm mainly consists of two steps, feature analysis to establish the initial affinity among superpixels, followed by construction of a correlation-enhanced superpixel. This work is very helpful to perform LRD effectively and find the affinity accurately and quickly. Experiments conducted on datasets validate the proposed method. Comparisons with the state-of-the-art algorithms show higher speed and more precise in video segmentation.


Video segmentation LRD The enhanced superpixel 



This work was supported by National Natural Science Foundation of China (Nos. 61602397, 61841103), The Natural Science Foundation of Hunan Province (2017JJ2251, 2017JJ3315), and Chinese Scholar-ship Council of the Ministry of Education.

Compliance with ethical standards

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2010) Slic superpixels. In Technical report, EPFLGoogle Scholar
  2. Brox T, Malik J (2010) Object segmentation by long term analysis of point trajectories. In: Proceedings of European conference on computer vision.
  3. Chen L, Papandreou G, Kokkinos I, Murphy K et al (2016) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848CrossRefGoogle Scholar
  4. Chen L, Zhu Y, Papandreou G, et al (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. Preprint arXiv:1802.02611
  5. Cheng B, Liu G, Wang J, et al (2011). Multi-task low-rank affinity pursuit for image segmentation. In Proceedings of IEEE international conference on computer vision, pp 2439–2446Google Scholar
  6. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 886–893Google Scholar
  7. Duta I, Uijlings J, Nguyen T, et al (2016) Histograms of motion gradients for real-time video classification. In: International workshop on content-based multi-media indexing.
  8. Farnoush Z, Borislav A, Jan S (2018). Superpixel-based road segmentation for real-time systems using CNN. In: Proceedings of the 13th international joint conference on computer vision, imaging and computer graphics theory and applications (VISIGRAPP), pp 257–265Google Scholar
  9. Felzenszwalb P, Huttenlocher D (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167–181CrossRefGoogle Scholar
  10. Galasso F, Cipolla R, Schiele B (2012) Video segmentation with superpixels. In: Proceedings of the Asian conference on computer vision, pp 760–774Google Scholar
  11. Galasso N, Nagaraja J, Cardenas T, Brox B, Schiele A (2013) Unified video segmentation benchmark: annotation, metrics and analysis. In: International conference on computer vision.
  12. Grundmann M, Kwatra V, Han M, et al (2010) Efficient hierarchical graph-based video segmentation. In Proceedings of IEEE conference on computer vision and pattern recognition, pp 2141–2148Google Scholar
  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. arXiv:1512.03385.
  14. Konstantinos G (2008) The bhattacharyya. Measure, Version 1.0, March 20Google Scholar
  15. Li C, Lin L, Zuo W, Wang W, Tang J,Yan S (2015) SOLD: sub-optimal low-rank decomposition for efficient video segmentation. In Proceedings of IEEE conference on computer vision and pattern recognition, Boston, MA, USA, pp 5519–5527.
  16. Li T, Bin Cheng B, Ni B et al (2016a) Multitask low-rank affinity graph for image segmentation and image annotation. ACM Trans Intell Syst Technol 7(4):1–18CrossRefGoogle Scholar
  17. Li C, Lin L, Zuo W, Wang W, Tang J (2016b) An approach to streaming video segmentation with sub-optimal low-rank decomposition. IEEE Trans Image Process T-IP 25(5):1947–1960MathSciNetCrossRefGoogle Scholar
  18. Liu R, Lin Z, Torre F, Su Z (2012) Fixed-rank representation for unsupervised visual learning. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 598–605Google Scholar
  19. Liu G, Lin Z, Yan S et al (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184CrossRefGoogle Scholar
  20. Luc P, Couprie C, Chintala S, et al (2016) Semantic segmentation using adversarial networks. In: NIPS-2016 NIPS workshop on adversarial training, Barcelona, Spain. arXiv:1611.08408
  21. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Med Image Comput Comput-Assist Interv (MICCAI) 9351:234–241Google Scholar
  22. Das A, Ghosh S, Sarkhel R, et al. (2018) Combining multi-level contexts of superpixel using convolutional neural networks to perform natural scene labeling. arXiv:1803.05200
  23. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651CrossRefGoogle Scholar
  24. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans PAMI 22(8):888–905CrossRefGoogle Scholar
  25. Wang L, Dong M (2012) Multi-level low-rank approximation based spectral clustering for image segmentation. Pattern Recognit Lett 33(16):2206–2215CrossRefGoogle Scholar
  26. Wang Y, Jiang Y, Wu Y et al (2011) Spectral clustering on multiple manifolds. IEEE Trans Neural Netw 22(7):1149–1161CrossRefGoogle Scholar
  27. Wen Z, Yin W, Zhang Y (2010) Solving a low-rank factorization model for matrix completion by a non-linear successive over-relaxation algorithm. Rice CAAM Tech Report TR10-07Google Scholar
  28. Xu C, Corso J (2012). Evaluation of super-voxel methods for early video processing. In: Proceedings of IEEE conference on computer vision and pattern recognition.
  29. Xu H, Zhou W, Wang Y, Wang W, Mo Y (2017) Matrix separation based on lmafit-seed. Comput J 60(11):1609–1618MathSciNetCrossRefGoogle Scholar
  30. Yin M, Gao J, Lin Z (2016) Laplacian regularized low-rank representation and its applications. IEEE Trans Pattern Anal Mach Intell 38(3):504–517CrossRefGoogle Scholar
  31. Zhang T, Ghanem B, Liu S, et al (2013) Low-rank sparse coding for image classification. In Proceedings of IEEE international conference on computer vision, pp 281–288Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Key Laboratory of Intelligent Computing and Information Processing, Ministry of Education, College of Information EngineeringXiangtan UniversityXiangtanChina
  2. 2.Department of Computer ScienceUniversity of YorkYorkUK

Personalised recommendations