Superpixel via coarse-to-fine boundary shift

Abstract

K-means is used by numerous superpixel algorithms, such as SLIC, MSLIC and LSC, because of its simplicity and efficiency. Yet those k-means based algorithm failed to perform well on connectivity and accuracy. In this paper, we propose a coarse-to-fine boundary shift strategy (CFBS) as a replacement of k-means. The CFBS solves the superpixel segmentation problem by shifting boundries rather than clustering pixels. In other words, it can be defined as a special k-means algorithm optimized for superpixel segmentation. By replacing k-means with CFBS, SLIC and LSC are upgraded to NeoSLIC and NeoLSC. Experiments show that NeoSLIC and NeoLSC outperform SLIC and LSC in accuracy and efficiency respectively, and NeoSLIC and NeoLSC alleviate dis-connectivity. In addition, experiments also show that CFBS achieves great improvements on semantic segmentation, class segmentation and segmented flow.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. 1.

    Liu S, Zhang L, Zhang Z, Wang C, Xiao B (2015) Automatic cloud detection for all-sky images using superpixel segmentation. Geosci Remote Sens Lett 12(2):354–358

    Article  Google Scholar 

  2. 2.

    Cheng M-M, Zhang G-X, Mitra NJ, Huang X, Hu Shi-Min (2011) Global contrast based salient region detection. In: 2011 IEEE conference on computer vision and pattern recognition. IEEE, pp 409–416

  3. 3.

    Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 1597–1604

  4. 4.

    Levinshtein A, Sminchisescu C, Dickinson S (2010) Optimal contour closure by superpixel grouping. In: Lecture notes in computer science. Springer, pp 480–493

  5. 5.

    Fulkerson B, Vedaldi A, Soatto S (2009) Class segmentation and object localization with superpixel neighborhoods. In: 2009 IEEE international conference on computer vision. IEEE, pp 670–677

  6. 6.

    Wang S, Lu H, Yang F, Yang M-H (2011) Superpixel tracking. In: 2011 IEEE international conference on computer vision. IEEE, pp 1323–1330

  7. 7.

    Mičušík B, Košecká J (2010) Multi-view superpixel stereo in urban environments. Int J Comput Vis 89(1):106–119

    Article  Google Scholar 

  8. 8.

    Gould S, Rodgers J, Cohen D, Elidan G, Koller D (2008) Multi-class segmentation with relative location prior. Int J Comput Vis 80(3):300–316

    Article  Google Scholar 

  9. 9.

    Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  10. 10.

    Chen J, Li Z, Bo H (2017) Linear spectral clustering superpixel. IEEE Trans Image Process 12(1):32–38

    MathSciNet  MATH  Google Scholar 

  11. 11.

    Liu Y-J, Yu C-C, Yu M-J, He Y (2016) Manifold SLIC : a fast method to compute content-sensitive superpixels. In: 2016 IEEE conference on computer vision and pattern recognition. IEEE, pp 651–659

  12. 12.

    Wang P, Zeng G, Gan R, Wang J, Zha H (2013) Structure-sensitive superpixels via geodesic distance. Int J Comput Vis 103(1):1–21

    MathSciNet  MATH  Article  Google Scholar 

  13. 13.

    Giraud R, Ta VT, Papadakis N (2018) Robust superpixels using color and contour features along linear path. Comput Vis Image Underst 170(123):1–13

    Article  Google Scholar 

  14. 14.

    Neubert P, Protzel P (2012) Superpixel benchmark and comparison. In: Forum Bildverarbeitung. IEEE, pp 1–12

  15. 15.

    Wang M, Liu X, Gao Y, Ma X, Soomro NQ (2017) Superpixel segmentation: a benchmark. Signal Process: Image Commun 56(34):28–39

    Google Scholar 

  16. 16.

    Moore AP, Prince SJD, Warrell J, Mohammed U, Jones G (2008) Superpixel lattices. In: 2008 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8

  17. 17.

    Yao J, Boben M, Fidler S, Urtasun R (2015) Real-time coarse-to-fine topologically preserving segmentation. In: 2015 IEEE conference on computer vision and pattern recognition. IEEE, pp 2947–2955

  18. 18.

    Tang D, Fu H, Cao X (2012) Topology preserved regular superpixel. In: 2012 IEEE international conference on multimedia and expo. IEEE, pp 765–768

  19. 19.

    Lai T, Fujita H, Yang C, Li Q, Chen R (2019) Robust model fitting based on greedy search and specified inlier threshold. IEEE Transactions on Industrial Electronics

  20. 20.

    Lai T, Chen R, Yang C, Li Q, Fujita H, Sadri A, Wang H (2019) Efficient robust model fitting for multistructure data using global greedy search. IEEE Trans Cybern 1–13

  21. 21.

    Chen Y, Yue X, Fujita H, Fu S (2017) Three-way decision support for diagnosis on focal liver lesions. Knowledge-based Systems 127:85–99

    Article  Google Scholar 

  22. 22.

    Achanta R, Susstrunk S (2017) Superpixels and polygons using simple non-iterative clustering. In: 2017 IEEE conferenceon computer vision and pattern recognition. IEEE, pp 4895– 4904

  23. 23.

    Van den Bergh M, Boix X, Roig G, Van Gool L (2013) SEEDS: superpixels extracted via energy-driven sampling. Lecture Notes in Computer Science 7(7):13–26

    Google Scholar 

  24. 24.

    Liu M-Y, Tuzel O, Ramalingam S, Chellappa R (2011) Entropy rate superpixel segmentation. In: 2011 IEEE conference on computer vision and pattern recognition. IEEE, pp 2097–2104

  25. 25.

    Arbelaez P, Maire M, Fowlkes C, Malik J (2009) From contours to regions: an empirical evaluation. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 2294–2301

  26. 26.

    Stutz D, Hermans A, Leibe B (2018) Superpixels: an evaluation of the state-of-the-art. Computer Vision and Image Understanding 166(12):1–27

    Article  Google Scholar 

  27. 27.

    Shen J, Hao X, Liang Z, Yu L, Wang W, Shao L (2016) Real-Time superpixel segmentation by DBSCAN clustering algorithm. IEEE Trans Image Process 25(12):5933–5942

    MathSciNet  MATH  Article  Google Scholar 

  28. 28.

    Farabet C, Couprie C, Najman L, LeCun Y (2013) Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell 35(8):1915–1929

    Article  Google Scholar 

  29. 29.

    Gould S, Fulton R, Koller D (2009) Decomposing a scene into geometric and semantically consistent regions. In: 2009 IEEE international conference on computer vision. IEEE, pp 1–8

  30. 30.

    Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167–181

    Article  Google Scholar 

  31. 31.

    Boykov Y, Kolmogorov V (2004) An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell 26(9):1124–1137

    MATH  Article  Google Scholar 

  32. 32.

    Opelt A, Pinz A, Fussenegger M, Auer P (2006) Generic object recognition with boosting. IEEE Trans Pattern Anal Mach Intell 28(3):416–431

    MATH  Article  Google Scholar 

  33. 33.

    Wulff J, Butler DJ, Stanley GB, Black MJ (2012) Lessons and insights from creating a synthetic optical flow benchmark. In: Lecture notes in computer science

  34. 34.

    Butler Daniel J. , Wulff J, Stanley GB, Black MJ (2012) A naturalistic open source movie for optical flow evaluation. In: Lecture notes in computer science. IEEE, pp 611–625

  35. 35.

    Jampani V, Sun D, Liu M-Y, Yang M-H, Kautz J (2018) Superpixel Sampling Networks. In: 2018 European conference on computer vision, vol 11. Springer, pp 363–380

Download references

Acknowledgment

This work was supported by the National High Technology Research, Development Program of China (No.2017YFB1401600), the National Natural Science Foundation of China (No.61573235, 61703260), the Shanghai Innovation Action Project of Science and Technology (No.18511107400), and the Fundamental Research Funds for the Central Universities.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yufei Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Chen, Y., Liu, X. et al. Superpixel via coarse-to-fine boundary shift. Appl Intell 50, 2079–2092 (2020). https://doi.org/10.1007/s10489-019-01595-1

Download citation

Keywords

  • Superpixel
  • Slic
  • K-means