Multimedia Tools and Applications

, Volume 75, Issue 6, pp 2969–2987 | Cite as

An integrated similarity metric for graph-based color image segmentation



Graph-based method has become one of the major trends in image segmentation. In this paper, we focus on how to build the affinity matrix which is one of the key issues in graph-based color image segmentation. Four different metrics are integrated in order to build an effective affinity matrix for segmentation. First, the quaternion-based color distance is utilized to measure color differences between color pixels and the oversegmented regions (superpixels), which is more accurate than the commonly used Euclidean distance. In order to describe the superpixels well, especially for texture images, we combine the mean and the variance information to represent the superpixels. Then the image boundary information is used to merge the oversegmented regions to preserve the image edge and reduce the computational complexity. An object for recognition may be cut into nonadjacent sub-parts by clutter or shadows, the affinities between adjacent and nonadjacent superpixels are computed in our study. This feature of affinity is not considered in other methods which only consider the similarity of adjacent regions. Experimental results on the Berkeley segmentation dataset (BSDS) and Weizmann segmentation evaluation datasets demonstrate the superiority of the proposed approach compared with some existing popular image segmentation methods.


Graph-based method Image segmentation Similarity metrics Quaternion 



The authors would like to thank the anonymous reviewers for their insightful suggestions, and Dr. Chih-Cheng Hung for his valuable comments and suggestions. This work was supported by the National Natural Science Foundation of China (Grant Nos. 61370181, 61075010 and 61370179), and National Key Technology Research and Development Program of China (Grant No. 2012BAI23B07).


  1. 1.
    Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intel 34(11):2274–2282CrossRefGoogle Scholar
  2. 2.
    Alpert S, Galun M, Basri R, Brandt A (2007) Image segmentation by probabilistic bottom-up aggregation and cue integration. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition, pp 1-8Google Scholar
  3. 3.
    Arbelaez P, Maire M, Fowlkes C, Malik J (2009) From contours to regions: An empirical evaluation. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition, pp 2294-2301Google Scholar
  4. 4.
    Cai C, Mitra SK (2000) A normalized color difference edge detector based on quaternion representation. Paper presented at the International Conference on Image Processing, pp 816-819Google Scholar
  5. 5.
    Collins MD, Xu J, Grady L, Singh V (2012) Random walks based multi-image segmentation: Quasiconvexity results and GPU-based solutions. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition, pp 1656-1663Google Scholar
  6. 6.
    Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intel 24(5):603–619CrossRefGoogle Scholar
  7. 7.
    Cour T, Benezit F, Shi J (2005) Spectral segmentation with multiscale graph decomposition. Paper presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 1124-1131Google Scholar
  8. 8.
    Deng Y, Manjunath B (2001) Unsupervised segmentation of color-texture regions in images and video. IEEE Trans Pattern Anal Mach Intel 23(8):800–810CrossRefGoogle Scholar
  9. 9.
    Donoser M, Urschler M, Hirzer M, Bischof H (2009) Saliency driven total variation segmentation. Paper presented at the IEEE 12th International Conference on Computer Vision, pp 817-824Google Scholar
  10. 10.
    Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167–181CrossRefGoogle Scholar
  11. 11.
    Freixenet J, Muñoz X, Raba D, Martí J, Cufí X (2002) Yet another survey on image segmentation: Region and boundary information integration. In: Computer Vision—ECCV 2002. Springer, pp 408-422Google Scholar
  12. 12.
    Gastal ES, Oliveira MM (2012) Adaptive manifolds for real-time high-dimensional filtering. ACM Trans Graph (TOG) 31(4):1–13CrossRefGoogle Scholar
  13. 13.
    Hamilton WR (1866) Elements of quaternions. Longmans, Green, & Company, LondonGoogle Scholar
  14. 14.
    Haris K, Efstratiadis SN, Maglaveras N, Katsaggelos AK (1998) Hybrid image segmentation using watersheds and fast region merging. IEEE Trans Image Process 7(12):1684–1699CrossRefGoogle Scholar
  15. 15.
    Jahne B (2002) Digital image processing. Springer, BerlinCrossRefMATHGoogle Scholar
  16. 16.
    Jin L, Liu H, Xu X, Song E (2012) Improved direction estimation for Di Zenzo’s multichannel image gradient operator. Pattern Recogn 45(12):4300–4311CrossRefMATHGoogle Scholar
  17. 17.
    Jin L, Liu H, Xu X, Song E (2013) Quaternion-based impulse noise removal from color video sequences. IEEE Trans Circ Syst Video Technol 23(5):741–755CrossRefGoogle Scholar
  18. 18.
    Kim TH, Lee KM, Lee SU (2013) Learning full pairwise affinities for spectral segmentation. IEEE Trans Pattern Anal Mach Intel 35(7):1690–1703CrossRefGoogle Scholar
  19. 19.
    Levinshtein A, Stere A, Kutulakos KN, Fleet DJ, Dickinson SJ, Siddiqi K (2009) Turbopixels: fast superpixels using geometric flows. IEEE Trans Pattern Analy Mach Intel 31(12):2290–2297CrossRefGoogle Scholar
  20. 20.
    Li Z, Wu X-M, Chang S-F (2012) Segmentation using superpixels: A bipartite graph partitioning approach. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition, pp 789-796Google Scholar
  21. 21.
    Makrogiannis S, Economou G, Fotopoulos S (2005) A region dissimilarity relation that combines feature-space and spatial information for color image segmentation. IEEE Trans Syst Man Cybernet B Cybernet 35(1):44–53CrossRefGoogle Scholar
  22. 22.
    Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Paper presented at the Eighth IEEE International Conference on Computer Vision, pp 416-423Google Scholar
  23. 23.
    Martin DR, Fowlkes CC, Malik J (2004) Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans Pattern Anal Mach Intel 26(5):530–549CrossRefGoogle Scholar
  24. 24.
    Meilǎ M (2005) Comparing clusterings: an axiomatic view. Paper presented at the Proceedings of the 22nd international conference on Machine learning, pp 577-584Google Scholar
  25. 25.
    Mobahi H, Rao SR, Yang AY, Sastry SS, Ma Y (2011) Segmentation of natural images by texture and boundary compression. Int J Comput Vis 95(1):86–98CrossRefGoogle Scholar
  26. 26.
    Nguyen HT, Worring M, Dev A (2000) Detection of moving objects in video using a robust motion similarity measure. IEEE Trans Image Process 9(1):137–141CrossRefGoogle Scholar
  27. 27.
    Pei S-C, Cheng C-M (1997) A novel block truncation coding of color images using a quaternion-moment-preserving principle. IEEE Trans Commun 45(5):583–595CrossRefGoogle Scholar
  28. 28.
    Rand WM (1971) Objective criteria for the evaluation of clustering methods. J Am Stat Assoc 66(336):846–850CrossRefGoogle Scholar
  29. 29.
    Sangwine SJ (1996) Fourier transforms of colour images using quaternion or hypercomplex, numbers. Electron Lett 32(21):1979–1980CrossRefGoogle Scholar
  30. 30.
    Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intel 22(8):888–905CrossRefGoogle Scholar
  31. 31.
    Subakan ÖN, Vemuri BC (2011) A quaternion framework for color image smoothing and segmentation. Int J Comput Vis 91(3):233–250MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Tao W, Jin H, Zhang Y (2007) Color image segmentation based on mean shift and normalized cuts. IEEE Trans Syst Man Cybernet B Cybernet 37(5):1382–1389CrossRefGoogle Scholar
  33. 33.
    Tilton JC (1998) Image segmentation by region growing and spectral clustering with natural convergence criterion. Paper presented at the International geoscience and remote sensing symposium, pp 1766-1768Google Scholar
  34. 34.
    Tilton JC, Tarabalka Y, Montesano PM, Gofman E (2012) Best merge region-growing segmentation with integrated nonadjacent region object aggregation. IEEE Trans Geosci Remote Sens 50(11):4454–4467CrossRefGoogle Scholar
  35. 35.
    Unnikrishnan R, Pantofaru C, Hebert M (2007) Toward objective evaluation of image segmentation algorithms. IEEE Trans Pattern Anal Mach Intel 29(6):929–944CrossRefGoogle Scholar
  36. 36.
    Vedaldi A, Soatto S (2008) Quick shift and Kernel methods for mode seeking. In: Forsyth D, Torr P, Zisserman A (eds) Computer vision – ECCV 2008, vol 5305. lecture notes in computer science. Springer, Berlin, pp 705–718. doi: 10.1007/978-3-540-88693-8_52 Google Scholar
  37. 37.
    Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416MathSciNetCrossRefGoogle Scholar
  38. 38.
    Wang C, Guo Y, Zhu J, Wang L, Wang W (2014) Video object co-segmentation via subspace clustering and quadratic pseudo-boolean optimization in an MRF framework. IEEE Trans Multimed 16(4):903–916MathSciNetCrossRefGoogle Scholar
  39. 39.
    Wang J, Jia Y, Hua X-S, Zhang C, Quan L (2008) Normalized tree partitioning for image segmentation. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition, pp 1-8Google Scholar
  40. 40.
    Wang S, Siskind JM (2003) Image segmentation with ratio cut. IEEE Trans Pattern Anal Mach Intel 25(6):675–690CrossRefGoogle Scholar
  41. 41.
    Wu Z, Leahy R (1993) An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. IEEE Trans Pattern Anal Mach Intel 15(11):1101–1113CrossRefGoogle Scholar
  42. 42.
    Xia T, Cao J, Zhang Y, Li J (2009) On defining affinity graph for spectral clustering through ranking on manifolds. Neurocomputing 72(13):3203–3211CrossRefGoogle Scholar
  43. 43.
    Zhang X, Li J, Yu H (2011) Local density adaptive similarity measurement for spectral clustering. Pattern Recogn Lett 32(2):352–358CrossRefGoogle Scholar
  44. 44.
    Zhu S-Y, Plataniotis KN, Venetsanopoulos AN (1999) Comprehensive analysis of edge detection in color image processing. Opt Eng 38(4):612–625CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Xiang Li
    • 1
    • 2
  • Lianghai Jin
    • 1
    • 2
  • Enmin Song
    • 1
    • 2
  • Zeng He
    • 3
  1. 1.School of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanPeople’s Republic of China
  2. 2.Key Laboratory of Education Ministry for Image Processing and Intelligent ControlWuhanPeople’s Republic of China
  3. 3.School of Civil Engineering and MechanicsHuazhong University of Science and TechnologyWuhanPeople’s Republic of China

Personalised recommendations