Abstract
Segmenting an image is an important step in many computer vision applications. However, image segmentation evaluation is far from being well-studied in contrast to the extensive studies on image segmentation algorithms. In this paper, we propose a framework to quantitatively evaluate the quality of a given segmentation with multiple ground truth segmentations. Instead of comparing directly the given segmentation to the ground truths, we assume that if a segmentation is “good”, it can be constructed by pieces of the ground truth segmentations. Then for a given segmentation, we construct adaptively a new ground truth which can be locally matched to the segmentation as much as possible and preserve the structural consistency in the ground truths. The quality of the segmentation can then be evaluated by measuring its distance to the adaptively composite ground truth. To the best of our knowledge, this is the first work that provides a framework to adaptively combine multiple ground truths for quantitative segmentation evaluation. Experiments are conducted on the benchmark Berkeley segmentation database, and the results show that the proposed method can faithfully reflect the perceptual qualities of segmentations.
Chapter PDF
Similar content being viewed by others
References
Liu, J., Yang, Y.: Multiresolution color image segmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence 16, 689–700 (1994)
Borsotti, M., Campadelli, P., Schettini, R.: Quantitative evaluation of color image segmenta- tion results. Pattern Recognition Letter 19, 741–747 (1998)
Zhang, H., Fritts, J., Goldman, S.: An entropy-based objective segmentation eval- uation method for image segmentation. In: SPIE Electronic Imaging Storage and Retrieval Methods and Applications for Multimedia, pp. 38–49 (2004)
Ren, X., Jitendra, M.: Learning a classification model for segmentation. In: IEEE Conference on Computer Vision, pp. 10–17 (2003)
Christensen, H., Phillips, P.: Empirical evaluation methods in computer vision. World Scientific Publishing Company (2002)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: International Conference on Computer Vision, pp. 416–423 (2001)
Freixenet, J., Muñoz, X., Raba, D., Martí, J., Cufí, X.: Yet Another Survey on Image Segmentation: Region and Boundary Information Integration. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part III. LNCS, vol. 2352, pp. 408–422. Springer, Heidelberg (2002)
Martin, D.: An empirical approach to grouping and segmentation. Ph.D. dissertation U.C. Berkeley (2002)
Rand, W.: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 66, 846–850 (1971)
Fowlkes, E., Mallows, C.: A method for comparing two hierarchical clusterings. Journal of the American Statistical Association 78, 553–569 (1983)
Unnikrishnan, R., Hebert, M.: Measures of similarity. In: IEEE Workshop on Applications of Computer Vision, pp. 394–400 (2005)
Felzenszwalb, P., Huttenlocher, D.: Efficient graph-based image segmentation. International Journal on Computer Vision 59, 167–181 (2004)
Viola, P., Wells, W.: Alignment by maximization of mutual information, vol. 3, pp. 16–23 (1995)
Wang, Z., Bovik, A.: Modern image quality assessment. Morgan and Claypool Publishing Company, New York (2006)
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Process. 13, 600–612 (2004)
Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., Cohen, M.: Interactive digital photomontage, vol. 23, pp. 294–302 (2004)
Russell, B., Efros, A., Sivic, J., Freeman, W., Zisserman, A.: Segmenting scenes by matching image composites. In: Advances in Neural Information Processing Systems, pp. 1580–1588 (2009)
Potts, R.: Some generalized order-disorder transformation. Proceedings of the Cambridge Philosophical Society 48, 106–109 (1952)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 1222–1239 (2001)
Martin, D., Fowlkes, C., Malik, J.: Learning to detect natural image boundaries using local brightness, color and texture cues. IEEE Trans. on Pattern Analysis and Machine Intelligence 26, 530–549 (2004)
Sampat, M., Wang, Z., Gupta, S., Bovik, A., Markey, M.: Complex wavelet structural similarity: A new image similarity index. IEEE Transactions on Image Processing 18, 2385–2401 (2009)
Comanicu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence 24, 603–619 (2002)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 888–905 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Peng, B., Zhang, L. (2012). Evaluation of Image Segmentation Quality by Adaptive Ground Truth Composition. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7574. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33712-3_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-33712-3_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33711-6
Online ISBN: 978-3-642-33712-3
eBook Packages: Computer ScienceComputer Science (R0)