Normalized Joint Mutual Information Measure for Image Segmentation Evaluation with Multiple Ground-Truth Images

Abstract

Supervised or ground-truth-based image segmentation evaluation paradigm plays an important role in objectively evaluating segmentation algorithms. So far, many evaluation methods in terms of comparing clusterings in machine learning field have been developed. Being different from recognition task, image segmentation is considered an ill-defined problem. In a hand-labeled segmentations dataset, for the same image, different human subjects always produce various segmented results, leading to more than one ground-truth segmentations for an image. Thus, it is necessary to extend the traditional pairwise similarity measures that compare a machine generated clustering and a “true” clustering to handle multiple ground-truth clusterings. In this paper, based on the Normalized Mutual Information (NMI) which is a popular information theoretic measure for clustering comparison, we propose to utilize the Normalized Joint Mutual Information (NJMI), an extension of the NMI, to achieve the goal mentioned above. We illustrate the effectiveness of NJMI for objective segmentation evaluation with multiple ground-truth segmentations by testing it on images from Berkeley segmentation dataset.