Error Metrics for Quantitative Evaluation of Medical Image Segmentation

Abstract

There is a large effort to construct algorithms performing objective, reproducible segmentations of medical image data. However, the introduction of these techniques into clinical practice has been hampered by the lack of thorough evaluation of performance. If the gold standard is defined by human performance, a full validation study is costly, since it should comprise both a large number of datasets and a large number of trained medical experts to inspect these. In this case one should compare the weighted average of the observers with the result of the algorithm. An algorithm performs well enough if it is closer to the average than the variance of the medical experts. Three important steps in this procedure are selecting the parameter to be compared, determining a gold standard (the weighted average) and an error metric to define how much an individual result differs from the gold standard. The different steps of this procedure are illustrated by a clinical evaluation of different techniques for segmenting intravascular ultrasound images.