Error Analysis

Abstract

Performance evaluation, discussed in the previous chapter, was mostly about the comparative evaluation of competing face recognition systems. Error analysis explores here complementary objectives. First, for operational purposes, there is the acute need to predict performance in order to properly assess the utility in deploying a particular face recognition system. This is in sharp contrast to a technological evaluation that has access to ground truth and merely records the results. Bounds on expected performance and how the results scale up for larger data sets are also sought. Second, the errors have to be analyzed and their source understood. In particular, it is important to know not only what works and to what extent it works, but also to know what does not work and why (Pankanti et al., 2002). Anecdotal evidence suggests that 90% of errors are due to only I0% of the face patterns. The faces that can be easily remembered are not necessarily those that are easily rejected when they have not been seen previously. The correlation between the hit rate and the false positive rate is zero (Bruce and Young, 1998). “Our current understanding of the performance evaluation facilitates a comparative assessment of the candidate solutions on a particular dataset but does not permit us to objectively predict the system performance in the real world. Neither do we have the capability of objectively describing the existing databases nor any means of quantifying/ comparing their complexities independent of a fully implemented matching system. Acquisition of these capabilities will indicate a level of understanding of the data that will instill end-user trust and result in a more widespread acceptance of the [biometrics] system” (Jain and Pankanti, 2001).