Face in a Crowd

Abstract

We have considered so far face recognition in isolation. The face, already captured, was available to identification or authentication. This is characteristic of controlled situations, e.g., security applications using machine readable travel documents (MRTD). It does not come as a big surprise to find that real life is much more complicated. The real challenge thus is to automatically identify a person without her active cooperation. The face has to be detected first and tracked before any attempt is made to identify or authenticate it. This problem, usually referred to as Face in a Crowd, is most challenging. There is much more variability, e.g., clutter, illumination, pose, and scale, compared to what one encounters during standard face recognition applications. The news are not all that bad because at the cost of increased variability one gains access to the temporal dimension, to accrue and smooth evidence that helps to disambiguate among alternative facial interpretations. Evidence accumulation, always watchful for change and ready to integrate it, is both progressive and adaptive. It is progressive because it is selective about what to look for and in what order, on one hand, and how to process it, on the other hand. It is adaptive because it learns how to combine exploration and exploitation in order to be effective and nimble. In addition, and perhaps most important, the temporal dimension begets coherence across spatial-temporal manifolds to constrain what is feasible and probable from what is either impossible or unlikely. Coherence supports invariance and makes tracking and recognition complementary to each other. This chapter addresses the time dimension and how to take advantage of it (see Sect. 8.4 on the complementary use of 3D for similar purposes). Time connects sensing and perception, anticipation and control, exploration and exploitation, and adaptation and learning.