Abstract
In spite of over two decades of intense research, illumination and pose invariance remain prohibitively challenging aspects of face recognition for most practical applications. The objective of this work is to recognize faces using video sequences both for training and recognition input, in a realistic, unconstrained setup in which lighting, pose and user motion pattern have a wide variability and face images are of low resolution. In particular there are three areas of novelty: (i) we show how a photometric model of image formation can be combined with a statistical model of generic face appearance variation, learnt offline, to generalize in the presence of extreme illumination changes; (ii) we use the smoothness of geodesically local appearance manifold structure and a robust same-identity likelihood to achieve invariance to unseen head poses; and (iii) we introduce an accurate video sequence “reillumination” algorithm to achieve robustness to face motion patterns in video. We describe a fully automatic recognition system based on the proposed method and an extensive evaluation on 171 individuals and over 1300 video sequences with extreme illumination, pose and head motion variation. On this challenging data set our system consistently demonstrated a nearly perfect recognition rate (over 99.7%), significantly outperforming state-of-the-art commercial software and methods from the literature.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Adini, Y., Moses, Y., Ullman, S.: Face recognition: The problem of compensating for changes in illumination direction. PAMI 19(7), 721–732 (1997)
Arandjelović, O., Cipolla, R.: A new look at filtering techniques for illumination invariance in automatic face recogition. In: FG (2006)
Arandjelović, O., Shakhnarovich, G., Fisher, J., Cipolla, R., Darrell, T.: Face recognition with image sets using manifold density divergence. In: CVPR (2005)
Arandjelović, O., Zisserman, A.: Automatic face recognition for film character retrieval in feature-length films. In: CVPR, vol. 1, pp. 860–867 (2005)
Barrett, W.A.: A survey of face recognition algorithms and testing results. Systems and Computers 1, 301–305 (1998)
BBC. Doubts over passport face scans. BBC Online, UK Edition (October 2004)
Belhumeur, P.N., Kriegman, D.J.: What is the set of images of an object under all possible illumination conditions? IJCV 28(3), 245–260 (1998)
Berg, T.L., Berg, A.C., Edwards, J., Maire, M., White, R., Teh, Y.W., Learned-Miller, E., Forsyth, D.A.: Names and faces in the news. In: CVPR, vol. 2, pp. 848–854 (2004)
Bichsel, M., Pentland, A.P.: Human face recognition and the face image set’s topology. Computer Vision, Graphics and Image Processing 59(2), 254–261 (1994)
Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. PAMI 25(9), 1063–1074 (2003)
Craw, I., Costen, N.P., Kato, T., Akamatsu, S.: How should we represent faces for automatic recognition? PAMI 21, 725–736 (1999)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wily & Sons, Inc., New York (2000)
Everingham, M., Zisserman, A.: Automated person identification in video. In: Enser, P.G.B., Kompatsiaris, Y., O’Connor, N.E., Smeaton, A.F., Smeulders, A.W.M. (eds.) CIVR 2004. LNCS, vol. 3115, pp. 289–298. Springer, Heidelberg (2004)
Fitzgibbon, A.W., Zisserman, A.: On affine invariant clustering and automatic cast listing in movies. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 304–320. Springer, Heidelberg (2002)
Fukui, K., Yamaguchi, O.: Facial feature point extraction method based on combination of shape extraction and pattern matching. Systems and Computers in Japan 29(6), 2170–2177 (1998)
Fukui, K., Yamaguchi, O.: Face recognition using multi-viewpoint patterns for robot vision. In: Int’l Symp. of Robotics Research (2003)
Gavrila, D.M.: Pedestrian detection from a moving vehicle. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 37–49. Springer, Heidelberg (2000)
Georghiades, A.S., Kriegman, D.J., Belhumeur, P.N.: Illumination cones for recognition under variable lighting: Faces. In: CVPR, pp. 52–59 (1998)
Gorodnichy, D.O.: Associative neural networks as means for low-resolution video-based recognition. In: International Joint Conference on Neural Networks (2005)
Gross, R., Matthews, I., Baker, S.: Generic vs. person specific active appearance models. In: BMVC (2004)
Hall, P., Marshall, D., Martin, R.: Merging and splitting eigenspace models. PAMI 22(9), 1042–1049 (2000)
Identix. Faceit, http://www.FaceIt.com/
Kepenekci, B.: Face Recognition Using Gabor Wavelet Transform. PhD thesis, The Middle East Technical University (2001)
Kim, T., Arandjelović, O., Cipolla, R.: Learning over sets using boosted manifold principal angles (BoMPA). In: BMVC (to appear, 2005)
Kimmel, R., Elad, M., Shaked, D., Keshet, R., Sobel, I.: A variational framework for retinex. IJCV 52(1), 7–23 (2003)
Lee, K., Kriegman, D.: Online learning of probabilistic appearance manifolds for video-based recognition and tracking. In: CVPR, vol. 1, pp. 852–859 (2005)
Lee, K., Yang, M., Kriegman, D.: Video-based face recognition using probabilistic appearance manifolds. In: CVPR, vol. 1, pp. 313–320 (2003)
Li, Y., Gong, S., Liddell, H.: Modelling faces dynamically across views and over time. ICCV 1, 554–559 (2001)
Liu, X., Chen, T.: Video-based face recognition using adaptive hidden Markov models. In: CVPR, vol. 1, pp. 340–345 (2003)
Murase, H., Nayar, S.: Visual learning and recognition of 3-D objects from appearance. IJCV 14, 5–24 (1995)
Palanivel, S., Venkatesh, B.S., Yegnanarayana, B.: Real time face recognition system using autoassociative neural network models. ASSP 2, 833–836 (2003)
Pentland, A., Moghaddam, B., Starner, T.: View-based and modular eigenspaces for face recognition. In: CVPR, pp. 84–91 (1994)
Phillips, P.J., Grother, P., Micheals, R.J., Blackburn, D.M., Tabassi, E., Bone, J.M.: FRVT 2002: Overview and summary. Technical report, National Institute of Justice (March 2003)
Riklin-Raviv, T., Shashua, A.: The quotient image: Class based re-rendering and recognition with varying illuminations. PAMI 23(2), 219–239 (2001)
Shakhnarovich, G., Fisher III, J.W., Darrell, T.: Face recognition from long-term observations. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 851–865. Springer, Heidelberg (2002)
Sim, T., Zhang, S.: Exploring face space. Face Processing in Video (2004)
Sivic, J., Everingham, M., Zisserman, A.: Person spotting: Video shot retrieval for face sets. In: Leow, W.-K., Lew, M., Chua, T.-S., Ma, W.-Y., Chaisorn, L., Bakker, E.M. (eds.) CIVR 2005. LNCS, vol. 3568, pp. 226–236. Springer, Heidelberg (2005)
Stenger, B., Thayananthan, A., Torr, P., Cipolla, R.: Filtering using a tree-based estimator. ICCV 2, 1063–1070 (2003)
Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Tipping, M.E., Bishop, C.M.: Mixtures of probabilistic principal component analyzers. Neural Computation 11(2), 443–482 (1999)
Toshiba. Facepass, http://www.toshiba.co.jp/mmlab/tech/w31e.htm
Viola, P., Jones, M.: Robust real-time face detection. IJCV 57(2) (2004)
Wang, H., Li, S.Z., Wang, Y.: Face recognition under varying lighting conditions using self quotient image. In: FG, pp. 819–824 (2004)
Wang, X., Tang, X.: Unified subspace analysis for face recognition. In: ICCV (2003)
Wolf, L., Shashua, A.: Learning over sets using kernel principal angles. JMLR 4(10), 913–931 (2003)
Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: A literature survey. ACM Computing Surveys 35(4), 399–458 (2004)
Zhou, S., Krueger, V., Chellappa, R.: Probabilistic recognition of human faces from video. Computer Vision and Image Understanding 91(1), 214–245 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Arandjelović, O., Cipolla, R. (2006). Face Recognition from Video Using the Generic Shape-Illumination Manifold. In: Leonardis, A., Bischof, H., Pinz, A. (eds) Computer Vision – ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3954. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11744085_3
Download citation
DOI: https://doi.org/10.1007/11744085_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33838-3
Online ISBN: 978-3-540-33839-0
eBook Packages: Computer ScienceComputer Science (R0)