Skip to main content
SpringerLink
Log in

Face Recognition Across Pose and Illumination

  • Chapter
Book cover Handbook of Face Recognition

Abstract

The last decade has seen automatic face recognition evolve from small-scale research systems to a wide range of commercial products. Driven by the FERET face database and evaluation protocol, the currently best commercial systems achieve verification accuracies comparable to those of fingerprint recognizers. In these experiments, only frontal face images taken under controlled lighting conditions were used. As the use of face recognition systems expands toward less restricted environments, the development of algorithms for view and illumination invariant face recognition becomes important. However, the performance of current algorithms degrades significantly when tested across pose and illumination, as documented in a number of evaluations. In this chapter, we review previously proposed algorithms for pose and illumination invariant face recognition. We then describe in detail two successful appearance-based algorithms for face recognition across pose, eigen light-fields, and Bayesian face subregions. We furthermore show how both of these algorithms can be extended toward face recognition across pose and illumination.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Version 2.5.0.17 of the FaceIt recognition engine was used in the experiments.

References

  1. Adelson, E., Bergen, J.: The plenoptic function and elements of early vision. In: Landy, M., Movshon, J.A. (eds.) Computational Models of Visual Processing. MIT Press, Cambridge (1991)

    Google Scholar 

  2. Adini, Y., Moses, Y., Ullman, S.: Face recognition: The problem of compensating for changes in illumination direction. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 721–732 (1997)

    Article  Google Scholar 

  3. Basri, R., Jacobs, D.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003)

    Article  Google Scholar 

  4. Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)

    Article  Google Scholar 

  5. Belhumeur, P., Kriegman, D.: What is the set of images of an object under all possible lighting conditions. Int. J. Comput. Vis. 28(3), 245–260 (1998)

    Article  Google Scholar 

  6. Beymer, D.: Face recognition under varying pose. Technical Report 1461, MIT AI Laboratory, Cambridge, MA (1993)

    Google Scholar 

  7. Beymer, D., Poggio, T.: Face recognition from one example view. A.I. Memo No. 1536, MIT AI Laboratory, Cambridge, MA (1995)

    Google Scholar 

  8. Black, M., Jepson, A.: Eigen-tracking: robust matching and tracking of articulated objects using a view-based representation. Int. J. Comput. Vis. 36(2), 101–130 (1998)

    Google Scholar 

  9. Blackburn, D., Bone, M., Phillips, P.: Facial recognition vendor test 2000: evaluation report (2000)

    Google Scholar 

  10. Blanz, V., Romdhani, S., Vetter, T.: Face identification across different poses and illumination with a 3D morphable model. In: Proceedings of the Fifth International Conference on Face and Gesture Recognition, pp. 202–207 (2002)

    Chapter  Google Scholar 

  11. Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1063–1074 (2003)

    Article  Google Scholar 

  12. Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)

    Article  Google Scholar 

  13. Cootes, T., Wheeler, G., Walker, K., Taylor, C.: View-based active appearance models. Image Vis. Comput. 20, 657–664 (2002)

    Article  Google Scholar 

  14. Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, San Diego (1990)

    MATH  Google Scholar 

  15. Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: Generative models for recognition under variable pose and illumination. In: Proceedings of the Fourth International Conference on Face and Gesture Recognition, pp. 277–284 (2000)

    Chapter  Google Scholar 

  16. Georghiades, A., Kriegman, D., Belhumeur, P.: Illumination cones for recognition under variable lighting: faces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (1998)

    Google Scholar 

  17. Georghiades, A., Kriegman, D., Belhumeur, P.: From few to many: generative models for recognition under variable pose and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 643–660 (2001)

    Article  Google Scholar 

  18. Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The lumigraph. In: Computer Graphics Proceedings, Annual Conference Series (SIGGRAPH), pp. 43–54 (1996)

    Google Scholar 

  19. Graham, D., Allison, N.: Face recognition from unfamiliar views: subspace methods and pose dependency. In: 3rd International Conference on Automatic Face and Gesture Recognition, pp. 348–353 (1998)

    Chapter  Google Scholar 

  20. Gross, R., Brajovic, V.: An image pre-processing algorithm for illumination invariant face recognition. In: 4th International Conference on Audio- and Video Based Biometric Person Authentication (AVBPA), pp. 10–18, June 2003

    Chapter  Google Scholar 

  21. Gross, R., Matthews, I., Baker, S.: Eigen light-fields and face recognition across pose. In: Proceedings of the Fifth International Conference on Face and Gesture Recognition, pp. 1–7 (2002)

    Google Scholar 

  22. Gross, R., Matthews, I., Baker, S.: Fisher light-fields for face recognition across pose and illumination. In: Proceedings of the German Symposium on Pattern Recognition (DAGM), pp. 481–489 (2002)

    Google Scholar 

  23. Gross, R., Matthews, I., Baker, S.: Appearance-based face recognition and light-fields. IEEE Trans. Pattern Anal. Mach. Intell. 26(4), 449–465 (2004)

    Article  Google Scholar 

  24. Gross, R., Shi, J., Cohn, J.: Quo vadis face recognition. In: Third Workshop on Empirical Evaluation Methods in Computer Vision (2001)

    Google Scholar 

  25. Horn, B.: Determining lightness from an image. Comput. Graph. Image Process. 3(1), 277–299 (1974)

    Article  Google Scholar 

  26. Horn, B.: Robot Vision. MIT Press, Cambridge (1986)

    Google Scholar 

  27. Jacobs, D., Belhumeur, P., Basri, R.: Comparing images under variable illumination. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 610–617 (1998)

    Google Scholar 

  28. Kanade, T., Yamada, A.: Multi-subregion based probabilistic approach toward pose-invariant face recognition. In: IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA2003), pp. 954–959 (2003)

    Chapter  Google Scholar 

  29. Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)

    Article  Google Scholar 

  30. Land, E., McCann, J.: Lightness and retinex theory. J. Opt. Soc. Am. 61(1), 1–11 (1971)

    Article  Google Scholar 

  31. Lando, M., Edelman, S.: Generalization from a single view in face recognition. In: International Workshop on Automatic Face-and Gesture-Recognition (1995)

    Google Scholar 

  32. Lanitis, A., Taylor, C., Cootes, T.: Automatic interpretation and coding of face images using flexible models. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 743–756 (1997)

    Article  Google Scholar 

  33. Leonardis, A., Bischof, H.: Robust recognition using eigenimages. Comput. Vis. Image Underst. 78(1), 99–118 (2000)

    Article  Google Scholar 

  34. Levoy, M., Hanrahan, M.: Light field rendering. In: Computer Graphics Proceedings, Annual Conference Series (SIGGRAPH), pp. 31–41 (1996)

    Google Scholar 

  35. Maurer, T., von der Malsburg, C.: Single-view based recognition of faces rotated in depth. In: International Workshop on Automatic Face and Gesture Recogition, pp. 248–253 (1995)

    Google Scholar 

  36. Murase, H., Nayar, S.: Visual learning and recognition of 3-D objects from appearance. Int. J. Comput. Vis. 14, 5–24 (1995)

    Article  Google Scholar 

  37. Pentland, A., Moghaddam, B., Starner, T.: View-based and modular eigenspaces for face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 84–91 (1994)

    Chapter  Google Scholar 

  38. Phillips, P.J., Grother, P., Ross, J., Blackburn, D., Tabassi, E., Bone, M.: Face recognition vendor test 2002: evaluation report, March 2003

    Google Scholar 

  39. Phillips, P.J., Moon, H., Rizvi, S., Rauss, P.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1090–1104 (2000)

    Article  Google Scholar 

  40. Romdhani, S., Blanz, V., Vetter, T.: Face identification by matching a 3D morphable model using linear shape and texture error functions. In: Proceedings of the European Conference on Computer Vision, pp. 3–19 (2002)

    Google Scholar 

  41. Romdhani, S., Gong, S., Psarrou, A.: Multi-view nonlinear active shape model using kernel PCA. In: 10th British Machine Vision Conference, vol. 2, pp. 483–492 (1999)

    Google Scholar 

  42. Romdhani, S., Psarrou, A., Gong, S.: On utilising template and feature-based correspondence in multi-view appearance models. In: 6th European Conference on Computer Vision, vol. 1, pp. 799–813 (2000)

    Google Scholar 

  43. Shashua, A.: Geometry and photometry in 3D visual recognition. PhD thesis, MIT (1994)

    Google Scholar 

  44. Shashua, A., Riklin-Raviv, T.: The Quotient image: class-based re-rendering and recognition with varying illumination conditions. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 129–139 (2001)

    Article  Google Scholar 

  45. Sim, T., Kanade, T.: Combining models and exemplars for face recognition: an illuminating example. In: Workshop on Models Versus Exemplars in Computer Vision (2001)

    Google Scholar 

  46. Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1615–1618 (2003)

    Article  Google Scholar 

  47. Turk, M., Pentland, A.: Face recognition using eigenfaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (1991)

    Google Scholar 

  48. Vetter, T., Poggio, T.: Linear object classes and image synthesis from a single example image. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 733–741 (1997)

    Article  Google Scholar 

  49. Wiskott, L., Fellous, J., Kruger, N., von der Malsburg, C.: Face recognition by elastic bunch graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 775–779 (1997)

    Article  Google Scholar 

  50. Zhao, W., Chellappa, R.: Robust face recognition using symmetric shape-from-shading. Technical report, Center for Automation Research, University of Maryland (1999)

    Google Scholar 

  51. Zhao, W., Krishnaswamy, A., Chellappa, R., Swets, D., Weng, J.: Discriminant analysis of principal components for face recognition. In: Wechsler, H., Phillips, P.J., Bruce, V., Huang, T. (eds.) Face Recognition: From Theory to Applications. Springer, New York (1998)

    Google Scholar 

Download references

Acknowledgements

The research described here was supported by U.S. Office of Naval Research contract N00014-00-1-0915 and in part by U.S. Department of Defense contract N41756-03-C4024. Portions of the research in this paper used the FERET database of facial images collected under the FERET program.

Author information

Authors and Affiliations

  1. Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Ralph Gross, Simon Baker, Iain Matthews & Takeo Kanade

Authors
  1. Ralph Gross
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Baker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Iain Matthews
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Takeo Kanade
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ralph Gross .

Editor information

Editors and Affiliations

  1. Institute of Automation, Center Biometrics Research & Security, Chinese Academy of Science, Room 1227, Zhongguancun 95, Beijing Donglu, 100080, China, People's Republic

    Stan Z. Li

  2. Dept. Computer Science & Engineering, Michigan State University, East Lansing, 48824-1226, Michigan, USA

    Anil K. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Gross, R., Baker, S., Matthews, I., Kanade, T. (2011). Face Recognition Across Pose and Illumination. In: Li, S., Jain, A. (eds) Handbook of Face Recognition. Springer, London. https://doi.org/10.1007/978-0-85729-932-1_8

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-932-1_8

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-931-4

  • Online ISBN: 978-0-85729-932-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation