Generalization to Novel Views from a Single Face Image

  • Thomas Vetter
  • Volker Blanz
Part of the NATO ASI Series book series (volume 163)


When only a single image of a face is available, can we generate new images of the face across changes in viewpoint or illumination? The approach presented in this paper acquires its knowledge about possible image changes from other faces and transfers this prior knowledge to a novel face image. In previous work we introduced the concept of linear object classes (Vetter and Poggio, 1997; Vetter, 1997): In an image based approach, a flexible image model of faces was used to synthesize new images of a face when only a single 2D image of that face is available.

In this paper we describe a new general flexible face model which is now “learned” from examples of individual 3D-face data (Cyberware-scans). In an analysis-by-synthesis loop the flexible 3D model is matched to the novel face image. Variation of the model parameters, similar to multidimensional morphing, allows for generating new images of the face where viewpoint, illumination or even the expression is changed.

The key problem for generating a flexible face model is the computation of dense correspondence between all given example faces. A new correspondence algorithm is described, which is a generalization of existing algorithms for optic flow computation to 3D-face data.


Optical Flow Face Image View Synthesis Shape Vector Optic Flow Computation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Akimoto, T., Suenaga, Y., and Wallace, R. (1993). Automatic creation of 3D facial models. IEEE Computer Graphics and Applications, 13(3):16–22.CrossRefGoogle Scholar
  2. Barron, J., Fleet, D., and Beauchemin, S. (1994). Performance of optical flow techniques. Int. Journal of Computer Vision, pages 43–77.Google Scholar
  3. Benson, P. J. and Perrett, D. (1993). Perception and recognition of photographic quality caricatures: implications for the recognition of natural images. European Journal of Cognitive Psychology, 3:105–135.CrossRefGoogle Scholar
  4. Bergen, J., Anandan, P., Hanna, K., and Hingorani, R. (1992). Hierarchical model-based motion estimation. In Proceedings of the European Conference on Computer Vision, pages 237–252, Santa Margherita Ligure, Italy.Google Scholar
  5. Bergen, J. and Hingorani, R. (1990). Hierarchical motion-based frame rate conversion. Technical report, David Sarnoff Research Center Princeton NJ 08540.Google Scholar
  6. Beymer, D. and Poggio, T. (1996). Image representation for visual learning. Science, 272:1905–1909.CrossRefGoogle Scholar
  7. Beymer, D., Shashua, A., and Poggio, T. (1993). Example-based image analysis and synthesis. A.L Memo No. 1431, Artificial Intelligence Laboratory, Massachusetts Institute of Technology.Google Scholar
  8. Brennan, S. E. (1985). The caricature generator. Leonardo, 18:170–178.CrossRefGoogle Scholar
  9. Burt, P. and Adelson, E. (1983). The Laplacian pyramide as a compact image code. IEEE Transactions on Communications, (31):532–540.CrossRefGoogle Scholar
  10. Choi, C, Okazaki, T., Harashima, H., and Takebe, T. (1991). A system of analyzing and synthesizing facial images. In Proc. IEEE Int. Symposium of Circuit and Syatems (ISCAS91), pages 2665–2668.Google Scholar
  11. Duda, R. and Hart, P. (1973). Pattern classification and scene analysis. John Wiley & Sons, New York.MATHGoogle Scholar
  12. Grenander, U. (1978). Pattern Analysis, Lectures in Pattern Theory. Springer, New York, 1 edition.Google Scholar
  13. Herpers, R., Michaelis, M., Lichtenauer, K. H., and Sommer, G. (1996). Edge and keypoint detection in facial regions. In Proc. International Conference on Automatic Face and Gesture Recognition, pages 22–27, Killington, VT.Google Scholar
  14. Jones, M. and Poggio, T. (1996). Model-based matching by linear combination of prototypes. A.i. memo no., Artificial Intelligence Laboratory, Massachusetts Institute of Technology.Google Scholar
  15. Kosslyn, S. M. (1994). Image and Brain. MIT Press, Cambridge, MA.Google Scholar
  16. Lanitis, A., Taylor, C., Gootes, T., and Ahmad, T. (1995). Automatic interpretation of human faces and hand gestures using flexible models. In M. Bichsel, editor, Proc. International Workshop on Face and Gesture Recognition, pages 98–103, Zurich, Switzerland.Google Scholar
  17. Marr, D. (1982). Vision,. W. H. Freeman, San Fancisco.Google Scholar
  18. Mumford, D. (1996). Pattern theory: A unifying perspective. In Knill, D. and Richards, W., editors, Perception as Bayesian Inference. Cambridge University Press.Google Scholar
  19. O’Toole, A., Deffenbacher, K., Valentin, D., and Abdi, H. (1994). Structural aspects of face recognition and the other-race effect. Memory and Cognition, 22:208–224.CrossRefGoogle Scholar
  20. O’Toole, A., Vetter, T., Volz, H., and Salter, E. (1997). Three-dimensional caricatures of human heads: distinctiveness and the perception of facial age. Perception, in press.Google Scholar
  21. Parke, F. (1974). A parametric model of human faces. Doctoral thesis, University of Utah, Salt Lake City.Google Scholar
  22. Poggio, T. and Vetter, T. (1992). Recognition and structure from one 2D model view: observations on prototypes, object classes, and symmetries. A.I. Memo No. 1347, Artificial Intelligence Laboratory, Massachusetts Institute of Technology.Google Scholar
  23. Ullman, S. (1989). Aligning pictorial descriptions: An approach for object recognition. Cognition, 32:193–254.CrossRefGoogle Scholar
  24. Vetter, T. (1997). Synthestis of novel views from a single face image. International Journal of Computer Vision, (in press).Google Scholar
  25. Vetter, T., Jones, M. J., and Poggio, T. (1997). A bootstrapping algorithm for learning linear models of object classes. In IEEE Conference on Computer Vision and Pattern Recognition - CVPR’97, Puerto Rico, USA. IEEE Computer Society Press.Google Scholar
  26. Vetter, T. and Poggio, T. (1997). Linear objectclasses and image synthesis from a single example image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):733–742.CrossRefGoogle Scholar
  27. Viola, P. (1995). Alignment by maximization of mutual information. A.I. Memo No. 1548, Artificial Intelligence Laboratory, Massachusetts Institute of Technology.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Thomas Vetter
    • 1
  • Volker Blanz
    • 1
  1. 1.Max-Planck-Institut für biologische KybernetikTübingenGermany

Personalised recommendations