Abstract
When only a single image of a face is available, can we generate new images of the face across changes in viewpoint or illumination? The approach presented in this paper acquires its knowledge about possible image changes from other faces and transfers this prior knowledge to a novel face image. In previous work we introduced the concept of linear object classes (Vetter and Poggio, 1997; Vetter, 1997): In an image based approach, a flexible image model of faces was used to synthesize new images of a face when only a single 2D image of that face is available.
In this paper we describe a new general flexible face model which is now “learned” from examples of individual 3D-face data (Cyberware-scans). In an analysis-by-synthesis loop the flexible 3D model is matched to the novel face image. Variation of the model parameters, similar to multidimensional morphing, allows for generating new images of the face where viewpoint, illumination or even the expression is changed.
The key problem for generating a flexible face model is the computation of dense correspondence between all given example faces. A new correspondence algorithm is described, which is a generalization of existing algorithms for optic flow computation to 3D-face data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Akimoto, T., Suenaga, Y., and Wallace, R. (1993). Automatic creation of 3D facial models. IEEE Computer Graphics and Applications, 13(3):16–22.
Barron, J., Fleet, D., and Beauchemin, S. (1994). Performance of optical flow techniques. Int. Journal of Computer Vision, pages 43–77.
Benson, P. J. and Perrett, D. (1993). Perception and recognition of photographic quality caricatures: implications for the recognition of natural images. European Journal of Cognitive Psychology, 3:105–135.
Bergen, J., Anandan, P., Hanna, K., and Hingorani, R. (1992). Hierarchical model-based motion estimation. In Proceedings of the European Conference on Computer Vision, pages 237–252, Santa Margherita Ligure, Italy.
Bergen, J. and Hingorani, R. (1990). Hierarchical motion-based frame rate conversion. Technical report, David Sarnoff Research Center Princeton NJ 08540.
Beymer, D. and Poggio, T. (1996). Image representation for visual learning. Science, 272:1905–1909.
Beymer, D., Shashua, A., and Poggio, T. (1993). Example-based image analysis and synthesis. A.L Memo No. 1431, Artificial Intelligence Laboratory, Massachusetts Institute of Technology.
Brennan, S. E. (1985). The caricature generator. Leonardo, 18:170–178.
Burt, P. and Adelson, E. (1983). The Laplacian pyramide as a compact image code. IEEE Transactions on Communications, (31):532–540.
Choi, C, Okazaki, T., Harashima, H., and Takebe, T. (1991). A system of analyzing and synthesizing facial images. In Proc. IEEE Int. Symposium of Circuit and Syatems (ISCAS91), pages 2665–2668.
Duda, R. and Hart, P. (1973). Pattern classification and scene analysis. John Wiley & Sons, New York.
Grenander, U. (1978). Pattern Analysis, Lectures in Pattern Theory. Springer, New York, 1 edition.
Herpers, R., Michaelis, M., Lichtenauer, K. H., and Sommer, G. (1996). Edge and keypoint detection in facial regions. In Proc. International Conference on Automatic Face and Gesture Recognition, pages 22–27, Killington, VT.
Jones, M. and Poggio, T. (1996). Model-based matching by linear combination of prototypes. A.i. memo no., Artificial Intelligence Laboratory, Massachusetts Institute of Technology.
Kosslyn, S. M. (1994). Image and Brain. MIT Press, Cambridge, MA.
Lanitis, A., Taylor, C., Gootes, T., and Ahmad, T. (1995). Automatic interpretation of human faces and hand gestures using flexible models. In M. Bichsel, editor, Proc. International Workshop on Face and Gesture Recognition, pages 98–103, Zurich, Switzerland.
Marr, D. (1982). Vision,. W. H. Freeman, San Fancisco.
Mumford, D. (1996). Pattern theory: A unifying perspective. In Knill, D. and Richards, W., editors, Perception as Bayesian Inference. Cambridge University Press.
O’Toole, A., Deffenbacher, K., Valentin, D., and Abdi, H. (1994). Structural aspects of face recognition and the other-race effect. Memory and Cognition, 22:208–224.
O’Toole, A., Vetter, T., Volz, H., and Salter, E. (1997). Three-dimensional caricatures of human heads: distinctiveness and the perception of facial age. Perception, in press.
Parke, F. (1974). A parametric model of human faces. Doctoral thesis, University of Utah, Salt Lake City.
Poggio, T. and Vetter, T. (1992). Recognition and structure from one 2D model view: observations on prototypes, object classes, and symmetries. A.I. Memo No. 1347, Artificial Intelligence Laboratory, Massachusetts Institute of Technology.
Ullman, S. (1989). Aligning pictorial descriptions: An approach for object recognition. Cognition, 32:193–254.
Vetter, T. (1997). Synthestis of novel views from a single face image. International Journal of Computer Vision, (in press).
Vetter, T., Jones, M. J., and Poggio, T. (1997). A bootstrapping algorithm for learning linear models of object classes. In IEEE Conference on Computer Vision and Pattern Recognition - CVPR’97, Puerto Rico, USA. IEEE Computer Society Press.
Vetter, T. and Poggio, T. (1997). Linear objectclasses and image synthesis from a single example image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):733–742.
Viola, P. (1995). Alignment by maximization of mutual information. A.I. Memo No. 1548, Artificial Intelligence Laboratory, Massachusetts Institute of Technology.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Vetter, T., Blanz, V. (1998). Generalization to Novel Views from a Single Face Image. In: Wechsler, H., Phillips, P.J., Bruce, V., Soulié, F.F., Huang, T.S. (eds) Face Recognition. NATO ASI Series, vol 163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-72201-1_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-72201-1_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-72203-5
Online ISBN: 978-3-642-72201-1
eBook Packages: Springer Book Archive