Abstract
Statistical shape and texture appearance models are powerful image representations, but previously had been restricted to 2D or 3D shapes with smooth surfaces and lambertian reflectance. In this paper we present a novel 3D appearance model using image-based rendering techniques, which can represent complex lighting conditions, structures, and surfaces. We construct a light field manifold capturing the multi-view appearance of an object class and extend the direct search algorithm of Cootes and Taylor to match new light fields or 2D images of an object to a point on this manifold. When matching to a 2D image the reconstructed light field can be used to render unseen views of the object. Our technique differs from previous view-based active appearance models in that model coefficients between views are explicitly linked, and that we do not model any pose variation within the shape model at a single view. It overcomes the limitations of polygonal based appearance models and uses light fields that are acquired in real-time.
Chapter PDF
References
Adelson, E.H., Bergen, J.: The Plenoptic Function and the Elements of Early Vision. In: Computation Models of Visual Processing. MIT Press, Cambridge (1991)
Beymer, D., Poggio, T.: Face recognition from one example view, Tech. Rep. AIM-1536 (September 1995)
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: SIGGRAPH, Los Angeles, pp. 187–194 (1999)
Brand, M.: Morphable 3D models from video. In: CVPR (May 2001)
Buehler, C., Bosse, M., McMillan, L., Gortler, S.J., Cohen, M.F.: Unstructured lumigraph rendering. In: SIGGRAPH, pp. 425–432 (2001)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998)
Cootes, T.F., Wheeler, G.V., Walker, K.N., Taylor, C.J.: View-based active appearance models. Image and Vision Computing 20, 657–664 (2002)
Edwards, G., Taylor, C., Cootes, T.: Interpreting face images using active appearance models. In: 3rd International Conference on Automatic Face and Gesture Recognition, pp. 300–305 (1998)
Fua, P., Miccio, C.: From regular images to animated heads: a least squares approach. In: Burkhardt, H.-J., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1406, pp. 188–202. Springer, Heidelberg (1999)
Goodall, C.: Procustes methods in the statistical analysis of shape. Journal of the Royal Statistical Society 53(2), 285–339 (1991)
Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The lumigraph. Computer Graphics 30(Annual Conference Series), 43–54 (1996)
Gross, R., Matthews, I., Baker, S.: Appearance-based face recognition and light fields. IEEE PAMI 26(4) (April 2004)
Jones, M.J., Poggio, T.: Multidimensional morphable models. In: ICCV, pp. 683–688 (1998)
Levoy, M., Hanrahan, P.: Light field rendering. Computer Graphics 30, 31–42 (1996)
Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)
Moghaddam, B., Pentland, A.: Probabilistic visual learning for object recognition. IEEE PAMI 19(7), 696–710 (1997)
Murase, H., Nayar, S.: Visual learning and recognition of 3-d objects from appearance. IJCV 14(1), 5–24 (1995)
Pighin, F.H., Szeliski, R., Salesin, D.: Resynthesizing facial animation through 3d model-based tracking. In: ICCV, pp. 143–150 (1999)
Romdhani, S., Gong, S., Psarrou, A.: A multi-view nonlinear active shape model using kernel pca. In: British Machine Vision Conference, pp. 483–492 (1999)
Stegmann, M.B.: Analysis and segmentation of face images using point annotations and linear subspace techniques. Technical University of Denmark, DTU, Tech. Rep. (August 2002)
Turk, M., Pentland, A.: Eigen faces for recognition. Journal of Cognitive Neuroscience 3(1) (1991)
Wilburn, B., Smulski, M., Lee, H.-H.K., Horowitz, M.: The light field video camera. In: Proceedings of Media Processors 2002, SPIE Electronic Imaging (2002)
Yang, J.C., Everett, M., Buehler, C., McMillan, L.: A real-time distributed light field camera. In: Eurographics Workshop on Rendering, pp. 1–10 (2002)
Zhang, Z., Wang, L., Guo, B., Shum, H.-Y.: Feature-based light field morphing. In: Conference on Computer graphics and interactive techniques, pp. 457–464. ACM Press, New York (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Christoudias, C.M., Morency, LP., Darrell, T. (2004). Light Field Appearance Manifolds. In: Pajdla, T., Matas, J. (eds) Computer Vision - ECCV 2004. ECCV 2004. Lecture Notes in Computer Science, vol 3024. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24673-2_39
Download citation
DOI: https://doi.org/10.1007/978-3-540-24673-2_39
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21981-1
Online ISBN: 978-3-540-24673-2
eBook Packages: Springer Book Archive