Advertisement

Face Synthesis

  • Yang Wang
  • Zicheng Liu
  • Baining Guo

Abstract

How to synthesize photorealistic images of human faces has been a fascinating yet difficult problem in computer graphics. Here, the term “face synthesis” refers to synthesis of still images as well as synthesis of facial animations. In this chapter, we focus more on the synthesis of still images and skip most of the aspects that mainly involve the motion over time. We review recent advances on face synthesis including 3D face modeling, face relighting, and facial expression synthesis.

Keywords

Facial Expression Feature Point Face Recognition Face Image Face Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

We thank Ying-Li Tian for carefully reading our manuscripts and providing critical reviews. We also thank Zhengyou Zhang, Alex Acero, and Heung-Yeung Shum for their support.

References

  1. 1.
    Akimoto, T., Suenaga, Y., Wallace, R.S.: Automatic 3d facial models. IEEE Comput. Graph. Appl. 13(5), 16–22 (1993) CrossRefGoogle Scholar
  2. 2.
    Badler, N., Platt, S.: Animating facial expressions. In: Computer Graphics, pp. 245–252. Siggraph, August 1981 Google Scholar
  3. 3.
    Basri, R., Jacobs, D.: Lambertian reflectance and linear subspaces. In: Proc. ICCV’01, pp. 383–390 (2001) Google Scholar
  4. 4.
    Beier, T., Neely, S.: Feature-based image metamorphosis. In: Computer Graphics, pp. 35–42. Siggraph, July 1992 Google Scholar
  5. 5.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: Computer Graphics, Annual Conference Series, pp. 187–194. Siggraph, August 1999 Google Scholar
  6. 6.
    Bregler, C., Covell, M., Slaney, M.: Video rewrite: Driving visual speech with audio. In: Computer Graphics, pp. 353–360. Siggraph, August 1997 Google Scholar
  7. 7.
    Chuang, E., Bregler, C.: Mood swings: expressive speech animation. ACM Trans. Graph. 24(2), 331–347 (2005) CrossRefGoogle Scholar
  8. 8.
    Dariush, B., Kang, S.B., Waters, K.: Spatiotemporal analysis of face profiles: Detection, segmentation, and registration. In: Proc. of the 3rd International Conference on Automatic Face and Gesture Recognition, April 1998, pp. 248–253. IEEE, New York (1998) CrossRefGoogle Scholar
  9. 9.
    Debevec, P.E.: Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: Computer Graphics, Annual Conference Series, pp. 189–198. Siggraph, July 1998 Google Scholar
  10. 10.
    Debevec, P.E., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., Sagar, M.: Acquiring the reflectance field of a human face. In: Computer Graphics, Annual Conference Series, pp. 145–156. Siggraph, July 2000 Google Scholar
  11. 11.
    Deng, Z., Neumann, U.: Data-Driven 3D Facial Animation. Springer, Berlin (2007) CrossRefGoogle Scholar
  12. 12.
    Dimitrijevic, M., Ilic, S., Fua, P.: Accurate face models from uncalibrated and ill-lit video sequences. In: Computer Vision and Pattern Recognition, vol. II, pp. 1034–1041 (2004) Google Scholar
  13. 13.
    Fua, P., Miccio, C.: From regular images to animated heads: A least squares approach. In: Eurographics of Computer Vision, pp. 188–202 (1996) Google Scholar
  14. 14.
    Fua, P., Miccio, C.: Animated heads from ordinary images: A least-squares approach. Comput. Vis. Image Underst. 75(3), 247–259 (1999) CrossRefGoogle Scholar
  15. 15.
    Georghiades, A., Belhumeur, P., Kriegman, D.: Illumination-based image synthesis: Creating novel images of human faces under differing pose and lighting. In: IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, pp. 47–54 (1999) CrossRefGoogle Scholar
  16. 16.
    Guenter, B., Grimm, C., Wood, D., Malvar, H., Pighin, F.: Making faces. In: Computer Graphics, Annual Conference Series, pp. 55–66. Siggraph, July 1998 Google Scholar
  17. 17.
    Horn, B.K.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. A 4(4), 629–642 (1987) MathSciNetCrossRefGoogle Scholar
  18. 18.
  19. 19.
  20. 20.
    Ip, H.H.S., Yin, L.: Constructing a 3d individualized head model from two orthogonal views. Vis. Comput. 12, 254–266 (1996) CrossRefGoogle Scholar
  21. 21.
    Jiang, X., Kong, Y., Huang, J., Zhao, R., Zhang, Y.: Learning from real images to model lighting variations for face images. In: European Conference on Computer Vision (ECCV’2008), vol. IV, pp. 284–297 (2008) Google Scholar
  22. 22.
    Joshi, P., Tien, W.C., Desbrun, M., Pighin, F.: Learning controls for blend shape based realistic facial animation. In: Proc. Symposium on Computer Animation (SCA’03), pp. 187–192, July 2003 Google Scholar
  23. 23.
    Kemelmacher, I., Basri, R.: Molding face shapes by example. In: European Conference on Computer Vision, pp. I:277–288 (2006) Google Scholar
  24. 24.
    Lau, M., Chai, J., Xu, Y.-Q., Shum, H.-Y.: Face poser: Interactive modeling of 3D facial expressions using facial priors. ACM Trans. Graph. 29(1), 1–17 (2009) CrossRefGoogle Scholar
  25. 25.
    Lee, Y., Terzopoulos, D., Waters, K.: Realistic modeling for facial animation. In: Computer Graphics, pp. 55–62. Siggraph, August 1995 Google Scholar
  26. 26.
    Lee, J., Moghaddam, B., Pfister, H., Machiraju, R.: A bilinear illumination model for robust face recognition. In: International Conference on Computer Vision (ICCV’05), vol. II, pp. 1177–1184 (2005) Google Scholar
  27. 27.
    Lei, Z., Bai, Q., He, R., Li, S.: Face shape recovery from a single image using cca mapping between tensor spaces. In: Computer Vision and Pattern Recognition (2008) Google Scholar
  28. 28.
    Li, S.Z., Gu, L.: Real-time multi-view face detection, tracking, pose estimation, alignment, and recognition. In: IEEE Conf. on Computer Vision and Pattern Recognition Demo Summary (2001) Google Scholar
  29. 29.
    Li, H., Weise, T., Pauly, M.: Example-based facial rigging. In: ACM Transactions on Graphics. Siggraph, July 2010 Google Scholar
  30. 30.
    Litwinowicz, P., Williams, L.: Animating images with drawings. In: Computer Graphics, pp. 235–242. Siggraph, August 1994 Google Scholar
  31. 31.
    Liu, Z.: A fully automatic system to model faces from a single image. Microsoft Research Technical Report: MST-TR-2003-55 (2003) Google Scholar
  32. 32.
    Liu, Z., Zhang, Z.: Robust head motion computation by taking advantage of physical properties. In: IEEE Workshop on Human Motion (HUMO), pp. 73–77 (2000) Google Scholar
  33. 33.
    Liu, Z., Shan, Y., Zhang, Z.: Expressive expression mapping with ratio images. In: Computer Graphics, Annual Conference Series, pp. 271–276. Siggraph, August 2001 Google Scholar
  34. 34.
    Liu, Z., Zhang, Z., Jacobs, C., Cohen, M.: Rapid modeling of animated faces from video. J. Vis. Comput. Animat. 12(4), 227–240 (2001) MATHCrossRefGoogle Scholar
  35. 35.
    Ma, W.-C., Jones, A., Chiang, J.-Y., Hawkins, T., Frederiksen, S., Peers, P., Vukovic, M., Ouhyoung, M., Debevec, P.: Facial performance synthesis using deformation-driven polynomial displacement maps. In: SIGGRAPH Asia ’08, pp. 1–10. ACM, New York (2008) CrossRefGoogle Scholar
  36. 36.
    Marschner, S.R., Greenberg, D.P.: Inverse lighting for photography. In: IST/SID Fifth Colort Imaging Conference, November 1997 Google Scholar
  37. 37.
    Marschner, S.R., Westin, S., Lafortune, E., Torance, K., Greenberg, D.: Image-based brdf measurement including human skin. In: Rendering Techniques (1999) Google Scholar
  38. 38.
    Marschner, S.R., Guenter, B., Raghupathy, S.: Modeling and rendering for realistic facial animation. In: Rendering Techniques, pp. 231–242. Springer, New York (2000) Google Scholar
  39. 39.
    Noh, J.J., Neumann, U.: Expression cloning. In: Computer Graphics, Annual Conference Series, pp. 277–288. Siggraph, August 2001 Google Scholar
  40. 40.
    Parke, F.I.: Computer generated animation of faces. In: ACM National Conference, November 1972 Google Scholar
  41. 41.
    Patel, A., Smith, W.: 3d morphable face models revisited. In: Computer Vision and Pattern Recognition (CVPR’09), pp. 1327–1334 (2009) Google Scholar
  42. 42.
    Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: Computer Graphics, Annual Conference Series, pp. 75–84. Siggraph, July 1998 Google Scholar
  43. 43.
    Pyun, H., Kim, Y., Chae, W., Kang, H.W., Shin, S.Y.: An example-based approach for facial expression cloning. In: Proc. Symposium on Computer Animation (SCA’03), pp. 167–176, July 2003 Google Scholar
  44. 44.
    Qing, L., Shan, S., Gao, W.: Face recognition with harmonic de-lighting. In: Asian Conference on Computer Vision (ACCV), January 2004 Google Scholar
  45. 45.
    Ramamoorthi, R., Hanrahan, P.: An efficient representation for irradiance environment maps. In: Proc. SIGGRAPH 2001, pp. 497–500, August 2001 CrossRefGoogle Scholar
  46. 46.
    Riklin-Raviv, T., Shashua, A.: The quotient image: Class based re-rendering and recognition with varying illuminations. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 566–571, June 1999 Google Scholar
  47. 47.
    Romdhani, S., Vetter, T.: Estimating 3d shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In: Computer Vision and Pattern Recognition, vol. II, pp. 986–993 (2005) Google Scholar
  48. 48.
    Romdhani, S., Blanz, V., Vetter, T.: Face identification by fitting a 3d morphable model using linear shape and texture error functions. In: European Conference on Computer Vision (ECCV’2002), vol. IV, pp. 3–19 (2002) Google Scholar
  49. 49.
    Shan, Y., Liu, Z., Zhang, Z.: Modle-based bundle adjustment with application to face modeling. In: International Conference on Computer Vision (ICCV’01), vol. II, pp. 644–651 (2001) Google Scholar
  50. 50.
    Shim, H., Luo, J., Chen, T.: A subspace model-based approach to face relighting under unknown lighting and poses. IEEE Trans. Image Process. 17(8), 1331–1341 (2008) MathSciNetCrossRefGoogle Scholar
  51. 51.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression (PIE) database. In: Face and Gesture’02 (2002) Google Scholar
  52. 52.
    Smith, W., Hancock, E.: Recovering facial shape using a statistical model of surface normal direction. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 1914–1930 (2006) CrossRefGoogle Scholar
  53. 53.
    Stoschek, A.: Image-based re-rendering of faces for continuous pose and illumination directions. In: Computer Vision and Pattern Recognition (CVPR’00), pp. 582–587 (2000) Google Scholar
  54. 54.
    Sumner, R.W., Popović, J.: Deformation transfer for triangle meshes. In: Computer Graphics, Annual Conference Series, pp. 399–405. Siggraph (2004) Google Scholar
  55. 55.
    Terzopoulos, D., Waters, K.: Physically-based facial modeling and animation. J. Vis. Comput. Animat. 1(4), 73–80 (1990) CrossRefGoogle Scholar
  56. 56.
    Wang, Y., Huang, X., Lee, C.-S., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., Huang, P.: High resolution acquisition, learning and transfer of dynamic 3-d facial expressions. Comput. Graph. Forum 23, 677–686 (2004) CrossRefGoogle Scholar
  57. 57.
    Wang, Y., Zhang, L., Liu, Z., Hua, G., Wen, Z., Zhang, Z., Samaras, D.: Face re-lighting from a single image under arbitrary unknown lighting conditions. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1968–1984 (2009) CrossRefGoogle Scholar
  58. 58.
    Waters, K.: A muscle model for animating three-dimensional facial expression. Comput. Graph. 22(4), 17–24 (1987) Google Scholar
  59. 59.
    Wen, Z., Liu, Z., Huang, T.S.: Face relighting with radiance environment maps. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. II, pp. 158–165, June 2003 Google Scholar
  60. 60.
    Williams, L.: Performance-driven facial animation. In: Computer Graphics, pp. 235–242. Siggraph, August 1990 Google Scholar
  61. 61.
    Wolberg, G.: Digital Image Warping. IEEE Computer Society Press, Los Alamitos (1990) Google Scholar
  62. 62.
    Yan, S., Li, M., Zhang, H., Cheng, Q.: Ranking prior likelihood distributions for Bayesian shape localization framework. In: International Conference on Computer Vision (ICCV’03) (2003) Google Scholar
  63. 63.
    Yu, Y., Debevec, P.E., Malik, J., Hawkins, T.: Inverse global illumination: Recovering reflectance models of real scenes from photographs. In: Proc. SIGGRAPH 99, pp. 215–224, July 1999 CrossRefGoogle Scholar
  64. 64.
    Zhang, L., Samaras, D.: Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 351–363 (2006) CrossRefGoogle Scholar
  65. 65.
    Zhang, Q., Liu, Z., Guo, B., Shum, H.: Geometry-driven photorealistic facial expression synthesis. In: Proc. Symposium on Computer Animation (SCA’03), pp. 177–186, July 2003 Google Scholar
  66. 66.
    Zhang, Q., Liu, Z., Guo, B., Terzopoulos, D., Shum, H.-Y.: Geometry-driven photorealistic facial expression synthesis. IEEE Trans. Vis. Comput. Graph. 12(1), 48–60 (2006) CrossRefGoogle Scholar
  67. 67.
    Zhou, S.K., Aggarwal, G., Chellappa, R., Jacobs, D.: Appearance characterization of linear Lambertian objects, generalized photometric stereo and illumination-invariant face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 330–245 (2007) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Carnegie Mellon UniversityPittsburghUSA
  2. 2.Microsoft ResearchRedmondUSA
  3. 3.Microsoft Research AsiaBeijingChina

Personalised recommendations