Real-Time Facial Expression Mapping for High Resolution 3D Meshes

  • Mingli Song
  • Zicheng Liu
  • Baining Guo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4035)


We present a learning-based 3D facial expression mapping technique that preserves facial expression details and works in real time for even high resolution meshes. Our approach is inspired by a previously developed technique called deformation transfer [1]. The deformation transfer technique preserves facial expression details but its computational overhead makes it not suitable for real time applications. To accelerate computation, we use a piecewise linear function to represent the mapping from the direct motion (the difference between the expression face and the neutral face) to the motion obtained by the deformation transfer method. This piecewise linear function is learned offline from a small set of training data. The online computation is thus reduced to the evaluation of the piecewise linear functions which is significantly faster. As a result, we are able to perform real time expression mapping for even high resolution 3D meshes.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sumner, R.W., Popović, J.: Deformation Transfer for Triangle Meshes. In: ACM Siggraph, pp. 399–405 (2004)Google Scholar
  2. 2.
    Kähler, K., Haber, J., Seidel, H.-P.: Geometry-based Muscle Modeling for Facial Animation. In: Graphics Interface, pp. 37–46 (2001)Google Scholar
  3. 3.
    Lee, Y., Terzopoulos, D., Waters, K.: Realistic Modelling for Facial Animation. In: ACM Siggraph, pp. 55–62 (1995)Google Scholar
  4. 4.
    Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: high resolution capture for modeling and animation. ACM Transaction on Graphics 23(3), 548–558 (2004)CrossRefGoogle Scholar
  5. 5.
    Parke, F.I., Waters, K.: Computer Facial Animation. AK Peters, Wellesley, Mass (1996)Google Scholar
  6. 6.
    Brennan, S.E.: Caricature Generator, MS Visual Studies, Dept. of Architecture, Massachusettes Inst. Of Technology, Cambridge, Mass. (1982)Google Scholar
  7. 7.
    Litwinowcz, P., Williams, L.: Animating Images with Drawings. In: ACM Siggraph, pp. 409–412 (1994)Google Scholar
  8. 8.
    Williams, L.: Performance-Driven Facial Animation. Computer Graphics, 235–242 (1990)Google Scholar
  9. 9.
    Liu, Z., Shan, Y., Zhang, Z.: Expressive expression mapping with ratio images. In: ACM Siggraph, pp. 271–276 (2001)Google Scholar
  10. 10.
    Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: ACM Siggraph, pp. 75–84 (1998)Google Scholar
  11. 11.
    Pyun, P., Kim, Y., Chae, W.: An Example-based Approach for Facial Expression Cloning. In: ACM Siggraph/EuroGraphics Symposium on Computer Animation, pp. 277–288 (2003)Google Scholar
  12. 12.
    Noh, J.-Y., Neumann, U.: Expression Cloning. In: ACM Siggraph, pp. 277–288 (2001)Google Scholar
  13. 13.
    Zhang, Q., Liu, Z., Guo, B., Shum, H.: Geometry-Driven Photorealistic Facial Expression Synthesis. In: Eurography/Siggraph Symposium on Computer Animation, pp. 177–187 (2003)Google Scholar
  14. 14.
    Zhang, Q., Liu, Z., Guo, B., Terzopoulos, D., Shum, H.: Geometry-Driven Photorealistic Facial Expression Synthesis. IEEE Transaction on Visualization and Computer Graphics 12(1), 48–60 (2006)CrossRefGoogle Scholar
  15. 15.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley Interscience, Chichester (2000)Google Scholar
  16. 16.
    Krüger, J., Westermann, R.: Linear Algebra Operators for GPU Implementation of Numerical Algorithms. In: ACM Siggraph, pp. 908–916 (2003)Google Scholar
  17. 17.
    Challis, J.H.: A procedure for determining rigid body transformation parameters. Journal of Biomechanics 28(6), 733–737 (1995)CrossRefGoogle Scholar
  18. 18.
    Davis, T.A.: Umfpack version 4.1 user guide. Technical report, University of Florida.TR-03-008 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Mingli Song
    • 1
  • Zicheng Liu
    • 2
  • Baining Guo
    • 3
  1. 1.College of Computer ScienceZhejiang University 
  2. 2.Microsoft ResearchRedmond
  3. 3.Microsoft Research, Asia 

Personalised recommendations