Skip to main content
SpringerLink
Log in

Animating Speech in Games

  • Conference paper
Motion in Games (MIG 2008)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 5277))

Included in the following conference series:

  • 954 Accesses

Abstract

Realism in games is constantly improving with increased computing power available to games and as game players demand more visual realism. Therefore, facial animation and particularly animated speech, is becoming more prevalent in games. We present a survey of facial animation techniques suitable for 3D computer games. We break our discussion into two areas: modeling and animation. To model a face, a method of representing the geometry is combined with a parameterization that is used to specify a new shape for that geometry. Changing the shape over time will create animation, and we discuss methods for animating expressions as well as achieving lip-synchronized speech.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Parke, F.I., Waters, K.: Computer Facial Animation. A K Peters (1996)

    Google Scholar 

  2. Fleming, B., Dobbs, D.: Animating Facial Features and Expressions. Charles River Media, Inc., Rockland (1999)

    Google Scholar 

  3. Osipa, J.: Stop Staring: Facial Modeling and Animation Done Right. SYBEX, Inc., Alameda (2003)

    Google Scholar 

  4. Ekman, P.: Darwin and cross cultural studies of facial expression. In: Ekman, P. (ed.) Darwin and Facial Expression: A Century of Research in Review. Academic Press, New York (1973)

    Google Scholar 

  5. Ekman, P., Friesen, W.V.: Unmasking the Face. Consulting Psychologists Press, Palo Alto (1984)

    Google Scholar 

  6. Ekman, P.: Telling lies: clues to deceit in the marketplace, politics, and marriage. Norton, New York (1985)

    Google Scholar 

  7. Faigin, G.: The Artist’s Complete Guide to Facial Expression. Watson-Guptill Publications, New York (1990)

    Google Scholar 

  8. King, S.A., Parent, R.E.: A 3d parametric tongue model for animated speech. JVCA 12(3), 107–115 (2001)

    MATH  Google Scholar 

  9. King, S.A., Parent, R.E., Olsafsky, B.: A muscle-based 3d parametric lip model for speech. In: Deformable Avatars, pp. 12–23. Kluwer Academic Publishers, Dordrecht (2001)

    Chapter  Google Scholar 

  10. Rydfalk, M.: Candide: A parameterized face. Technical Report LiTH-ISY-I-0866, Linkoping University, Sweden (October 1987)

    Google Scholar 

  11. King, S.A.: A Facial Model and Animation Techniques for Animated Speech. PhD thesis, The Ohio State University, Columbus, OH (June 2001)

    Google Scholar 

  12. Bergeron, P., Lachapelle, P.: Controlling facial expressions and body movements in the computer-generated animated short Tony De Peltrie. In: SIGGRAPH 1985 Advanced Computer Animation seminar notes, 1–19 (July 1985)

    Google Scholar 

  13. Elson, M.: Displacement facial animation techniques. In: SIGGRAPH 1990 Course Notes, Course 26, State of the Art in Facial Animation, pp. 21–42 (1990)

    Google Scholar 

  14. Parke, F.I.: A parametric model for human faces. PhD thesis, University of Utah, Salt Lake City, Utah (December 1974)

    Google Scholar 

  15. Nahas, M., Huitric, H., Saintourens, M.: Animation of a B-spline figure. The Visual Computer 3(5), 272–276 (1988)

    Article  Google Scholar 

  16. Waters, K.: A muscle model for animating three-dimensional facial expression. In: Computer Graphics (SIGGRAPH 1987 Proceedings), Anaheim, California, vol. 21(4), pp. 17–24 (July 1987)

    Google Scholar 

  17. Magnenat-Thalmann, N., Primeau, E., Thalmann, D.: Abstract muscle action procedures for human face animation. Visual Computer 3(5), 290–297 (1988)

    Article  Google Scholar 

  18. Guenter, B.: A computer system for simulating human facial expression. PhD thesis, The Ohio State University (1989)

    Google Scholar 

  19. Reeves, W.T.: Simple and complex facial animation: Case studies. In: SIGGRAPH 1990 Course Notes 26: State of the Art in Facial Animation, Dallas, Texas, pp. 88–106 (August 1990)

    Google Scholar 

  20. Pelachaud, C., Badler, N.I., Steedman, M.: Linguistic issues in facial animation. In: Computer Animation 1991, Tokyo, pp. 15–30. Springer, Heidelberg (1991)

    Google Scholar 

  21. Patel, M., Willis, P.J.: FACES — facial animation, construction and editing system. In: Purgathofer, W. (ed.) Eurographics 1991, pp. 33–45. North-Holland, Amsterdam (1991)

    Google Scholar 

  22. Patel, M.: Making Faces: The Facial Animation, Construction and Editing System. PhD thesis, School of Mathematical Sciences, University of Bath, Bath, UK (December 1991)

    Google Scholar 

  23. Pelachaud, C., Viaud, M.L., Yahia, H.: Rule-structured facial animation system. In: Proceedings of the 13th IJCAI, Chambery, France, pp. 1610–1615 (August 1993)

    Google Scholar 

  24. Essa, I.A.: Analysis, Interpretation and Synthesis of Facial Expressions. PhD thesis, MIT Media Lab (1994)

    Google Scholar 

  25. Saulnier, A., Viaud, M.L., Geldreich, D.: Real-time facial analysis and synthesis chain. In: Proc. Automatic Face and Gesture Recognition, Zurich (June 1995)

    Google Scholar 

  26. Konno, T., Mitani, H., Chiyokura, H., Tanaka, I.: Surgical simulation of facial paralysis. In: Sieburg, H.B., Weghorst, S.J., Morgan, K.S. (eds.) Medicine Meets Virtual Reality: Health Care in the Information Age, pp. 488–497. IOS Press, Amsterdam (1996)

    Google Scholar 

  27. Ekman, P., Friesen, W.: Facial Action Coding System. Consulting Psychologists Press, Inc., Palo Alto (1978)

    Google Scholar 

  28. Patterson, E.C., Litwinowicz, P.C., Greene, N.: Facial animation by spatial mapping. In: Computer Animation 1991, Tokyo. Springer, Heidelberg (1991)

    Google Scholar 

  29. Sturman, D.J.: Computer puppetry. Computer Graphics and Applications 18(1), 38–45 (1998)

    Article  Google Scholar 

  30. Williams, L.: Performance-driven facial animation. In: Computer Graphics (SIGGRAPH 1990 Proceedings), vol. 24, pp. 235–242 (August 1990)

    Google Scholar 

  31. de Graf, B.: Performance’ facial animation. In: SIGGRAPH 1989 Course Notes 22: State of the Art in Facial Animation, pp. 8–17 (July 1989)

    Google Scholar 

  32. Guenter, B., Grimm, C., Wood, D., Malvar, H., Pighin, F.: Making faces. In: SIGGRAPH 1998 (August 1998)

    Google Scholar 

  33. Lorenzo, M.S., James, D., Edge, S.A.K., Maddock, S.: Use and re-use of facial motion capture data. In: Vision, Video, and Graphics 2003, pp. 1–8, July 10-11(2003)

    Google Scholar 

  34. Deng, Z., Chiang, P.Y., Fox, P., Neumann, U.: Animating blendshape faces by cross-mapping motion capture data. In: Proc. of ACM SIGGGRAPH Symposium on Interactive 3D Graphics and Games, pp. 43–48 (2006)

    Google Scholar 

  35. Kshirsagar, S., Magnenat-Thalmann, N.: Visyllable based speech animation. Computer Graphics Forum 22(3), 631 (2003)

    Article  Google Scholar 

  36. Cao, Y., Tien, W.C., Faloutsos, P., Pighin, F.: Expressive speech-driven facial animation. ACM Trans. Graph 24(4), 1283–1302 (2005)

    Article  Google Scholar 

  37. Deng, Z., Neumann, U.: eFASE: expressive facial animation synthesis and editing with phoneme-isomap controls. In: Proceedings of SCA 2006, pp. 251–260. Eurographics Association (2006)

    Google Scholar 

  38. Deng, Z., Chiang, P.Y., Fox, P., Neumann, U.: Animating blendshape faces by cross-mapping motion capture data. In: I3D 2006: Proceedings of the 2006 symposium on Interactive 3D graphics and games, pp. 43–48. ACM, New York (2006)

    Google Scholar 

  39. Sifakis, E., Neverov, I., Fedkiw, R.: Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph 24(3), 417–425 (2005)

    Article  Google Scholar 

  40. Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., Gross, M.: Multi-scale capture of facial geometry and motion. ACM Trans. Graph 26(3), 33 (2007)

    Article  Google Scholar 

  41. Borshukov, G., et al.: Playable universal capture. In: ACM SIGGRAPH 2006 Sketches and Applications Program, Bostom, Ma (August 2006)

    Google Scholar 

  42. Ezzat, T., Poggio, T.: Videorealistic talking faces: A morphing approach. In: Proceedings of the ESCA Workshop on Audiovisual Speech Processing, Rhodes, Greece, pp. 141–144 (September 1997)

    Google Scholar 

  43. de Graf, B.: Performance’ facial animation. In: SIGGRAPH 1990 Course Notes 26: State of the Art in Facial Animation, pp. 9–20 (August 1990)

    Google Scholar 

  44. Pelachaud, C.: Communication and Coarticulation in Facial Animation. PhD thesis, Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, 19104-6389 (1991)

    Google Scholar 

  45. Ezzat, T., Poggio, T.: Miketalk: A talking facial display based on morphing visemes. In: Computer Animation 1998, pp. 96–102 (June 1998)

    Google Scholar 

  46. Gould, R.L.: Making 3-d computer character animation a great future of unlimited possibility or just tedious? In: SIGGRAPH 1989 Course Notes 4: 3-D Character Animation, pp. 31–63 (1989)

    Google Scholar 

  47. Kleiser, J.: A fast, efficient, accurate way to represent the human face. In: SIGGRAPH 1989 Course Notes 22: State of the Art in Facial Animation, pp. 36–40 (July 1989)

    Google Scholar 

  48. Pighin, F., Auslander, J., Lischinski, D., Salesin, D.: Realistic facial animation using image-based 3d morphing. Technical Report UW-CSE-97-01-03, University of Washington, Department of Computer Sceince & Engineering (1997)

    Google Scholar 

  49. Pighin, F., Hecker, J., Lischinski, D., ard Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: SIGGRAPH 1998, pp. 75–84 (1998)

    Google Scholar 

  50. Provine, J.A., Bruton, L.T.: Lip synchronization in 3-d model based coding for video-conferencing. In: Proc. of the IEEE Int. Symposium on Circuits and Systems, Seattle, pp. 453–456 (May 1995)

    Google Scholar 

  51. Waite, C.T.: The facial action control editor, face: A parametric facial expression editor for computer generated animation. Master’s thesis, Massachusetts Institue of Technology (February 1989)

    Google Scholar 

  52. Morishima, S., Aizawa, K., Harashima, H.: A real-time facial action image synthesis system driven by speech and text. SPIE Visual Communications and Image Processing 1360, 1151–1157 (1990)

    Google Scholar 

  53. Waters, K., Levergood, T.M.: DECface: An automatic lip-synchronization algorithm for synthetic faces. Technical Report CRL 93/4, Digital Equipment Corporation Cambridge Research Lab (September 1993)

    Google Scholar 

  54. Platt, S.M., Badler, N.I.: Animating facial expressions. Computer Graphics (Proceedings of SIGGRAPH 1981) 15(3), 245–252 (1981)

    Article  Google Scholar 

  55. Kähler, K., Haber, J., Seidel, H.P.: Geometry-based muscle modeling for facial animation. In: Graphics Interface 2001, 7–9 June 2001, pp. 37–46 (2001)

    Google Scholar 

  56. Koch, R.M., Gross, M.H., Carls, F.R., von Buren, D.F., Fankhauser, G., Parish, Y.I.: Simulating facial surgery using finite element models. In: Proceedings of SIGGRAPH 1996, pp. 421–428. Addison-Wesley, Reading (1996)

    Google Scholar 

  57. Koch, R.M., Gross, M.H., Bosshard, A.A.: Emotion editing using finite elements. Technical Report 281, Computer Science Department, ETH Zrich (1998)

    Google Scholar 

  58. Larrabee Jr., W.F.: A finite element model of skin deformation: Part III - the finite element model. Laryngoscope, 399–419 (April 1986)

    Google Scholar 

  59. Lee, Y., Terzopoulos, D., Waters, K.: Constructing physics-based facial models of individuals. In: Proceedings of Graphics Interface 1993, Toronto, Ontario, Canada, Canadian Information Processing Society, pp. 1–8 (May 1993)

    Google Scholar 

  60. Lee, Y.V.: The construction and animation of functional facial models from cylindrical range/reflectance data. Master’s thesis, University of Toronto (1993)

    Google Scholar 

  61. Pieper, S.: Physically-based animation of facial tissue for surgical simulation. In: SIGGRAPH 1989 Course Notes 22: State of the Art in Facial Animation, pp. 71–124 (1989)

    Google Scholar 

  62. Rosenblum, R.E., Carlson, W.E., Tripp III, E.: Simulating the structure and dynamics of human hair: modelling, rendering and animation. Journal of Visualization and Computer Animation 2(4), 141–148 (1991)

    Article  Google Scholar 

  63. Terzopoulos, D., Waters, K.: Physically-based facial modelling, analysis, and animation. JVCA 1(2), 73–80 (1990)

    Google Scholar 

  64. Viaud, M.L., Yahia, H.: Facial animation with wrinkles. In: Eurographics 1992, Cambridge, United Kingdom (September 1992)

    Google Scholar 

  65. Waters, K., Terzopoulos, D.: Modeling and animating faces using scanned data. The Journal of Visualization and Computer Animation 2(4), 123–128 (1991)

    Article  Google Scholar 

  66. Wu, Y., Kalra, P., Magnenat-Thalmann, N.: Simulation of static and dynamic wrinkles of skin. In: Proceedings Computer Animation 1996, pp. 90–97 (1996)

    Google Scholar 

  67. Platt, S.M.: A Structural Model of the Human Face. PhD thesis, University of Pennsylvania, Philadelphia, Pennsylvania (1985)

    Google Scholar 

  68. Hofer, E.E.: A sculpting based solution for three-dimensional computer character facial animation. Master’s thesis, The Ohio State University (1993)

    Google Scholar 

  69. Cassell, J., Vilhjálmsson, H.H., Bickmore, T.: BEAT: the behavior expression animation toolkit. In: Fiume, E. (ed.) Proceedings of SIGGRAPH 2001, pp. 477–486. ACM Press, New York (2001)

    Google Scholar 

  70. Pearce, A., Wyvill, B.M., Wyvill, G., Hill, D.: Speech and expression: A computer solution to face animation. In: Green, M. (ed.) Proceedings of Graphics Interface 1986, pp. 136–140 (May 1986)

    Google Scholar 

  71. Takashima, Y., Shimazu, H., Tomono, M.: Story driven animation. In: Carroll, J.M., Tanner, P.P. (eds.) Proceedings of Human Factors in Computing Systems and Graphics Interface 1987, pp. 149–153 (April 1987)

    Google Scholar 

  72. Magnenat-Thalmann, N., Thalmann, D. (eds.): Synthetic Actors in Computer-Generated 3D Films. Springer, Heidelberg (1987)

    Google Scholar 

  73. Fuchs, T., Haber, J., Seidel, H.P.: MIMIC – a language for specifying facial animations. In: WSCG SHORT Communication papers proceedings, pp. 71–78 (2004)

    Google Scholar 

  74. Cassell, J., et al.: Animated conversation: Rule–based generation of facial expression gesture and spoken intonation for multiple converstaional agents. In: SIGGRAPH 1994, 24–29 July 1994, pp. 413–420. ACM Press, New York (1994)

    Google Scholar 

  75. King, S.A., Knott, A., McCan, B.: Language-driven nonverbal communication in a bilingual conversational agent. In: CASA 2003, pp. 17–22 (May 7-9, 2003)

    Google Scholar 

  76. Lee, S.P., Badler, J.B., Badler, N.I.: Eyes alive. ACM Transactions on Graphics 21(3), 637–644 (2002)

    Google Scholar 

  77. Kent, R.D., Minifie, F.D.: Coarticulation in recent speech production models. Journal of Phonetics 5, 115–135 (1977)

    Google Scholar 

  78. Wang, A., Emmi, M., Faloutsos, P.: Assembling an expressive facial animation system. In: Sandbox 2007: Proceedings of the 2007 ACM SIGGRAPH symposium on Video games, pp. 21–26. ACM, New York (2007)

    Chapter  Google Scholar 

  79. Löfqvist, A.: Speech as audible gestures. In: Hardcastle, W.J., Marchal, A. (eds.) Speech Production and Speech Modeling, pp. 289–322. Kluwer Academic Publishers, Dordrecht (1990)

    Chapter  Google Scholar 

  80. Cohen, M., Massaro, D.: Modeling coarticulation in synthetic visual speech. In: Models and Techniques in Computer Animation, pp. 139–156. Springer, Heidelberg (1993)

    Chapter  Google Scholar 

  81. Le Goff, B.: Automatic modeling of coarticulation in text-to-visual speech synthesis. In: Proccedings of Eurospeech 1997, Rhodes, Greece, vol. 3, pp. 1667–1670 (September 1997)

    Google Scholar 

  82. Albrecht, I., Haber, J., Seidel, H.P.: Speech synchronization for physics-based facial animation. In: Proceedings of WSCG 2002, 4-8 February 2002, pp. 9–16 (2002)

    Google Scholar 

  83. Kähler, K., Haber, J., Yamauchi, H., Seidel, H.P.: Head shop: Generating animated head models with anatomical structure. In: SCA 2002, 21-22 July 2002, pp. 55–64 (2002)

    Google Scholar 

  84. Albrecht, I., Haber, J., Seidel, H.P.: Automatic generation of non-verbal facial expressions from speech. In: Proceeding of CGI 2002, 3–5 July 2002, pp. 283–293 (2002)

    Google Scholar 

  85. Kim, T.Y., Neumann, U.: Interactive multiresolution hair modeling and editing. ACM Transactions on Graphics 21(3), 620–629 (2002); (Proceedings of SIGGRAPH 2002)

    Google Scholar 

  86. Marschner, S., Jensen, H.W., Cammarano, M., Worley, S., Hanrahan, P.: Light scattering from human hair fibers. ACM Transactions on Graphics 22(3), 780–791 (2003); (Proceedings of SIGGRAPH 2003)

    Article  Google Scholar 

  87. Bando, Y., Chen, B.Y., Nishita, T.: Animating hair with loosely connected particles. Computer Graphics Forum 22(3), 411 (2003)

    Article  Google Scholar 

  88. Koster, M., Haber, J., Seidel, H.P.: Real-time rendering of human hair using programmable graphics hardware. In: Proc of CGI 2004 (June 2004)

    Google Scholar 

  89. Haro, A., Guenter, B., Essa, I.: Real-time, photo-realistic, physically based rendering of fine scale human skin structure. In: Rendering Techniques, pp. 53–62 (2001)

    Google Scholar 

  90. Jensen, H.W., Marschner, S.R., Levoy, M., Hanrahan, P.: A practical model for subsurface light transport. In: Fiume, E. (ed.) Proceedings of SIGGRAPH 2001, August 2001, pp. 511–518. ACM Press, New York (2001)

    Google Scholar 

  91. Marschner, S.R., Guenter, B.K., Raghupathy, S.: Modeling and rendering for realistic facial animation. In: Rendering Techniques 2000, pp. 231–242 (2000)

    Google Scholar 

  92. Weyrich, T., et al.: Analysis of human faces using a measurement-based skin reflectance model. In: SIGGRAPH ’06: ACM SIGGRAPH 2006 Papers, pp. 1013–1024. ACM, New York (2006)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Texas A&M University Corpus Christi, 6300 Ocean Drive, Unit 5824, Corpus Christi, Texas, 78412, USA

    Scott A. King

Authors
  1. Scott A. King
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Games and Virtual Worlds Group, Universiteit Utrecht, PO Box 80.089, 3508TB, Utrecht, The Netherlands

    Arjan Egges

  2. Games and Virtual Worlds Group, Universiteit Utrecht, PO Box 80.089, 3508TB, Utrecht,, The Netherlands

    Arno Kamphuis

  3. Games and Virtual Worlds Gruop, Universiteit Utrecht, PO Box 80.089, 3508TB, Utrecht, The Netherlands

    Mark Overmars

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

King, S.A. (2008). Animating Speech in Games. In: Egges, A., Kamphuis, A., Overmars, M. (eds) Motion in Games. MIG 2008. Lecture Notes in Computer Science, vol 5277. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89220-5_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-89220-5_23

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-89219-9

  • Online ISBN: 978-3-540-89220-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation