3D model-based video coding: Computer vision meets computer graphics

  • Thomas S. Huang
  • Li-an Tang
Invited Talk
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1024)


This paper describes the model-based video coding scheme which combines computer vision and computer graphics techniques. Human face modeling system is used as an example. In this system, a generic 3D face model is used for both facial motion analysis and expression synthesis. The motion trajectories of the feature points are derived using computer vision techniques and computer graphics techniques are used to reconstruct the original motions using the 3D face model and the motion parameters. The potential application of this system in model-based video compression is also discussed in this paper. Some preliminary experimental results show promising future of this approach.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    H. Agawa, G. Xu, etc. “Image Analysis for Face Modelling and Facial Image Reconstruction,” SPIE: Visual Communications and image Processing, Vol. 1360, 1990, pp. 1184–1197.Google Scholar
  2. 2.
    J. K. Aggarwal and N. Nandhakumar, “On the computation of motion from sequences of images — A review,” in Proceedings IEEE, August, 1988, pp. 917–935.Google Scholar
  3. 3.
    K. Aizawa and H. Harashima, “Model-based Analysis Synthesis Image Coding System for a Person's Face,” Signal Processing: Image Communication, Vol. 1, No. 2, 1989, pp. 139–152.Google Scholar
  4. 4.
    K. Aizawa and T. S. Huang, “Model-based image Coding: Advanced Video Coding Techniques for Very Low Bit-rate Applications,” Proceedings of the IEEE, Vol. 83, No. 2, February 1995, pp. 259–271.Google Scholar
  5. 5.
    P. Ekman and W. V. Friesen, Facial Action Coding System, Consulting Psychologists Press, Inc., Palo Alto, California, 1978.Google Scholar
  6. 6.
    P. Ekman, T.S. Huang, T.J. Sejnowski, and J.C. Hager(Eds), “Final Report to NSF of the Planning Workshop on Facial Expression Understanding,” Human Interaction Laboratory, University of California, San Francisco, March, 1993.Google Scholar
  7. 7.
    R. Forchheimer and O. Fahlander, “Low Bit-rate Coding Through Animation,” in Proceedings of Picture Coding Symposium, March 1983, pp. 113–114.Google Scholar
  8. 8.
    M. Kaneko, A. Koike and Y. Hatori, “Coding of Facial Image Sequence Based on a 3D Model of the Head and Motion Detection,” Journal of Visual Communication and Image Representation, Vol. 2, No. 1, March 1991, pp. 39–54.Google Scholar
  9. 9.
    M. Kass, A. Witkin and D. Terzopoulos, “Snakes: Active Contour models,” in Proceedings of International Conference on Computer Vision, 1987, pp. 259–269.Google Scholar
  10. 10.
    R. Koch, “Dynamic 3-D Scene Analysis through Synthesis Feedback Control,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 15, No. 6, June 1993, pp. 556–568.Google Scholar
  11. 11.
    H. Li, P. Roivainen and R. Forchheimer, “3-D Motion Estimation in Model-Based Facial Image Coding,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 15, No. 6, June 1993, pp. 545–555.Google Scholar
  12. 12.
    K. Mase, “Recognition of Facial Expression from Optical Flow,” IEICE Transactions, Vol. E74, No. 10, October 1991, pp. 3474–3482.Google Scholar
  13. 13.
    F. I. Parke, “Parameterized Models for Facial Animation,” IEEE Computer Graphics and Applications, Vol. 12, November 1982, pp. 61–68.Google Scholar
  14. 14.
    S. M. Platt and N. I. Badler, “Animating Facial Expressions,” Computer Graphics, Vol. 15, No. 3, 1981, pp. 245–252.Google Scholar
  15. 15.
    M. Rydfalk, “CANDIDE: A parameterized Face,” Technical Report LiTH-ISY-I-0866, Department of Electrical Engineering, Linköping University, Sweden, October 1987.Google Scholar
  16. 16.
    L. Tang, L. S. Chen, Y. Kong, T. S. Huang and C. R. Lansing, “Performance Evaluation of a Facial Feature Tracking Algorithm,” in Proceedings of NSF/ARPA Workshop: Performance vs. Methodology in Computer Vision, Seattle, Washington, June 1994, pp. 218–226.Google Scholar
  17. 17.
    L. Tang and T. S. Huang, “Analysis-based Facial Expression Synthesis,” in Proceedings of IEEE International Conference on Image Processing, Vol. III, Austin, Texas, November 1994, pp. 98–102.Google Scholar
  18. 18.
    D. Terzopoulos and K. Waters, “Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 15, No. 6, June 1993, pp. 569–579.Google Scholar
  19. 19.
    F. I. Parke, “Techniques for Facial Animation,” in New Trends in Animation and Visualization, Eds. N. M. Thalmann and D. Thalmann, John Wiley & Sons Inc., 1991, pp. 229–241.Google Scholar
  20. 20.
    K. Waters, “A Muscle Model for Animating Three-Dimensional Facial Expression,” Computer Graphics, Vol. 21, No. 4, July 1987, pp. 17–24.Google Scholar
  21. 21.
    W. J. Welsh, “Model-based Coding of Images,” Ph.D. dissertation, British Telecom Research Laboratories, January 1991.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Thomas S. Huang
    • 1
  • Li-an Tang
    • 1
  1. 1.Beckman Institute and Coordinated Science LaboratoryUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations