Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Visual Computing

ISVC 2006: Advances in Visual Computing pp 334–343Cite as

Investigating the Dynamics of Facial Expression

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 4292))

Abstract

This paper is concerned with capturing the dynamics of facial expression. The dynamics of facial expression can be described as the intensity and timing of a facial expression and its formation. To achieve this we developed a technique that can accurately classify and differentiate between subtle and similar expressions, involving the lower face. This is achieved by using Local Linear Embedding (LLE) to reduce the dimensionality of the dataset and applying Support Vector Machines (SVMs) to classify expressions. We then extended this technique to estimate the dynamics of facial expression formation in terms of intensity and timing.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Darwin, C., Ekman, P.: The expression of the emotions in man and animal. The University of Chicago Press, Chicago (1998); 1st edition in 1872, 2nd edition in 1889, 3rd edition with additional commentry by Ekman, P. (1998)

    Google Scholar 

  2. Cohn, J., Kanade, T., Moriyama, T., Ambadar, Z., Xiao, J., Gao, J., Imamura, H.: A comparative study of alternative facs coding algorithms. Technical Report CMU-RI-TR-02-06, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA (2001)

    Google Scholar 

  3. Ekman, P., Friesen, W., Hager, J.: Facial action coding system. Consulting Psychologists Press, Palo Alto (1978)

    Google Scholar 

  4. Cohn, J.F., Schmidt, K., Gross, R., Ekman, P.: Individual differences in facial expression: Stability over time, relation to self-reported emotion, and ability to inform person identification. In: Proceedings of Intel. Conf. On Multimedia and Expo., 2001 (2002)

    Google Scholar 

  5. Ambadar, Z., Schooler, J., Cohn, J.: Deciphering the enigmatic face: The importance of facial dynamics to interpreting subtle facial expressions. Psychological Science (2005)

    Google Scholar 

  6. Bartlett, M.S., Movellan, J., Littlewort, G., Braathen, B., Frank, M.G., Sejnowski, T.J.: Towards automatic recognition of spontaneous facial actions. In: Ekman, P. (ed.) What the face reveals. Oxford University Press, Oxford (2003)

    Google Scholar 

  7. Cohen, I., Sebe, N., Huang, L.C.A.G.: Facial expression recognition from video sequences: Temporal and static modeling. Computer Vision and Image Understanding, 91(1-2) (2003)

    Google Scholar 

  8. Ekman, P.: Facial expression of emotion: New findings, new questions (1992)

    Google Scholar 

  9. Hadid, A., Pietikdinen, M.: An experimental investigation about the integration of facial dynamics in video-based face recognition. ELCVIA 5, 1–13 (2005)

    Google Scholar 

  10. Littlewort, G., Bartlett, M.S., Fasel, I., Susskind, J., Movellan, J.: Dynamics of facial expression extracted automatically from video. Computer vision and Image understanding (2006)

    Google Scholar 

  11. Ekman, P., Friesen, W., Hager, J.: Facial Action Coding System Manual (2002)

    Google Scholar 

  12. Ghent, J., McDonald, J., Harper, J.: Statistical model for expression generation using the facial action coding system. Technical Report NUIM-CS-TR-2003-01, NUI Maynooth (2003)

    Google Scholar 

  13. Saul, L.K., Roweis, S.T.: An introduction to locally linear embedding (2001), http://www.cs.toronto.edu/roweis/lle/publications.html

  14. Schmidt, K., Cohn, J.: Dynamics of facial expression: Normative characteristics and individual differences. In: IEEE International Conference on Multimedia and Expo (ICME 2001), pp. 728–731 (2001)

    Google Scholar 

  15. Cohn, J.: Automated analysis of the configuration and timing of facial expression. In: Ekman, P., Rosenberg, E. (eds.) Afterword of What the face reveals (2nd edn.): Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS) (2005)

    Google Scholar 

  16. Pantic, M., Patras, I.: Dynamics of facial expression: Recognition of facial actions and their temporal segments from face profile image sequences. SMC-B 36, 433–449 (2006)

    Google Scholar 

  17. Zhang, Y., Ji, Q.: Facial expression understanding in image sequences using dynamic and active visual information fusion. In: ICCV, pp. 113–118 (2003)

    Google Scholar 

  18. Donato, G., Bartlett, M., Hager, J., Ekman, P., Sejnowski, T.: Classifying facial actions. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(10), 974–989 (1999)

    Article  Google Scholar 

  19. Bartlett, M.S., Littlewort, G., Frank, M., lainscsek, C., Fasel, I., Movellan, J.: Recognising facial expression: Machine learning and application to spontaneous behaviour. In: IEEE International conference on computer vision and pattern recognition (2005)

    Google Scholar 

  20. Bartlett, M.S., Littlewort, G., Lainscsek, C., Fasel, I., Movellan, J.: Machine learning methods for fully automatic recognition of facial expressions and facial actions. In: IEEE International conference on systems, man and cybernetics, pp. 592–597 (2004)

    Google Scholar 

  21. Bartlett, M.S., Braathen, B., Movellan, J.: Automatic analysis of spontaneous facial behavior: A final report. MPLAB-TR-2001-06, Institute for Neural Computation, University of California, San Diego (2001)

    Google Scholar 

  22. Bartlett, M.S., Littlewort, G., Fasel, I., Movellan, J.: Real-time face detection and facial expression recognition: Development and applications to human computer interaction (2003)

    Google Scholar 

  23. Bartlett, M.S., Littlewort, G., Braathen, B., Sejnowski, T., Movellan, J.: A prototype for automatic recognition of spontaneous facial actions (2003)

    Google Scholar 

  24. Bartlet, M., Littlewort, G., Lainscsek, C., Fasel, I., Frank, M., Movellan, J.: Fully automatic facial action recognition in spontaneous behavior bartlett (2006)

    Google Scholar 

  25. Goneid, A., el Kaliouby, R.: Facial feature analysis of spontaneous facial expression. In: Proceedings of the 10th International AI Applications Conference (2002)

    Google Scholar 

  26. Fasel, B., Luettin, J.: Automatic facial expression analysis: A survey. Pattern Recognition 36(1), 259–275 (2003)

    Article  MATH  Google Scholar 

  27. Saul, L.K., Roweis, S.T.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. Journal of Machine Learning Research 4 (2003)

    Google Scholar 

  28. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by local linear embedding. Science 290, 2323–2326 (2000)

    Article  Google Scholar 

  29. Chang, Y., Yeung, D.Y.: Robust local linear embedding. Technical report, Department of Computer Science, Hong Kong University of Science and Technology, HKUST-CS05-12 (2005)

    Google Scholar 

  30. Hadid, A., PietikAanien, M.: Efficient locally linear embeddings of imperfect manifolds. In: Proceedings of the Third International Conference on Machine Learning and Data Mining in Pattern Recognition, Leipzig, Germany, pp. 188–201 (2003)

    Google Scholar 

  31. Ridder, D.D., Kouropteva, O., Okun, O., PietikAanien, M., Duin, R.P.W.: Supervised locally linear embedding. In: ICANN 2003, pp. 333–341 (2003)

    Google Scholar 

  32. Kouropteva, O., Okun, O., Hadid, A., Soriano, M., Marcos, S., PietikAainen, M.: Beyond locally linear embedding algorithm. Technical report, Department of Electrical and Information Engerring, University of Oulu, Oulu, Finland, MVG-01-2002 (2002)

    Google Scholar 

  33. Ridder, D.D., Duin, R.P.W.: Locally linear embedding for classification. Technical report, Pattern Recognition Group, Department of Imaging Science and Technology, Delft University of Technology, Delft, The Netherlands, PH-2002-01 (2002)

    Google Scholar 

  34. Campbell, C.: kernel methods: A survey of current techniques. Neurocomputing 48, 63–84 (2002)

    Article  MATH  Google Scholar 

  35. Rogers, S., Williams, R.D., Campbell, C.: Class prediction with Microarray Datasets. In: BioInformatics with computational intelligence paradigms. Springer, Heidelberg (2004)

    Google Scholar 

  36. Rogers, S.: Machine Learning Techniques for Microarray Analysis. Faculty of engineering mathematics, University of Bristol (2004)

    Google Scholar 

  37. ScholKopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)

    Google Scholar 

  38. Ghent, J.: A Computational Model of Facial Expression. PhD thesis, National University of Ireland Maynooth, Co., kildare, Ireland (2005)

    Google Scholar 

  39. Cohn, J., Kanade: Cohn-kanade au-coded facial expression database. Technical report, Pittsburgh University (1999)

    Google Scholar 

  40. Gower, J.C.: Generalised procrustes analysis. Psychometrika 40, 33–50 (1975)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National University of Ireland, Maynooth

    Jane Reilly, John Ghent & John McDonald

Authors
  1. Jane Reilly
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Ghent
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John McDonald
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, University of Nevada, Reno, USA

    George Bebis

  2. NASA Ames Research Center, Moffett Field, CA, USA

    Richard Boyle

  3. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    Bahram Parvin

  4. Desert Research Institute, Reno, NV, USA

    Darko Koracin

  5. Digital Image Research Center, Kingston University, London, UK

    Paolo Remagnino

  6. Intel, 95052, Santa Clara, CA, USA

    Ara Nefian

  7. University of California, 430, Computer Science Building, 92697-3425, Irvine, CA, USA

    Gopi Meenakshisundaram

  8. Institute for Data Analysis and Visualization, P.O. Box

    Valerio Pascucci

  9. Department of Computer Science and Engineering, Czech Technical University in Prague, Czech

    Jiri Zara

  10. Rockwell Scientific, 1049 Camino Dos Rios, 91360, Thousand Oaks, CA, USA

    Jose Molineros

  11. Computer Graphics Group, Bielefeld University, D-33501, Bielefeld, Germany

    Holger Theisel

  12. Hewlett Packard Labs, Palo Alto, CA, USA

    Tom Malzbender

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Reilly, J., Ghent, J., McDonald, J. (2006). Investigating the Dynamics of Facial Expression. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2006. Lecture Notes in Computer Science, vol 4292. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11919629_35

Download citation

  • DOI: https://doi.org/10.1007/11919629_35

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-48626-8

  • Online ISBN: 978-3-540-48627-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation