Skip to main content

Advertisement

SpringerLink
Log in

Facial Expression Recognition of Various Internal States via Manifold Learning

  • Short Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Emotions are becoming increasingly important in human-centered interaction architectures. Recognition of facial expressions, which are central to human-computer interactions, seems natural and desirable. However, facial expressions include mixed emotions, continuous rather than discrete, which vary from moment to moment. This paper represents a novel method of recognizing facial expressions of various internal states via manifold learning, to achieve the aim of human-centered interaction studies. A critical review of widely used emotion models is described, then, facial expression features of various internal states via the locally linear embedding (LLE) are extracted. The recognition of facial expressions is created with the pleasure-displeasure and arousal-sleep dimensions in a two-dimensional model of emotion. The recognition result of various internal state expressions that mapped to the embedding space via the LLE algorithm can effectively represent the structural nature of the two-dimensional model of emotion. Therefore our research has established that the relationship between facial expressions of various internal states can be elaborated in the two-dimensional model of emotion, via the locally linear embedding algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Nelson C A. The recognition of facial expressions in the first two years of life: Mechanisms of development. Child Development, 1987, 58(4): 889–909.

    Article  Google Scholar 

  2. Bartlett M, Viola P, Sejnowski T, Larsen J, Hager J, Ekman P. Classifying Facial Action. Advances in Neural Information Processing Systems 8. Touretzky D et al. (eds.), Cambridge: MIT Press, MA, 1996.

    Google Scholar 

  3. Essa I, Pentland A. Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. Pattern Analysis and Machine Intelligence, 1997, 19(7): 757–763.

    Article  Google Scholar 

  4. Lien J. Automatic recognition of facial expressions using hidden Markov models and estimation of expression intensity [Ph.D. Dissertation]. Carnegie Mellon University, 1998.

  5. Oliver N, Pentland A, Berard F. LAFTER: A real-time face and lips tracker with facial expression recognition. Pattern Recognition, 2000, 33(8): 1369–1382.

    Article  Google Scholar 

  6. Bartlett M. Face Image Analysis by Unsupervised Learning. Kluwer Academic Publishers, 2001.

  7. Cohen I, Sebe N, Garg A, Chen L S, Huang T S. Facial expression recognition from video sequence. In Proc. Int. Conf. Multimedia and Exp (ICME), Lausanne, Switzerland, Aug. 26–29, 2002, pp.121–124.

  8. Yang P, Liu Q, Metaxas D N. Boosting coded dynamic features for facial action units and facial expression recognition. In Proc. CVPR, Minneapolis, USA, June 18–23, 2007, pp.511–518.

  9. Zhao G, Pietikainen M. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Analysis and Machine Intelligence, 2007, 29(6): 915–928.

    Article  Google Scholar 

  10. Ekman P. Universal and cultural difference in facial expressions of emotions. Nebraska Symposium on Motivation, 1971, Cole J K (ed.), Lincoln: University of Nebraska Press, 1972, 19: 207–283.

    Google Scholar 

  11. Scimmack U. Response latencies of pleasure and displeasure ratings: Further evidence for mixed feeling. Cognition and Emotion, 2005, 19(5): 671–691.

    Article  Google Scholar 

  12. Lang P J. The emotion probe: Studies of motivation and attention. American Psychologist, 1995, 50(5): 372–385.

    Article  Google Scholar 

  13. Russell J A. Evidence of convergent validity on the dimension of affect. J. Personality and Social Psychology, 1978, 30(38): 1152–1168.

    Article  Google Scholar 

  14. Peter C, Herbon A. Emotion representation and physiology assignments in digital systems. Interacting with Computers, 2006, 18(2): 139–170.

    Article  Google Scholar 

  15. Donato G, Bartlett M, Hager J, Ekman P, Sejnowski T. Classifying facial actions. IEEE PAMI, 1999, 21(10): 974–989.

    Google Scholar 

  16. Schmidt K, Cohn J. Dynamics of facial expression: Normative characteristics and individual difference. In Proc. Int. Conf. Multimedia and Expo, Tokyo, Japan, Aug. 22–25, 2001, pp.547–550.

  17. Tong Y, Wang Y, Zhu Z, Ji Q. Robust facial feature tracking under varying face pose and facial expression. Pattern Recognition, 2007, 40(11): 3195–3208.

    Article  MATH  Google Scholar 

  18. Kong H, Wang L, Teoh E K, Li X. Generalized 2D principal component analysis for face image representation and recognition. Neural Network, 2005, 18(5/6): 585–594.

    Article  Google Scholar 

  19. Penev P S. Local feature analysis: A statistical theory for information representation and transmission [Ph.D. Dissertation]. Rockefeller University, 1998.

  20. Seung H S, Lee D D. The manifold ways of perception. Science, 2000, 290(12): 2268–2269.

    Article  Google Scholar 

  21. Roweis S T, Saul L K. Nonlinear dimensionality reduction by locally linear embedding. Science, 2000, 290(5500): 2323–2326.

    Article  Google Scholar 

  22. Carrol J M, Russell J A. Do facial expressions signal specific emotions? Judging emotion from the face in context. Journal of Personality and Social Psychology, 1996, 70(2): 205–218.

    Article  Google Scholar 

  23. Russell J A. Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 1994, 115(1): 112–141.

    Article  Google Scholar 

  24. Russell J A. Culture and categorization of emotion. Psychological Bulletin, 1991, 110: 426–450.

    Article  Google Scholar 

  25. Osgood C E, MayWH, Miron M S. Cross-Cultural Universals of Affective Meaning. Urbana: Univ. Illinois Press., 1975.

  26. Russell JA, Lewicka M, Nitt T. A cross-cultural study of a circumplex model of affect. Journal of Personality and Social Psychology, 1989, 57(5): 848–856.

    Article  Google Scholar 

  27. Russell J A, Ridgeway D. Dimension underlying children’s emotion concepts. Developmental Psychology, 1983, 19: 795–804.

    Article  Google Scholar 

  28. Kim Y A, Kim J K, Park S K, Oh K J, Chung C S. The study of dimension of internal states through word analysis about emotion. Korean Journal of the Science of Emotion and Sensibility, 1998, 1(1): 145–152.

    Google Scholar 

  29. Kim J K, Mun H S, Oh K J. Stability of two-dimension structure of emotion. Korean Journal of the Science of Emotion and Sensibility, 1999, 2(1): 43–52.

    Google Scholar 

  30. Bahn S B, Han J H, Chung C S. Facial expression database for mapping facial expression onto internal state. In Proc. Emotion Conf. Korea, Seoul, Korea, Nov. 1997, pp.215–219.

Download references

Author information

Authors and Affiliations

  1. Department of Information and Communication Engineering, Chosun University, #375 Seosuk-dong, Dong-gu, Gwangju, 501-759, Korea

    Young-Suk Shin

Authors
  1. Young-Suk Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Young-Suk Shin.

Additional information

This study was supported by research funds from Chosun University, 2008.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(PDF 144 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shin, YS. Facial Expression Recognition of Various Internal States via Manifold Learning. J. Comput. Sci. Technol. 24, 745–752 (2009). https://doi.org/10.1007/s11390-009-9257-9

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-009-9257-9

Keywords

Search

Navigation