Skip to main content
SpringerLink
Log in

Fuzzy Models for Facial Expression-Based Emotion Recognition and Control

  • Chapter
Emotional Intelligence

Part of the book series: Studies in Computational Intelligence ((SCI,volume 234))

Abstract

The chapter examines the scope of fuzzy relational approach to human emotion recognition from facial expressions, and its control. Commercial audio-visual movies pre-selected for exciting specific emotions have been presented before subjects to arouse their emotions. The video clips of their facial expressions describing the emotions are recorded and analyzed by segmenting and localizing the individual frames into regions of interest. Selected facial features such as eye-opening, mouth-opening and the length of eyebrow-constriction are next extracted from the localized regions. These features are then fuzzified, and mapped on to an emotion space by employing Mamdani type relational model. A scheme for the validation of the system parameters is also presented. The later part of the chapter provides a fuzzy scheme for controlling the transition of emotion dynamics toward a desired state using suitable audio-visual movies. Experimental results and computer simulations indicate that the proposed scheme for emotion recognition and control is simple and robust with a good level of experimental accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bezdek, J.C.: Fuzzy Mathematics in Pattern Classification, Ph.D. Thesis, Applied Mathematics Center. Cornell University, Ithaca (1973)

    Google Scholar 

  2. Biswas, B., Mukherjee, A.K.: Template Matching with Fuzzy Descriptors. J. of Inst. of Engineers (1997)

    Google Scholar 

  3. Black, M.T., Yacoob, Y.: Recognizing facial expressions in image sequences using local parameterized models of image motion. Int. J. Com. Vis. 25, 23–48 (1997)

    Article  Google Scholar 

  4. Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C.M., Kazemzadeh, A., Lee, S., Neumann, U.: Analysis of emotion recognition using facial expressions, speech and multimodal information. In: Proc. of ICMI 2004, Pennsylvania (October 2004)

    Google Scholar 

  5. Cohen, I., Garg, A., Huang, T.S.: Emotion recognition using multilevel HMM. In: Proc. of the NIPS Workshop on Affective Computing, Colorado (2000)

    Google Scholar 

  6. Cohen, I.: Facial Expression Recognition from Video Sequences, MS Thesis, Univ. of Illinois at Urbana-Champaign, Dept. of Electrical Engg. (2000)

    Google Scholar 

  7. Conati, C., Zhou, X.: Modeling students’ emotions from cognitive appraisal in educational games. In: Proc. of the Sixth Int. Conf. On Intelligent Tutoring System, France (2002)

    Google Scholar 

  8. Donato, G., Bartlett, M.S., Hager, J.C., Ekman, P., Sejnowski, T.J.: Classifying facial actions. IEEE Trans. Pattern Anal. Machine Intell. 21, 974–989 (1999)

    Article  Google Scholar 

  9. Ekman, P., Friesen, W.V.: Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues. Prentice-Hall, New Jersey (1975)

    Google Scholar 

  10. Essa, I.A., Pentland, A.P.: Coding, analysis, interpretation and recognition of facial expressions. IEEE Trans. Pattern Anal. Machine Intell. 19, 757–763 (1997)

    Article  Google Scholar 

  11. Fellenz, W.A., Taylor, J.G., Cowie, R., Douglas-Cowie, E., Piat, F., Kollias, S., Orovas, C., Apolloni, B.: On emotion recognition of faces and of speech using neural networks, fuzzy logic and the ASSESS Systems. In: Proc. of the IEEE -INNS-ENNS Int. Joint Conf. Neural Networks, p. 2093 (2000)

    Google Scholar 

  12. Fernandez-Dols, J.M., Wallbotl, H., Sanchez, F.: Emotion Category Accessibility and the Decoding of Emotion from Facial Expression and Context. Journal of Nonverbal Behavior 15 (1991)

    Google Scholar 

  13. Gao, Y., Leung, M.K.H., Hui, S.C., Tananda, M.W.: Facial expression recognition from line-based caricatures. IEEE Trans. Systems, Man and Cybernetics- Part A: Systems and Humans 33(3) (May 2003)

    Google Scholar 

  14. Gordon, R.N.: The Structure of Emotions: Investigations in Cognitive Philosophy. Cambridge Studies in Philosophy. Cambridge University Press, Cambridge (1990)

    Google Scholar 

  15. Izumitani, K., Mikami, T., Inoue, K.: A Model of Expression Grade for Face Graphs Using Fuzzy Integral. System and Control 28(10), 590–596 (1984)

    Google Scholar 

  16. Kawakami, F., Morishima, S., Yamada, H., Harashima, H.: Construction of 3-D Emotion Space Using Neural Network. In: Proc. of the 3rd International Conference on Fuzzy Logic, Neural Nets and Soft Computing, Iizuka, pp. 309–310 (1994)

    Google Scholar 

  17. Klir, G.J., Yuan, B.: Fuzzy Sets and Fuzzy Logic: Theory and Applications. Prentice- Hall, New Jersey (1995)

    MATH  Google Scholar 

  18. Kobayashi, H., Hara, F.: The Recognition of basic Facial Expressions by neural network. Trans. on the society of Instrument and Control Engineers 29(1), 112–118 (1993)

    Google Scholar 

  19. Kobayashi, H., Hara, F.: Measurement of the Strength of Six Basic Facial Expressions by Neural Network. Trans. of the Japan Society of Mechanical Engineers (C) 59(567), 177–183 (1993)

    Google Scholar 

  20. Kobayashi, H., Hara, F.: Recognition of Mixed Facial Expressions by Neural Network, ibid., pp. 184–189 (1993)

    Google Scholar 

  21. Konar, A.: Computational Intelligence: Principles, Techniques and Applications. Springer, Heidelberg (2005)

    MATH  Google Scholar 

  22. Krammer, A.F., Sirevaag, E.J., Braune, R.: A psycho-physiological assessment of operator workload during simulated flight missions. Human Factors 29(2), 145–160 (1987)

    Google Scholar 

  23. Lanitis, A., Taylor, C.J., Cootes, T.F.: Automatic interpretation and coding of face images using flexible models. IEEE Trans. Pattern Anal. Machine Intell. 19, 743–756 (1997)

    Article  Google Scholar 

  24. Li, H., Roivainan, P., Forchheimer, R.: 3D Motion estimation in model-based facial image coding. IEEE Trans. Pattern Anal. Machine Intell. 15, 545–555 (1993)

    Article  Google Scholar 

  25. Li, X., Ji, Q.: Active affective state detection and user assistance with dynamic Bayesian networks. IEEE Trans. Systems, Man and Cybernetics-Part A: Systems and Humans 35(1) (January 2005)

    Google Scholar 

  26. Mase, K.: Recognition of facial expression from optical flow. Proc. IEICE Trans., Special Issue Coput. Vis. And Its Applications 74(10), 3474–3483 (1991)

    Google Scholar 

  27. Pantic, M., Rothkrantz, L.: Automatic analysis of facial expressions: State of the Art. IEEE Trans. Pattern Anal. Machine Intell. 22(2), 1424–1445 (2000)

    Article  Google Scholar 

  28. Pedrycz, W., Gomide, F.: An Introduction to Fuzzy Sets: Analysis and Design. MIT Press, Massachusetts (1998)

    MATH  Google Scholar 

  29. Pentland, E.A.P.: Coding, analysis, interpretation and recognition of facial expressions. IEEE Trans. on Pattern Analysis and Machine Intelligence 19(7), 757–763 (1997)

    Article  Google Scholar 

  30. Picard, R.: Affective Computing. MIT Press, Cambridge (1997)

    Google Scholar 

  31. Picard, R.W., Vyzas, E., Healy, J.: Toward machine emotional intelligence: analysis of affective psychological states. IEEE Trans. on Pattern Analysis and Machine Intelligence 23(10), 1175–1191 (2001)

    Article  Google Scholar 

  32. Rani, P., Sarkar, N., Adams, J.: Anxiety-based affective communication for implicit human-machine interaction. Advanced Engineering, Informatics 21(3), 323–334 (2007)

    Article  Google Scholar 

  33. Rani, P., Sarkar, N., Smith, C., Kirby, L.: Anxiety detecting robotic systems-towards implicit human-robot collaboration. Robotica 22(1), 83–93 (2004)

    Article  Google Scholar 

  34. Rosenblum, M., Yacoob, Y., Davis, L.: Human expression recognition from motion using a radial basis function network architecture. IEEE Trans. Neural Networks 7, 1121–1138 (1996)

    Article  Google Scholar 

  35. Scheirer, J., Fernadez, R., Klein, J., Picard, R.: Frustrating the user on purpose: a step toward building an affective computer. Interacting with Computers 14(2), 93–118 (2002)

    Google Scholar 

  36. Simon, H.: Motivational and Emotional Control of Cognition. In: Models of Thought, pp. 29–38. Yale University Press, New Haven (1979)

    Google Scholar 

  37. Terzopoulus, D., Waters, K.: Analysis and Synthesis of facial image sequences using physical and anatomical models. IEEE Trans. Pattern Anal. Machine Intell. 15, 569–579 (1993)

    Article  Google Scholar 

  38. Tian, Y., Kanade, T., Cohn, J.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Machine Intell. 23(2), 97–116 (2001)

    Article  Google Scholar 

  39. Ueki, N., Morishima, S., Harashima, H.: Expression Analysis/Synthesis System Based on Emotion Space Constructed by Multilayered Neural Network. Systems and Computers in Japan 25(13) (1994)

    Google Scholar 

  40. Uwechue, O.A., Pandya, S.A.: Human Face Recognition Using Third-Order Synthetic Neural Networks. Kluwer Academic publishers, Boston (1997)

    Google Scholar 

  41. Vanger, P., Honlinger, R., Haykin, H.: Applications of Synergetic in Decoding Facial Expressions of Emotions. In: Proc. of Int. Workshop on Automatic face and Gesture recognition, Zurich, pp. 24–29 (1995)

    Google Scholar 

  42. Vasilakos, A., Pedrycz, W.: Ambient Intelligence, Wireless Networking and Ubiquitous Computing. Artech House, Norwood (June 2006)

    Google Scholar 

  43. Yacoob, Y., Davis, L.: Computing spatio-temporal representations of human faces. In: Proc. of Computer Vision and Pattern Recognition, pp. 70–75. IEEE Computer Society Conference, Los Alamitos (1994)

    Chapter  Google Scholar 

  44. Yacoob, Y., Davis, L.: Recognizing human facial expression from long image sequences using optical flow. IEEE Trans. Pattern Anal. Machine Intell. 16, 636–642 (1996)

    Article  Google Scholar 

  45. Yamada, H.: Visual Information for categorizing Facial expression of Emotion. Applied Cognitive Psychology 7, 257–270 (1993)

    Article  Google Scholar 

  46. Zeng, Z., Fu, Y., Roisman, G.I., Wen, Z., Hu, Y., Huang, T.S.: Spontaneous emotional facial expression detection. J. of Multimedia 1(5) (August 2006)

    Google Scholar 

Download references

Authors
  1. Aruna Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amit Konar
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Chakraborty, A., Konar, A. (2009). Fuzzy Models for Facial Expression-Based Emotion Recognition and Control. In: Emotional Intelligence. Studies in Computational Intelligence, vol 234. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68609-5_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-68609-5_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-68606-4

  • Online ISBN: 978-3-540-68609-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation