Skip to main content
SpringerLink
Log in

An Online Fuzzy-Based Approach for Human Emotions Detection: An Overview on the Human Cognitive Model of Understanding and Generating Multimodal Actions

  • Chapter
Intelligent Assistive Robots

Part of the book series: Springer Tracts in Advanced Robotics ((STAR,volume 106))

Abstract

An intelligent robot needs to be able to understand human emotions, and to understand and generate actions through cognitive systems that operate in a similar way to human cognition. In this chapter, we mainly focus on developing an online incremental learning system of emotions using Takagi-Sugeno (TS) fuzzy model. Additionally, we present a general overview for understanding and generating multimodal actions from the cognitive point of view. The main objective of this system is to detect whether the observed emotion needs a new corresponding multimodal action to be generated in case it constitutes a new emotion cluster not learnt before, or it can be attributed to one of the existing actions in memory in case it belongs to an existing cluster.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Fogassi, L., Ferrari, P., Gesierich, B., Rozzi, S., Chersi, F., Rizzolatti, G.: Parietal lobe: From action organization to intention understanding. Science 308, 662–667 (2005)

    Article  Google Scholar 

  2. Gallese, V., Fadiga, L., Fogassi, L., Rizzolatti, G.: Action recognition in the premotor cortex. Brain 119, 593–609 (1996)

    Article  Google Scholar 

  3. Schaffler, L., Luders, H., Dinner, D., Lesser, R., Chelune, G.: Comprehension deficits elicited by electrical stimulation of broca’s area. Brain 116, 695–715 (1993)

    Article  Google Scholar 

  4. Gazzola, V., Keysers, C.: The observation and execution of actions share motor and somatosensory voxels in all tested subjects: Single-subject analyses of unsmoothed fmri data. Cerebral Cortex 19, 1239–1255 (2009)

    Article  Google Scholar 

  5. Iacoboni, M., Woods, R., Brass, M., Bekkering, H., Mazziotta, J., Rizzolatti, G.: Cortical mechanisms of human imitation. Science 286, 2526–2528 (1999)

    Article  Google Scholar 

  6. Ramachandran, V.: Mirror neurons and imitation learning as the driving force behind ”the great leap forward” in human evolution. Edge 69 (2000)

    Google Scholar 

  7. Rizzolatti, G., Fogassi, L., Gallese, V.: Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews Neuroscience 2, 661–670 (2001)

    Article  Google Scholar 

  8. Rizzolatti, G., Arbib, M.: Language within our grasp. Trends in Neurosciences 21, 188–194 (1998)

    Article  Google Scholar 

  9. Gallese, V., Goldman, A.: Mirror neurons and the simulation theory of mind reading. Trends in Cognitive Sciences 2, 493–500 (1998)

    Article  Google Scholar 

  10. Ramachandran, V., Oberman, L.: Broken mirrors: A theory of autism. Scientific American 295, 62–69 (2006)

    Article  Google Scholar 

  11. Ojemann, G., Ojemann, J., Lettich, E., Berger, M.: Cortical language localization in left, dominant hemisphere: An electrical stimulation mapping investigation in 117 patients. Neurosurgery 71, 316–326 (1989)

    Article  Google Scholar 

  12. Whiten, A., Ham, R.: On the nature and evolution of imitation in the animal kingdom: Reappraisal of a century of research. Advances in the Study of Behavior 21, 239–283 (1992)

    Article  Google Scholar 

  13. Whiten, A., Custance, D., Gomez, J., Teixidor, P., Bard, K.: Imitative learning of artificial fruit processing in children (homo sapiens) and chimpanzees (pan troglodytes). Comparative Psychology 110, 3–14 (1996)

    Article  Google Scholar 

  14. Whiten, A.: Imitation of sequential and hierarchical structure in action: Experimental studies with children and chimpanzees. In: Cambridge, M.P. (ed.) Imitation in Animals and Artifacts, MA, USA, pp. 191–209 (2002)

    Google Scholar 

  15. Tomasello, M., Davis-Dasilva, M., Camak, L., Bard, K.: Observational learning of tool use by young chimpanzees and enculturated chimpanzees. Human Evolution 2, 175–183 (1987)

    Article  Google Scholar 

  16. Tomasello, M.: Emulation learning and cultural learning. Behavior and Brain Science 21, 703–704 (1998)

    Article  Google Scholar 

  17. Wood, D.: Social interaction as tutoring. In: Bornsten, M.H., Bruner, J. (eds.) Interaction in Human Development, Hillsdale, NJ, USA, pp. 59–80 (1989)

    Google Scholar 

  18. Whiten, A.: The scope of culture in chimpanzees, humans and ancestral apes. Philosophical Transactions of the Royal Society 366, 935–1187 (2011)

    Google Scholar 

  19. Galef, B.: The question of animal culture. Human Nature 3, 157–178 (1992)

    Article  Google Scholar 

  20. Heyes, C.: Imitation, culture and cognition. Animal Behavior 46, 999–1010 (1993)

    Article  Google Scholar 

  21. Tomasello, M., Savage-Rumbaugh, E., Kruger, A.: Imitative learning of actions on objects by children, chimpanzees and enculturated chimpanzees. Child Development 64, 1688–1705 (1993)

    Article  Google Scholar 

  22. Buchsbaum, D., Griffiths, T., Gopnik, A., Baldwin, D.: Learning from actions and their consequences: Inferring causal variables from continuous sequences of human action. In: Proceedings of the 31st Annual Conference of the Cognitive Science Society (2009)

    Google Scholar 

  23. Buchsbaum, D., Canini, K., Griffiths, T.: Segmenting and recognizing human action using low-level video. In: Proceedings of the 33rd Annual Conference of the Cognitive Science Society (2011)

    Google Scholar 

  24. Tani, J.: Learning to generate articulated behavior through the bottom-up and the top-down interaction processes. Neural Networks 16, 11–23 (2003)

    Article  Google Scholar 

  25. Tani, J., Ito, M., Sugita, Y.: Self-organization of distributedly represented multiple behavior schemata in a mirror system: Reviews of robot experiments using rnnpb. Neural Networks 17, 1273–1289 (2004)

    Article  Google Scholar 

  26. Issar, S., Ward, W.: Cmu’s robust spoken language understanding system. In: Proceedings of the 3rd European Conference on Speech Communication and Technology, EUROSPEECH (1993)

    Google Scholar 

  27. Bennacef, S., Bonnea-Maynard, H., Gauvain, J., Lamel, L., Minker, W.: A spoken language system for information retrieval. In: Proceedings of the 3rd International Conference on Spoken Language Processing, ICSLP (1994)

    Google Scholar 

  28. Miller, S., Bobrow, R., Schwartz, R., Ingria, R.: Statistical language processing using hidden understanding models. In: Proceedings of the Human Language Technology Workshop, NJ, USA (1994)

    Google Scholar 

  29. Levin, E., Pieraccini, R.: Concept-based spontaneous speech understanding system. In: Proceedings of the 4th European Conference on Speech Communication and Technology, EUROSPEECH (1995)

    Google Scholar 

  30. Goldberg, E., Driedger, N., Kittredge, R.: Using natural language processing to produce weather forecasts. IEEE Intelligent Systems and their Applications 9, 45–53 (1994)

    Google Scholar 

  31. Busemann, S.: Ten years after: An update on tg/2 (and friends). In: Proceedings of the European Natural Language Generation Workshop (2005)

    Google Scholar 

  32. Mcroy, S., Channarukul, S., Ali, S.: An augmented template-based approach to text realization. Natural Language Engineering 9, 381–420 (2003)

    Article  Google Scholar 

  33. Bateman, A.: Enabling technology for multilingual natural language generation: The kmpl development. Natural Language Engineering 3, 15–55 (1997)

    Article  Google Scholar 

  34. Lavoie, B., Rambow, O.: A fast and portable realizer for text generation. In: Proceedings of the 5th Conference on Applied Natural-Language Processing, ANLP (1997)

    Google Scholar 

  35. Gergely, G.: What should a robot learn from an infant? mechanisms of action interpretation and observational learning in infancy. Connection Science 15, 191–209 (2003)

    Article  Google Scholar 

  36. Kozima, H., Nakagawa, C., Yano, H.: Emergence of imitation mediated by objects. In: Proceedings of the 2nd International Workshop on Epigenetic Robotics (2002)

    Google Scholar 

  37. Rudolph, M., Muhlig, M., Gienger, M., Bohme, H.: Learning the consequences of actions: Representing effects as feature changes. In: Proceedings of the International Symposium on Learning and Adaptive Behavior in Robotic System (2010)

    Google Scholar 

  38. Murray, I., Arnott, J.: Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotion. Journal of the Acoustical Society of America 93, 1097–1108 (1993)

    Article  Google Scholar 

  39. Cahn, J.: Generating expression in synthesized speech. In Master’s thesis, MIT Media Lab, USA (1990)

    Google Scholar 

  40. Roy, D., Pentland, A.: Automatic spoken affect analysis and classification. In: Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition, Vermont, USA (1996)

    Google Scholar 

  41. Slaney, M., McRoberts, G.: Baby ears: A recognition system for affective vocalizations. In: Proceedings of the 1998 International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Seattle, USA (1998)

    Google Scholar 

  42. Breazeal, C., Aryananda, L.: Recognition of affective communicative intent in robot-directed speech. Autonomous Robots Journal 12, 83–104 (2002)

    Article  MATH  Google Scholar 

  43. Vogt, T., Andre, E.: Improving automatic emotion recognition from speech via gender differentiation. In: Proceedings of the Language Resources and Evaluation Conference, LREC 2006 (2006)

    Google Scholar 

  44. Voeffra, C.: Emotion-sensitive human-computer interaction (hci): State of the art. In: Seminar Emotion Recognition (2011), http://diuf.unifr.ch/main/diva/teaching/seminars/emotion-recognition

  45. Pierre-Yves, O.: The production and recognition of emotions in speech: features and algorithms. Human-Computer Studies 59 (2003)

    Google Scholar 

  46. Jones, C., Deeming, A.: Affective human-robot interaction. In: Peter, C., Beale, R. (eds.) Affect and Emotion in Human-Computer Interaction: From Theory to Applications, pp. 175–185 (2008)

    Google Scholar 

  47. Zadeh, L.: Fuzzy sets. Information and Control 8, 338–353 (1965)

    Article  MATH  MathSciNet  Google Scholar 

  48. Zadeh, L.: Outline of a new approach to the analysis of complex systems and decision processes. IEEE Transactions on Systems, Man, and Cybernetics 3, 28–44 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  49. Mamdani, E., Assilian, S.: An experiment in linguistic synthesis with a fuzzy logic controller. International Journal of Man-Machine Studies 7, 1–13 (1975)

    Article  MATH  Google Scholar 

  50. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its application to modeling and control. IEEE Trans. on Systemsm Man, and Cybernetics 15, 116–132 (1985)

    Article  MATH  Google Scholar 

  51. Sugeno, M.: Industrial applications of fuzzy control. Elsevier Science Pub. Co. (1985)

    Google Scholar 

  52. Bezdek, J.: Pattern recognition with fuzzy objective function algorithms. Plenum Press, New York (1981)

    Book  MATH  Google Scholar 

  53. Vapnik, V.: Statistical learning theory. In: Haykin, S. (ed.) Adaptive and Learning Systems. John Wiley and Sons (1998)

    Google Scholar 

  54. Dunn, J.: A fuzzy relative of the isodata process and its use in detecting compact well-separated clusters. Journal of Cybernetics 3, 32–57 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  55. Gustafsson, D., Kessel, W.: Fuzzy clustering with a fuzzy covariance matrix. In: Proceedings of the IEEE CDC, San Diego, CA, USA, pp. 761–766 (1979)

    Google Scholar 

  56. Gath, I., Geva, A.: Unsupervised optimal fuzzy clustering. IEEE Trans. on Pattern Analysis and Machine Intelligence 11, 773–781 (1989)

    Article  Google Scholar 

  57. Yager, R., Filev, D.: Approximate clustering via the mountain method. In Technical Report MII 1305, Machine Intelligence Institute, Iona College, New Rochelle (1992)

    Google Scholar 

  58. Yager, R., Filev, D.: Learning of fuzzy rules by mountain clustering. In: Proceedings of SPIE Conference on Applications of Fuzzy Logic Technology, Boston, MA, pp. 246–254 (1993)

    Google Scholar 

  59. Chiu, S.: Fuzzy model identification based on cluster estimation. Journal of Intelligent and Fuzzy Systems 2, 267–278 (1994)

    Google Scholar 

  60. Searle, J.: Austin on locutionary and illocutionary acts. The Philosophical Review 77, 405–424 (1968)

    Article  Google Scholar 

  61. Searle, J.: Speech acts: An essay in the philosophy of language. Cambridge University Press (1969)

    Google Scholar 

  62. Goldberg, L.: An alternative description of personality: The big-five factor structure. Personality and Social Psychology 59, 1216–1229 (1990)

    Article  Google Scholar 

  63. Aly, A., Tapus, A.: A model for synthesizing a combined verbal and nonverbal behavior based on personality traits in human-robot interaction. In: Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction, HRI (2013)

    Google Scholar 

  64. Summers-Stay, D., Teo, C., Yang, Y., Fermuller, C., Aloimonos, Y.: Using a minimal action grammar for activity understanding in the real world. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS (2012)

    Google Scholar 

  65. Pastra, K., Aloimonos, Y.: The minimalist grammar of action. Philosophical Transactions B 367, 103–117 (2012)

    Article  Google Scholar 

  66. Izard, C.: Face of emotion. Appleton, New York (1971)

    Google Scholar 

  67. Plutchik, R.: The nature of emotions. University Press of America, Lanham (1991)

    Google Scholar 

  68. Ekman, P.: Emotion in the human face: Guidelines for research and an integration of findings. Pergamon Press, New York (1972)

    Google Scholar 

  69. Ekman, P., Friesen, W., Ellsworth, P.: What emotion categories or dimensions can observers judge from facial behavior? In: Ekman, P. (ed.) Emotion in the Human Face. Cambridge University Press, New York (1982)

    Google Scholar 

  70. Izard, C.: Human emotions. Plenum Press, New York (1977)

    Book  Google Scholar 

  71. Tomkins, S.: Affect theory. In: Scherer, K., Ekman, P. (eds.) Approaches to Emotion, pp. 163–195. Erlbaum, Hillsdale (1984)

    Google Scholar 

  72. Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20, 273–297 (1995)

    MATH  Google Scholar 

  73. Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W., Weiss, B.: A database of german emotional speech. In: Proc. of Interspeech, Germany (2005), http://database.syntheticspeech.de

  74. Banse, R., Scherer, K.: Acoustic profiles in vocal emotion expression. Journal of Personality and Social Psychology 70, 614–636 (1996)

    Article  Google Scholar 

  75. Montero, J., Gutierrez-Arriola, J., Palazuelos, S., Enriquez, E., Aguilera, S., Pardo, J.: Emotional speech synthesis: from speech database to tts. In: Proceedings of the International Conference on Spoken Language Processing 1998, pp. 923–925 (1998)

    Google Scholar 

  76. Talkin, D.: A robust algorithm for pitch tracking. In: Kleijn, W.B., Paliwal, K. (eds.) Speech Coding and Synthesis, pp. 497–518. Elsevier (1995)

    Google Scholar 

  77. Sondhi, M.: New methods of pitch extraction. IEEE Trans. Audio and Electroacoustics 16, 262–266 (1968)

    Article  Google Scholar 

  78. Rabiner, L., Atal, B., Sambur, M.: Lpc prediction error: Analysis of its variation with the position of the analysis frame. IEEE Trans. on Systems Man, and Cybernetics 25, 434–442 (1977)

    Google Scholar 

  79. Rong, J., Li, G., Chen, Y.P.: Acoustic feature selection for automatic emotion recognition from speech. Information Processing and Management 45, 315–328 (2008)

    Article  Google Scholar 

  80. Cristianini, N., Shawe-Taylor, J.: Introduction to support vector machines. Cambridge University Press (2000)

    Google Scholar 

  81. Platt, J.: Fast training of support vector machines using sequential mininal optimization. In Microsoft Research Technical Report MSR-TR-98-14 (1998)

    Google Scholar 

  82. Angelov, P.: Evolving rule-based models: A tool for design of flexible adaptive systems. STUDFUZZ, vol. 92. Springer, Heidelberg (2002)

    Google Scholar 

  83. Aly, A., Tapus, A.: Towards an online real time fuzzy modeling for human internal states detection. In: Proceedings of the 12th IEEE International Conference on Control, Automation, Robotics and Vision, ICARCV (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Robotics and Computer Vision Lab, ENSTA ParisTech, 828 Boulevard des Maréchaux, 91120, Palaiseau, France

    Amir Aly & Adriana Tapus

Authors
  1. Amir Aly
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adriana Tapus
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amir Aly .

Editor information

Editors and Affiliations

  1. LiSSi Laboratory, University Paris-Est/UPEC, Vitry-sur-Seine, France

    Samer Mohammed

  2. Bioengineering Group (GBIO), Spanish National Research Council (CSIC), Madrid, Spain

    Juan C. Moreno

  3. Department of Mechanical Engineering Robotic Systems Control Laboratory, Sogang University, Seoul, Korea, Republic of (South Korea)

    Kyoungchul Kong

  4. LiSSi Laboratory, University Paris-Est/UPEC, Vitry-sur-Seine, France

    Yacine Amirat

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Aly, A., Tapus, A. (2015). An Online Fuzzy-Based Approach for Human Emotions Detection: An Overview on the Human Cognitive Model of Understanding and Generating Multimodal Actions. In: Mohammed, S., Moreno, J., Kong, K., Amirat, Y. (eds) Intelligent Assistive Robots. Springer Tracts in Advanced Robotics, vol 106. Springer, Cham. https://doi.org/10.1007/978-3-319-12922-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-12922-8_7

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-12921-1

  • Online ISBN: 978-3-319-12922-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation