Learning Effective Models of Emotions from Physiological Signals: The Seven Principles

  • Rui HenriquesEmail author
  • Ana Paiva
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8908)


Learning effective models from emotion-elicited physiological responses for the classification and description of emotions is increasingly required to derive accurate analysis from affective interactions. Despite the relevance of this task, there is still lacking an integrative view of existing contributions. Additionally, there is no agreement on how to deal with the differences of physiological responses across individuals, and on how to learn from flexible sequential behavior and subtle but meaningful spontaneous variations of the signals. In this work, we rely on empirical evidence to define seven principles for a robust mining physiological signals to recognize and characterize affective states. These principles compose a coherent and complete roadmap for the development of new methods for the analysis of physiological signals. In particular, these principles address the current over-emphasis on feature-based models by including critical generative views derived from different streams of research, including multivariate data analysis and temporal data mining. Additionally, we explore how to use background knowledge related with the experimental setting and psychophysiological profiles from users to shape the learning of emotion-centered models. A methodology that integrates these principles is proposed and validated using signals collected during human-to-human and human-to-robot affective interactions.


Physiological signals Emotion recognition Emotion description Affective interactions 



This article is an extended version of our previous work [10]. This work is supported by Fundação para a Ciência e Tecnologia under the project PEst-OE/EEI/LA0021/2013 and PhD grant SFRH/BD/ 75924/2011, and by the project EMOTE from the EU 7thFramework Program (FP7/2007–2013). The authors would like to thank: Tiago Ribeiro for implementing the robots’ behavior with sharp expressiveness, Iolanda Leite and Ivo Capelo for their support during the preparation and execution of the experiments, and Arvid Kappas for his contributions on the design of the experiments.


  1. 1.
    Andreassi, J.: Psychophysiology: Human Behavior and Physiological Response. Lawrence Erlbaum, Mahwah (2007)Google Scholar
  2. 2.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics. Springer, New York (2006)zbMATHGoogle Scholar
  3. 3.
    Bos, D.O.: EEG-based emotion recognition the influence of visual and auditory stimuli. Emotion 57(7), 1798–1806 (2006)Google Scholar
  4. 4.
    Cacioppo, J., Tassinary, L., Berntson, G.: Handbook of Psychophysiology. Cambridge University Press, New York (2007)CrossRefGoogle Scholar
  5. 5.
    Chang, C., Zheng, J., Wang, C.: Based on support vector regression for emotion recognition using physiological signals. In: IJCNN, pp. 1–7 (2010)Google Scholar
  6. 6.
    Ekman, P., Friesen, W.V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W.A., Pitcairn, T., Ricci-Bitti, P.E., Scherer, K.R., Tomita, M., Tzavaras, A.: Universals and cultural differences in the judgments of facial expressions of emotion. J. Pers. Soc. Psychol. 53, 712–717 (1988)CrossRefGoogle Scholar
  7. 7.
    Haag, A., Goronzy, S., Schaich, P., Williams, J.: Emotion recognition using bio-sensors: first steps towards an automatic system. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS (LNAI), vol. 3068, pp. 36–48. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  8. 8.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. Newsl. 11(1), 10–18 (2009)CrossRefGoogle Scholar
  9. 9.
    Henriques, R., Paiva, A.: Descriptive models of emotion: learning useful abstractions from physiological responses during affective interactions. In: PhyCS Special Session on Recognition of Affect Signals from PhysiologIcal Data for Social Robots (OASIS’14). SCITEPRESS, Lisbon (2014)Google Scholar
  10. 10.
    Henriques, R., Paiva, A.: Seven principles to mine flexible behavior from physiological signals for effective emotion recognition and description in affective interactions. In: Physiological Computing Systems (PhyCS’14). SCITEPRESS, Lisbon (2014)Google Scholar
  11. 11.
    Henriques, R., Paiva, A., Antunes, C.: On the need of new methods to mine electrodermal activity in emotion-centered studies. In: Cao, L., Zeng, Y., Symeonidis, A.L., Gorodetsky, V.I., Yu, P.S., Singh, M.P. (eds.) ADMI. LNCS (LNAI), vol. 7607, pp. 203–215. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  12. 12.
    Jerritta, S., Murugappan, M., Nagarajan, R., Wan, K.: Physiological signals based human emotion recognition: a review. In: IEEE 7th International Colloquium on Signal Processing and its Applications (CSPA) 2011, pp. 410–415 (2011)Google Scholar
  13. 13.
    Katsis, C., Katertsidis, N., Ganiatsas, G., Fotiadis, D.: Toward emotion recognition in car-racing drivers: a biosignal processing approach. IEEE Trans. Syst. Man Cybern. Syst. Hum. 38(3), 502–512 (2008)CrossRefGoogle Scholar
  14. 14.
    Kulic, D., Croft, E.A.: Affective state estimation for human-robot interaction. Trans. Rob. 23(5), 991–1000 (2007)CrossRefGoogle Scholar
  15. 15.
    Lang, P.: The emotion probe: studies of motivation and attention. Am. Psychol. 50, 372–372 (1995)CrossRefGoogle Scholar
  16. 16.
    Leite, I., Henriques, R., Martinho, C., Paiva, A.: Sensors in the wild: exploring electrodermal activity in child-robot interaction. In: HRI, pp. 41–48. ACM/IEEE (2013)Google Scholar
  17. 17.
    Lessard, C.S.: Signal Processing of Random Physiological Signals. Synthesis Lectures on Biomedical Engineering. Morgan and Claypool Publishers, San Rafael (2006)Google Scholar
  18. 18.
    Lin, J., Keogh, E., Lonardi, S., Chiu, B.: A symbolic representation of time series, with implications for streaming algorithms. In: ACM SIGMOD Workshop on DMKD, pp. 2–11. ACM, New York (2003)Google Scholar
  19. 19.
    Lin, J., Keogh, E.J., Lonardi, S., Chiu, B.Y.: A symbolic representation of time series, with implications for streaming algorithms. In: Zaki, M.J., Aggarwal, C.C. (eds.) DMKD, pp. 2–11. ACM (2003)Google Scholar
  20. 20.
    Maaoui, C., Pruski, A., Abdat, F.: Emotion recognition for human-machine communication. In: IROS, pp. 1210–1215. IEEE/RSJ (2008)Google Scholar
  21. 21.
    Mitsa, T.: Temporal data mining. In: DMKD. Chapman & Hall/CRC (2009)Google Scholar
  22. 22.
    Murphy, K.: Dynamic Bayesian networks: representation, inference and learning. Ph.D. thesis, UC Berkeley, CS Division (2002)Google Scholar
  23. 23.
    Oatley, K., Keltner, D., Jenkins, J.M.: Understanding Emotions. Blackwell, Cambridge (2006)Google Scholar
  24. 24.
    Petrantonakis, P., Hadjileontiadis, L.: Emotion recognition from EEG using higher order crossings. TITB 14(2), 186–197 (2010)Google Scholar
  25. 25.
    Picard, R.W.: Affective computing: challenges. Int. J. Hum. Comput. Stud. 59(1–2), 55–64 (2003)CrossRefGoogle Scholar
  26. 26.
    Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)CrossRefGoogle Scholar
  27. 27.
    Rani, P., Liu, C., Sarkar, N., Vanman, E.: An empirical study of machine learning techniques for affect recognition in human-robot interaction. Pattern Anal. Appl. 9(1), 58–69 (2006)CrossRefGoogle Scholar
  28. 28.
    Rigas, G., Katsis, C.D., Ganiatsas, G., Fotiadis, D.I.: A user independent, biosignal based, emotion recognition method. In: Conati, C., McCoy, K., Paliouras, G. (eds.) UM 2007. LNCS (LNAI), vol. 4511, pp. 314–318. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  29. 29.
    Villon, O., Lisetti, C.: Toward recognizing individual’s subjective emotion from physiological signals in practical application. In: Computer-Based Medical Systems, pp. 357–362 (2007)Google Scholar
  30. 30.
    Vyzas, E.: Recognition of emotional and cognitive states using physiological data. Master’s thesis, MIT (1999)Google Scholar
  31. 31.
    Wagner, J., Kim, J., Andre, E.: From physiological signals to emotions: implementing and comparing selected methods for feature extraction and classification. In: ICME, pp. 940–943. IEEE (2005)Google Scholar
  32. 32.
    Wu, C.K., Chung, P.C., Wang, C.J.: Extracting coherent emotion elicited segments from physiological signals. In: WACI, pp. 1–6. IEEE (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.KDBIO, Inesc-ID, Instituto Superior TécnicoUniversidade de LisboaLisboaPortugal
  2. 2.GAIPS, Inesc-ID, Instituto Superior TécnicoUniversidade de LisboaLisboaPortugal

Personalised recommendations