On the Need of New Methods to Mine Electrodermal Activity in Emotion-Centered Studies

  • Rui Henriques
  • Ana Paiva
  • Cláudia Antunes
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7607)


Monitoring the electrodermal activity is increasingly accomplished in agent-based experimental settings as the skin is believed to be the only organ to react only to the sympathetic nervous system. This physiological signal has the potential to reveal paths that lead to excitement, attention, arousal and anxiety. However, electrodermal analysis has been driven by simple feature-extraction, instead of using expressive models that consider a more flexible behavior of the signal for improved emotion recognition. This paper proposes a novel approach centered on sequential patterns to classify the signal into a set of key emotional states. The approach combines SAX for pre-processing the signal and hidden Markov models. This approach was tested over a collected sample of signals using Affectiva-QSensor. An extensive human-to-human and human-to-robot experimental setting is under development for further validation and characterization of emotion-centered patterns.


Hide Markov Model Emotion Recognition Physiological Signal Skin Conductance Response Dynamic Bayesian Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andreassi, J.: Psychophysiology: Human Behavior and Physiological Response. In: Psychophysiology: Human Behavior & Phy. Response. Lawrence Erlbaum (2007)Google Scholar
  2. 2.
    Ben-Shakhar, G.: A Further Study of the Dichotomization Theory in Detection of Information. Psychophysiology 14, 408–413 (1977)CrossRefGoogle Scholar
  3. 3.
    Ben-Shakhar, G.: Standardization within individuals: A simple method to neutralize individual differences in skin conductance. Psychophy 22(3), 292–299 (1985)CrossRefGoogle Scholar
  4. 4.
    Bilmes, J.A.: What hmms can do. IEICE Journal E89-D(3), 869–891 (2006)Google Scholar
  5. 5.
    Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York, Inc., Secaucus (2006)Google Scholar
  6. 6.
    Bos, D.O.: Eeg-based emotion recognition the influence of visual and auditory stimuli. Emotion 57(7), 1798–1806 (2006)Google Scholar
  7. 7.
    Brown, G., Birley, J., Wing, J.: Influence of family life on the course of schizophrenic disorders: a replication. B.J. of Psychiatry 121(562), 241–258 (1972)CrossRefGoogle Scholar
  8. 8.
    Cacioppo, J., Tassinary, L., Berntson, G.: Handbook of psychophysiology. Cambridge University Press (2007)Google Scholar
  9. 9.
    Cao, L.: Data mining and multi-agent integration. Springer, Dordrecht (u.a) (2009)Google Scholar
  10. 10.
    Chang, C., Zheng, J., Wang, C.: Based on support vector regression for emotion recognition using physiological signals. In: IJCNN, pp. 1–7 (2010)Google Scholar
  11. 11.
    Crider, A.: Electrodermal response lability-stability: Individual difference correlates. In: Prog. in Electrod. Research, vol. 249, pp. 173–186. Springer, US (1993)CrossRefGoogle Scholar
  12. 12.
    Ding, H., Trajcevski, G., Scheuermann, P., Wang, X., Keogh, E.J.: Querying and mining of time series data: experimental comparison of representations and distance measures. PVLDB 1(2), 1542–1552 (2008)Google Scholar
  13. 13.
    Ekman, P., Friesen, W.: Universals and cultural differences in the judgments of facial expressions of emotion. J. of Personality and Social Psychology 53, 712–717 (1988)CrossRefGoogle Scholar
  14. 14.
    Haag, A., Goronzy, S., Schaich, P., Williams, J.: Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS (LNAI), vol. 3068, pp. 36–48. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  15. 15.
    Jerritta, S., Murugappan, M., Nagarajan, R., Wan, K.: Physiological signals based human emotion recognition: a review. In: 2011 IEEE 7th International Colloquium on Signal Processing and its Applications (CSPA), pp. 410–415 (2011)Google Scholar
  16. 16.
    Katsis, C., Katertsidis, N., Ganiatsas, G., Fotiadis, D.: Toward emotion recognition in car-racing drivers: A biosignal processing approach. IEEE Transactions on Systems, Man and Cybernetics, Systems and Humans 38(3), 502–512 (2008)CrossRefGoogle Scholar
  17. 17.
    Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (IAPS): Technical Manual and Affective Ratings. NIMH (1997)Google Scholar
  18. 18.
    Lang, P.: The emotion probe: Studies of motivation and attention. American Psychologist 50, 372–372 (1995)CrossRefGoogle Scholar
  19. 19.
    Lessard, C.S.: Signal Processing of Random Physiological Signals. Synthesis Lectures on Biomedical Engineering, Morgan and Claypool Publishers (2006)Google Scholar
  20. 20.
    Lin, J., Keogh, E., Lonardi, S., Chiu, B.: A symbolic representation of time series, with implications for streaming algorithms. In: ACM SIGMOD Workshop on DMKD, pp. 2–11. ACM, New York (2003)Google Scholar
  21. 21.
    Lorber, M.F.: Psychophysiology of aggression, psychopathy, and conduct problems: a meta-analysis. Psychological Bulletin 130(4), 531–552 (2004)CrossRefGoogle Scholar
  22. 22.
    Lykken, D.T.: The gsr in the detection of guilt. J. A. Psych. 43(6), 385–388 (1959)Google Scholar
  23. 23.
    Lykken, D.: A study of anxiety in the sociopathic personality. U. Minnesota (1955)Google Scholar
  24. 24.
    Maaoui, C., Pruski, A., Abdat, F.: Emotion recognition for human-machine communication. In: IROS, pp. 1210–1215. IEEE/RSJ (September 2008)Google Scholar
  25. 25.
    Mitsa, T.: Temporal Data Mining. In: DMKD. Chapman & Hall/CRC (2009)Google Scholar
  26. 26.
    Murphy, K.: Dynamic Bayesian Networks: Representation, Inference and Learning. Ph.D. thesis, UC Berkeley, Computer Science Division (July 2002)Google Scholar
  27. 27.
    Oatley, K., Keltner, Jenkins: Understanding Emotions. Blackwell P. (2006)Google Scholar
  28. 28.
    Petrantonakis, P.C., Hadjileontiadis, L.J.: Emotion recognition from eeg using higher order crossings. Trans. Info. Tech. Biomed. 14(2), 186–197 (2010)CrossRefGoogle Scholar
  29. 29.
    Picard, R.W.: Affective computing: challenges. International Journal of Human-Computer Studies 59(1-2), 55–64 (2003)CrossRefGoogle Scholar
  30. 30.
    Rabiner, L., Juang, B.: An introduction to hidden Markov models. ASSP Magazine 3(1), 4–16 (2003)CrossRefGoogle Scholar
  31. 31.
    Rigas, G., Katsis, C.D., Ganiatsas, G., Fotiadis, D.I.: A User Independent, Biosignal Based, Emotion Recognition Method. In: Conati, C., McCoy, K., Paliouras, G. (eds.) UM 2007. LNCS (LNAI), vol. 4511, pp. 314–318. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  32. 32.
    Schell, A.M., Dawson, M.E., Filion, D.L.: Psychophysiological correlates of electrodermal lability. Psychophysiology 25(6), 619–632 (1988)CrossRefGoogle Scholar
  33. 33.
    Shieh, J., Keogh, E.: isax: indexing and mining terabyte sized time series. In: ACM SIGKDD, KDD 2008, pp. 623–631. ACM, New York (2008)Google Scholar
  34. 34.
    Tranel, D., Damasio, H.: Neuroanatomical correlates of electrodermal skin conductance responses. Psychophysiology 31(5), 427–438 (1994)CrossRefGoogle Scholar
  35. 35.
    Villon, O., Lisetti, C.: Toward recognizing individual’s subjective emotion from physiological signals in practical application. In: Computer-Based Medical Systems, pp. 357–362 (2007)Google Scholar
  36. 36.
    Viterbi, A.: Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans. on Inf. Theory 13(2), 260–269 (1967)zbMATHCrossRefGoogle Scholar
  37. 37.
    Vyzas, E.: Recognition of Emotional and Cognitive States Using Physiological Data. Master’s thesis. MIT (1999)Google Scholar
  38. 38.
    Wagner, J., Kim, J., Andre, E.: From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification. In: ICME, pp. 940–943. IEEE (2005)Google Scholar
  39. 39.
    Wu, C.K., Chung, P.C., Wang, C.J.: Extracting coherent emotion elicited segments from physiological signals. In: WACI, pp. 1–6. IEEE (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Rui Henriques
    • 1
  • Ana Paiva
    • 1
    • 2
  • Cláudia Antunes
    • 1
  1. 1.DEI, Instituto Superior TécnicoTechnical University of LisbonPortugal
  2. 2.GAIPS, INESC–IDPortugal

Personalised recommendations