Real-Time EEG-Based Emotion Recognition and Its Applications

  • Yisi Liu
  • Olga Sourina
  • Minh Khoa Nguyen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6670)


Since emotions play an important role in the daily life of human beings, the need and importance of automatic emotion recognition has grown with increasing role of human computer interface applications. Emotion recognition could be done from the text, speech, facial expression or gesture. In this paper, we concentrate on recognition of “inner” emotions from electroencephalogram (EEG) signals. We propose real-time fractal dimension based algorithm of quantification of basic emotions using Arousal-Valence emotion model. Two emotion induction experiments with music stimuli and sound stimuli from International Affective Digitized Sounds (IADS) database were proposed and implemented. Finally, the real-time algorithm was proposed, implemented and tested to recognize six emotions such as fear, frustrated, sad, happy, pleasant and satisfied. Real-time applications were proposed and implemented in 3D virtual environments. The user emotions are recognized and visualized in real time on his/her avatar adding one more so-called “emotion dimension” to human computer interfaces. An EEG-enabled music therapy site was proposed and implemented. The music played to the patients helps them deal with problems such as pain and depression. An EEG-based web-enable music player which can display the music according to the user’s current emotion states was designed and implemented.


emotion recognition EEG emotion visualization fractal dimension HCI BCI 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
  3. 3.
    American electroencephalographic society guidelines for standard electrode position nomenclature. Journal of Clinical Neurophysiology 8(2), 200–202 (1991)Google Scholar
  4. 4.
    Accardo, A., Affinito, M., Carrozzi, M., Bouquet, F.: Use of the fractal dimension for the analysis of electroencephalographic time series. Biological Cybernetics 77(5), 339–350 (1997)CrossRefzbMATHGoogle Scholar
  5. 5.
    Block, A., Von Bloh, W., Schellnhuber, H.J.: Efficient box-counting determination of generalized fractal dimensions. Physical Review A 42(4), 1869–1874 (1990)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bos, D.O.: EEG-based emotion recognition (2006),
  7. 7.
    Bradley, M.M.: Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25(1), 49–59 (1994)CrossRefGoogle Scholar
  8. 8.
    Bradley, M.M., Lang, P.J.: The international affective digitized sounds (2nd edition; IADS-2): Affective ratings of sounds and instruction manual. Tech. rep., University of Florida, Gainesville (2007)Google Scholar
  9. 9.
    Canli, T., Desmond, J.E., Zhao, Z., Glover, G., Gabrieli, J.D.E.: Hemispheric asymmetry for emotional stimuli detected with fMRI. NeuroReport 9(14), 3233–3239 (1998)CrossRefGoogle Scholar
  10. 10.
    Chanel, G.: Emotion assessment for affective-computing based on brain and peripheral signals. Ph.D. thesis, University of Geneva, Geneva (2009)Google Scholar
  11. 11.
    Chanel, G., Kierkels, J.J.M., Soleymani, M., Pun, T.: Short-term emotion assessment in a recall paradigm. International Journal of Human Computer Studies 67(8), 607–627 (2009)CrossRefGoogle Scholar
  12. 12.
    Chanel, G., Kronegg, J., Grandjean, D., Pun, T.: Emotion assessment: Arousal evaluation using EEG’s and peripheral physiological signals (2006)Google Scholar
  13. 13.
    Ekman, P.: Basic emotions. In: Dalgleish, T., Power, M. (eds.) Handbook of Cognition and Emotion. Wiley, New York (1999)Google Scholar
  14. 14.
    Grocke, D.E., Wigram, T.: Receptive Methods in Music Therapy: Techniques and Clinical Applications for Music Therapy Clinicians, Educators and Students, 1st edn. Jessica Kingsley Publishers (2007)Google Scholar
  15. 15.
    Guetin, S., Portet, F., Picot, M.C., Defez, C., Pose, C., Blayac, J.P., Touchon, J.: Impact of music therapy on anxiety and depression for patients with alzheimer’s disease and on the burden felt by the main caregiver (feasibility study). Interets de la musicotherapie sur l’anxiete, la depression des patients atteints de la maladie d’Alzheimer et sur la charge ressentie par l’accompagnant principal 35(1), 57–65 (2009)Google Scholar
  16. 16.
    Hamann, S., Canli, T.: Individual differences in emotion processing. Current Opinion in Neurobiology 14(2), 233–238 (2004)CrossRefGoogle Scholar
  17. 17.
    Higuchi, T.: Approach to an irregular time series on the basis of the fractal theory. Physica D: Nonlinear Phenomena 31(2), 277–283 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Horlings, R.: Emotion recognition using brain activity. Ph.D. thesis, Delft University of Technology (2008)Google Scholar
  19. 19.
    IDM-Project: Emotion-based personalized digital media experience in co-spaces (2008),
  20. 20.
    James, W.: What is an emotion. Mind 9(34), 188–205 (1984)Google Scholar
  21. 21.
    Jones, N.A., Fox, N.A.: Electroencephalogram asymmetry during emotionally evocative films and its relation to positive and negative affectivity. Brain and Cognition 20(2), 280–299 (1992)CrossRefGoogle Scholar
  22. 22.
    Khalili, Z., Moradi, M.H.: Emotion recognition system using brain and peripheral signals: Using correlation dimension to improve the results of eeg. In: Proceedings of the International Joint Conference on Neural Networks, pp. 1571–1575 (2009)Google Scholar
  23. 23.
    Kulish, V., Sourin, A., Sourina, O.: Analysis and visualization of human electroencephalograms seen as fractal time series. Journal of Mechanics in Medicine and Biology 26(2), 175–188 (2006)CrossRefGoogle Scholar
  24. 24.
    Kulish, V., Sourin, A., Sourina, O.: Human electroencephalograms seen as fractal time series: Mathematical analysis and visualization. Computers in Biology and Medicine 36(3), 291–302 (2006)CrossRefGoogle Scholar
  25. 25.
    Lane, R.D., Reiman, E.M., Bradley, M.M., Lang, P.J., Ahern, G.L., Davidson, R.J., Schwartz, G.E.: Neuroanatomical correlates of pleasant and unpleasant emotion. Neuropsychologia 35(11), 1437–1444 (1997)CrossRefGoogle Scholar
  26. 26.
    Lang, P., Bradley, M., Cuthbert, B.: International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Tech. rep., University of Florida, Gainesville, FL (2008)Google Scholar
  27. 27.
    Li, M., Chai, Q., Kaixiang, T., Wahab, A.: EEG emotion recognition system. In: In-Vehicle Corpus and Signal Processing for Driver Behavior, pp. 125–135. Springer, US (2009)CrossRefGoogle Scholar
  28. 28.
    Lin, Y.P., Wang, C.H., Wu, T.L., Jeng, S.K., Chen, J.H.: EEG-based emotion recognition in music listening: A comparison of schemes for multiclass support vector machine. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, Taipei, pp. 489–492 (2009)Google Scholar
  29. 29.
    Liu, Y., Sourina, O., Nguyen, M.K.: Real-time EEG-based human emotion recognition and visualization. In: Proc. 2010 Int. Conf. on Cyberworlds, Singapore, pp. 262–269 (2010)Google Scholar
  30. 30.
    Lutzenberger, W., Elbert, T., Birbaumer, N., Ray, W.J., Schupp, H.: The scalp distribution of the fractal dimension of the EEG and its variation with mental tasks. Brain Topography 5(1), 27–34 (1992)CrossRefGoogle Scholar
  31. 31.
    Maragos, P., Sun, F.-K.: Measuring the fractal dimension of signals: morphological covers and iterative optimization. IEEE Transactions on Signal Processing 41(1), 108–121 (1993)CrossRefzbMATHGoogle Scholar
  32. 32.
    Mauss, I.B., Robinson, M.D.: Measures of emotion: A review. Cognition and Emotion 23(2), 209–237 (2009)CrossRefGoogle Scholar
  33. 33.
    Murugappan, M., Rizon, M., Nagarajan, R., Yaacob, S., Zunaidi, I., Hazry, D.: Lifting scheme for human emotion recognition using EEG. In: International Symposium on Information Technology, ITSim 2008, vol. 2 (2008)Google Scholar
  34. 34.
    Pardo, J.V., Pardo, P.J., Raichle, M.E.: Neural correlates of self-induced dysphoria. American Journal of Psychiatry 150(5), 713–719 (1993)CrossRefGoogle Scholar
  35. 35.
    Petrantonakis, P.C., Hadjileontiadis, L.J.: Emotion recognition from EEG using higher order crossings. IEEE Transactions on Information Technology in Biomedicine 14(2), 186–197 (2010)CrossRefGoogle Scholar
  36. 36.
    Plutchik, R.: Emotions and life: perspectives from psychology, biology, and evolution, 1st edn. American Psychological Association, Washington, DC (2003)Google Scholar
  37. 37.
    Pradhan, N., Narayana Dutt, D.: Use of running fractal dimension for the analysis of changing patterns in electroencephalograms. Computers in Biology and Medicine 23(5), 381–388 (1993)Google Scholar
  38. 38.
    Russell, J.A.: Affective space is bipolar. Journal of Personality and Social Psychology 37(3), 345–356 (1979)CrossRefGoogle Scholar
  39. 39.
    Russell, J.A.: A circumplex model of affect. Journal of Personality and Social Psychology 39(6), 1161–1178 (1980)CrossRefGoogle Scholar
  40. 40.
    Sammler, D., Grigutsch, M., Fritz, T., Koelsch, S.: Music and emotion: Electrophysiological correlates of the processing of pleasant and unpleasant music. Psychophysiology 44(2), 293–304 (2007)CrossRefGoogle Scholar
  41. 41.
    Sanei, S., Chambers, J.: EEG signal processing. John Wiley & Sons, Chichester (2007)CrossRefGoogle Scholar
  42. 42.
    Savran, A., Ciftci, K., Chanel, G., Mota, J., Viet, L., Sankur, B., Akarun, L., Caplier, A., Rombaut, M.: Emotion detection in the loop from brain signals and facial images (2006),
  43. 43.
    Schaaff, K.: EEG-based emotion recognition. Ph.D. thesis, Universitat Karlsruhe (TH) (2008)Google Scholar
  44. 44.
    Sourina, O., Kulish, V.V., Sourin, A.: Novel tools for quantification of brain responses to music stimuli. In: Proc. of 13th International Conference on Biomedical Engineering, ICBME 2008, pp. 411–414 (2008)Google Scholar
  45. 45.
    Sourina, O., Liu, Y.: A fractal-based algorithm of emotion recognition from EEG using arousal-valence model. In: Biosignals 2011, Rome, Italy (accepted, 2011)Google Scholar
  46. 46.
    Sourina, O., Sourin, A., Kulish, V.: EEG data driven animation and its application. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2009. LNCS, vol. 5496, pp. 380–388. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  47. 47.
    Sourina, O., Wang, Q., Liu, Y., Nguyen, M.K.: A real-time fracal-based brain state recognition from EEG and its application. In: Biosignals 2011, Rome, Italy (accepted, 2011)Google Scholar
  48. 48.
    Sourina, O., Wang, Q., Nguyen, M.K.: EEG-based ”serious” games and monitoring tools for pain management. In: Proc. MMVR18, Newport Beach, California (accepted, 2011)Google Scholar
  49. 49.
    Stam, C.J.: Nonlinear dynamical analysis of EEG and MEG: Review of an emerging field. Clinical Neurophysiology 116(10), 2266–2301 (2005)CrossRefGoogle Scholar
  50. 50.
    Stevens, K.: Patients’ perceptions of music during surgery. Journal of Advanced Nursing 15(9), 1045–1051 (1990)CrossRefGoogle Scholar
  51. 51.
    Takahashi, K.: Remarks on emotion recognition from multi-modal bio-potential signals. In: IEEE International Conference on Industrial Technology, IEEE ICIT 2004, vol. 3, pp. 1138–1143 (2004)Google Scholar
  52. 52.
    Wang, Q., Sourina, O., Nguyen, M.K.: EEG-based ”serious” games design for medical applications. In: Proc. 2010 Int. Conf. on Cyberworlds, Singapore, pp. 270–276 (2010)Google Scholar
  53. 53.
    Wang, Q., Sourina, O., Nguyen, M.K.: Fractal dimension based algorithm for neurofeedback games. In: Proc. CGI 2010, Singapore, p. SP25 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Yisi Liu
    • 1
  • Olga Sourina
    • 1
  • Minh Khoa Nguyen
    • 1
  1. 1.Nanyang Technological UniversitySingapore

Personalised recommendations