Advertisement

EEG-Based Brain-Computer Interface for Emotional Involvement in Games Through Music

  • Raffaella Folgieri
  • Mattia G. Bergomi
  • Simone Castellani
Chapter

Abstract

The reliability of commercial non-invasive BCI (Brain Computer Interface) devices and the lower cost of these EEG-based systems, determined the increasing interest in their application in different research fields, also thanks to the portability of the equipment. The latter feature makes BCI devices particularly suited for entertainment applications especially due to the possibility to detect the mental state of the users. The relationship between emotions and entertainment is obvious, as the influence of music in human emotional states. While BCI devices represent a challenge in gaming motion control, they have been successfully applied in music production (Dan et al., PloS ONE, 2009) and composition (Hamadicharef et al., CW2010 282–286, 2010). In our previous work (Folgieri et al., Proceedings of the 4th International Conference on Applied Human Factors and Ergonomics, San Francisco, USA) we focused on conscious production of music notes with the aim of developing a prototype for applications in entertainment. In this work we trace the state-of-the art of our research and present our opinion on possible applications of the preliminary obtained results.

Keywords

Independent Component Analysis Blind Source Separation Brain Computer Interface Recurrence Plot International Affective Picture System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Allison BZ, Wolpaw EW, Wolpaw JR (2007) Brain-computer interface systems: progress and prospects. Expert Rev Med Devices 4(4):463–474CrossRefGoogle Scholar
  2. Ansari-Asl K, Chanel G, Pun T (2007) A channel selection method for EEG classification in emotion assessment based on synchronization likelihood. In: Eusipco 2007, 15th European signal processing conference, Poznan, Poland, 2007Google Scholar
  3. bin Yunus J et al (2012) The effect of noise removing on emotional classification. In: IEEE Computer & Information Science (ICCIS), 2012 International Conference on, vol 1, pp 485–489Google Scholar
  4. Brown JC (1991) Calculation of a constant q spectral transform. J Acoust Soc Am 89:425CrossRefGoogle Scholar
  5. Call of Duty. Retrieved 23rd February, 2014, from http://www.callofduty.com
  6. Choi S, Cichocki A, Park HM, Lee SY (2005). Blind source separation and independent component analysis: A review. Neural Information Processing-Letters and Reviews, 6(1), 1–57.Google Scholar
  7. Comon P (1994) Independent component analysis, a new concept? Signal Process 36(3):287–314Google Scholar
  8. Dan W, Chao-Yi L, De-Zhong Y (2009) Scale-free music of the brain. PloS ONEGoogle Scholar
  9. Davidson RJ, Schwartz GE, Saron C, Bennett J, Goleman DJ (1979) Frontal versus parietal EEG asymmetry during positive and negative affect [Abstract]. Psychophysiology 16:202–203Google Scholar
  10. Szibbo D, Luo A, Sullivan TJ (2012) Removal of blink artifacts in single channel EEG. In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE:3511-3514. IEEE.Google Scholar
  11. Eckmann J-P, Olifson Kamphorst S, Ruelle D (1987) Recurrence plots of dynamical systems. Europhys Lett 4(9):973–977CrossRefGoogle Scholar
  12. Emotive System Inc. Retrieved 23rd February, 2014, from http://www.emotiv.com
  13. Folgieri, R, Zichella M (2012) Conscious and unconscious music from the brain: design and development of a tool translating brainwaves into music using a BCI device. In: Proceedings of the 4th International conference on applied human factors and Ergonomics, San Francisco, USA, CRC press.Google Scholar
  14. Gaetano Valenza AL, Scilingo EP (2012) The role of nonlinear dynamics in affective valence and arousal recognition. IEEE Trans Affect Comput 3(2):237–249CrossRefGoogle Scholar
  15. Garcés Correa A, Laciar E, Patiño HD, Valentinuzzi ME (2007) Artifact removal from EEG signals using adaptive filters in cascade. J Phys: Conf Ser 90:012081 (IOP Publishing)Google Scholar
  16. Hamadicharef B et al (2010) Brain-Computer Interface (BCI) based musical composition. In: CW2010, pp 282–286Google Scholar
  17. Hamilton JD (1994) Time series analysis, vol. 2. Princeton University Press, PrincetonGoogle Scholar
  18. Huang D, Zhang H, Ang K, Guan C, Pan Y, Wang C, Yu J (2012, March). Fast emotion detection from EEG using asymmetric spatial filtering. In Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, IEEE, pp. 589–592Google Scholar
  19. Hyvärinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural networks 13(4):411–430Google Scholar
  20. Jackson DC, Mueller MJ, Dolski I, Dalton KM, Nitschke JB, Urry HL, Rosenkranz MA, Ryff CD, Singer BH, Davidson RJ (2003) Now you feel it, now you don’t: frontal brain electrical asymmetry and individual differences in emotion regulation. Psychol Sci 14:612–617CrossRefGoogle Scholar
  21. Keil A, Müller M, Gruber T, Wienbruch C, Stolarova M, Elbert T (2001) Effects of emotional arousal in the cerebral hemispheres: a study of oscillatory brain activity and event-related potentials. Clin Neurophysiol 112(11):2057–2068CrossRefGoogle Scholar
  22. Kirmizialsan E, Bayraktaroglu Z, Gurvit H, Keskin Y, Emre M, Demiralp T (2006) Comparative analysis of event-related potentials during Go/NoGo and CPT: decomposition of electrophysiological markers of response inhibition and sustained attention. Brain Res 1104(1):114–128. doi:10.1016/j.brainres.2006.03.010CrossRefGoogle Scholar
  23. Ko M, Bae K, Oh G, Ryu T (2009) A study on new gameplay based on brain-computer interface. In: Barry A, Helen K, Tanya K (eds.) Breaking new ground: innovation in games, play, practice and theory. Proceedings of the 2009 digital games research association conference, Brunel UniversityGoogle Scholar
  24. Lalor EC, Kelly SP, Finucane C, Burke R, Smith R, Reilly RB, McDarby G (2005) Steady-state VEP-based brain-computer interface control in an immersive 3D gaming environment. EURASIP journal on applied signal processing, 2005, 3156–3164Google Scholar
  25. Laurier C, Sordo M, Serra J, Herrera P (2009) Music mood representations from social tags. In: Proceedings of international society for music information retrieval conference (ISMIR), Kobe, JapanGoogle Scholar
  26. Lee TW (1998). Independent component analysis (pp. 27–66). Springer US.Google Scholar
  27. Lu D, Liu L, Zhang H (January 2006) Automatic mood detection and tracking of music audio signals. In IEEE transactions on audio, speech and language processing, vol 14, no 1, pp 5–18Google Scholar
  28. Metal Gear Solid. Retrieved 23rd February, 2014, from http://www.konami.jp/kojima_pro/english/index.html
  29. Morris JD (1995) SAM: the self-assessment Manikin, an efficient cross-cultural measurement of emotional response. J Advertising ResGoogle Scholar
  30. Mühl C, Gürkök H, Plass-Oude Bos D, Thurlings ME, Scherffig L, Duvinage M, Elbakyan AA, Kang S, Poel M, Heylen DKJ (2010) Bacteria hunt: a multimodal, multiparadigm BCI game. In: Proceedings of the international summer workshop on multimodal interfaces, GenuaGoogle Scholar
  31. Müller M, Keil A, Gruber T, Elbert T (1999) Processing of affective pictures modulates right-hemispheric gamma band EEG activity. Clin Neurophysiol 110(11):1913–1920CrossRefGoogle Scholar
  32. Murugappan M, Nagarajan R, Yaacob S (2009) Appraising human emotions using time frequency analysis based EEG alpha band features. pp 70–75Google Scholar
  33. NeuroSky Inc. Retrieved 23rd February, 2014, from http://neurosky.biz
  34. Petrantonakis PC, Hadjileontiadis LJ (2012) Adaptive emotional information retrieval from EEG signals in the time-frequency domain. IEEE Trans Signal Process 60(5):2604–2616CrossRefMathSciNetGoogle Scholar
  35. Pineda JA, Silverman DS, Vankov A, Hestenes J (2003) Learning to control brain rhythms: making a brain-computer interface possible. Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 11(2), 181–184Google Scholar
  36. Plass-Oude Bos D, Reuderink B, Laar B, Gürkök H, Mühl C, Poel M, Nijholt A, Heylen D (2010) Brain-computer interfacing and games. In: Tan DS, Nijholt A (eds) Brain-computer interfaces, series human-computer interaction seriesGoogle Scholar
  37. Resident Evil. Retrieved 23rd February, 2014, from http://www.capcom.co.jp/bio5/
  38. Russell JA (1980) A circumplex model of affect. J Pers Soc Psychol 39(6):1161–1178CrossRefGoogle Scholar
  39. Ryu CS, Song Y, Kim SH, Yi I, Kim JE, Sohn JH (1998). A time-frequency analysis of the EEG evoked by negative and positive visual stimuli. In Engineering in Medicine and Biology Society, 1998. Proceedings of the 20th Annual International Conference of the IEEE, IEEE, Vol. 4, pp. 2012–2015Google Scholar
  40. Schmidt LA, Fox NA (1999) Conceptual, biological, and behavioral distinctions among different types of shy children. In: Schmidt LA, Schulkin J (eds) Extreme fear, shyness and social phobia: origins, biological mechanisms, and clinical outcomes. Oxford University Press, New York, pp 47–66Google Scholar
  41. Schmidt LA, Trainor LJ (2001) Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cognition Emotion 15:487–500CrossRefGoogle Scholar
  42. Scott M, Anthony JB, Tzyy-Ping J, Terrence JS et al (1996) Independent component analysis of electroencephalographic data. Adv Neur Inform Process Syst :145–151Google Scholar
  43. Silent Hill. Retrieved 23rd February, 2014, from http://www.konami.com/games/shh/
  44. Sniper Elite. Retrieved 23rd February, 2014, from http://www.microids.com/en/catalogue/28/sniper-elite-berlin-1945.html
  45. Stone JV (2005) Independent Component Analysis. Encyclopedia of Statistics in Behavioral Science. Wiley Online LibraryGoogle Scholar
  46. Tomarken AJ, Davidson RJ, Wheeler RE, Doss RC (1992) Individual differences in anterior brain asymmetry and fundamental dimensions of emotions. J Pers Soc Psychol 62:676–678CrossRefGoogle Scholar
  47. Tseng KC, Wang YT, Lin BS, Hsieh PH (2012, July). Brain Computer Interface-based Multimedia Controller. In Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP), 2012 Eighth International Conference on, IEEE, pp. 277–280Google Scholar
  48. Zhou W, Gotman J (2009) Automatic removal of eye movement artifacts from the eeg using ica and the dipole model. Prog Nat Sci 19(9):1165–1170CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Raffaella Folgieri
    • 1
  • Mattia G. Bergomi
    • 2
    • 3
  • Simone Castellani
    • 4
  1. 1.DEMM, Dipartimento di Economia, Management e Metodi quantitativiUniversità degli Studi di MilanoMilanoItaly
  2. 2.Dipartimento di InformaticaUniversità degli Studi di MilanoMilanoItaly
  3. 3.IrcamUniversité Pierre et Marie CurieParisFrance
  4. 4.CdL Informatica per la ComunicazioneUniversità degli Studi di MilanoMilanoItaly

Personalised recommendations