Advertisement

Cognitive Neurodynamics

, Volume 13, Issue 1, pp 13–31 | Cite as

Music of brain and music on brain: a novel EEG sonification approach

  • Shankha SanyalEmail author
  • Sayan Nag
  • Archi Banerjee
  • Ranjan Sengupta
  • Dipak Ghosh
Research Article

Abstract

Can we hear the sound of our brain? Is there any technique which can enable us to hear the neuro-electrical impulses originating from the different lobes of brain? The answer to all these questions is YES. In this paper we present a novel method with which we can sonify the electroencephalogram (EEG) data recorded in “control” state as well as under the influence of a simple acoustical stimuli—a tanpura drone. The tanpura has a very simple construction yet the tanpura drone exhibits very complex acoustic features, which is generally used for creation of an ambience during a musical performance. Hence, for this pilot project we chose to study the nonlinear correlations between musical stimulus (tanpura drone as well as music clips) and sonified EEG data. Till date, there have been no study which deals with the direct correlation between a bio-signal and its acoustic counterpart and also tries to see how that correlation varies under the influence of different types of stimuli. This study tries to bridge this gap and looks for a direct correlation between music signal and EEG data using a robust mathematical microscope called Multifractal Detrended Cross Correlation Analysis (MFDXA). For this, we took EEG data of 10 participants in 2 min “control condition” (i.e. with white noise) and in 2 min ‘tanpura drone’ (musical stimulus) listening condition. The same experimental paradigm was repeated for two emotional music, “Chayanat” and “Darbari Kanada”. These are well known Hindustani classical ragas which conventionally portray contrast emotional attributes, also verified from human response data. Next, the EEG signals from different electrodes were sonified and MFDXA technique was used to assess the degree of correlation (or the cross correlation coefficient γx) between the EEG signals and the music clips. The variation of γx for different lobes of brain during the course of the experiment provides interesting new information regarding the extraordinary ability of music stimuli to engage several areas of the brain significantly unlike any other stimuli (which engages specific domains only).

Keywords

EEG Sonification Tanpura drone Hindustani Ragas MFDXA Cross-correlation coefficient 

Notes

Acknowledgement

The first author, SS acknowledges the Council of Scientific and Industrial Research (CSIR), Govt. of India for providing the Senior Research Fellowship (SRF) to pursue this research (09/096(0876)/2017-EMR-I).One of the authors, AB acknowledges the Department of Science and Technology (DST), Govt. of India for providing (A.20020/11/97-IFD) the DST Inspire Fellowship to pursue this research work. All the authors acknowledge Department of Science and Technology, Govt. of West Bengal for providing the RMS EEG equipment as part of R&D Project (3/2014).

References

  1. Adrian ED, Matthews BH (1934) The Berger rhythm: potential changes from the occipital lobes in man. Brain 57(4):355–385CrossRefGoogle Scholar
  2. Akin M, Arserim MA, Kiymik MK, Turkoglu I (2001) A new approach for diagnosing epilepsy by using wavelet transform and neural networks. In: Engineering in medicine and biology society, 2001. Proceedings of the 23rd annual international conference of the IEEE, vol 2. IEEE, pp 1596–1599Google Scholar
  3. Arslan B, Brouse A, Castet J, Filatriau JJ, Lehembre R, Noirhomme Q, Simon C (2005) Biologically-driven musical instrument. In: Proceedings of the summer workshop on multimodal interfaces (eNTERFACE’05). Faculté Polytechnique de Mons, Mons, BLGoogle Scholar
  4. Babiloni C, Buffo P, Vecchio F, Marzano N, Del Percio C, Spada D, Rossi S, Bruni I, Rossini PM, Perani D (2012) Brains “in concert”: frontal oscillatory alpha rhythms and empathy in professional musicians. Neuroimage 60(1):105–116CrossRefGoogle Scholar
  5. Baier G, Hermann T, Stephani U (2007) Event-based sonification of EEG rhythms in real time. Clin Neurophysiol 118(6):1377–1386CrossRefGoogle Scholar
  6. Balkwill LL, Thompson WF (1999) A cross-cultural investigation of the perception of emotion in music: psychophysical and cultural cues. Music Percept Interdiscip J 17(1):43–64CrossRefGoogle Scholar
  7. Banerjee A, Sanyal S, Patranabis A, Banerjee K, Guhathakurta T, Sengupta R, Ghose D, Ghose P (2016) Study on brain dynamics by non linear analysis of music induced EEG signals. Physica A 444:110–120CrossRefGoogle Scholar
  8. Behrman A (1999) Global and local dimensions of vocal dynamics. J Acoust Soc Am 105:432–443CrossRefGoogle Scholar
  9. Bhattacharya J (2009) Increase of universality in human brain during mental imagery from visual perception. PLoS ONE 4(1):e4121CrossRefGoogle Scholar
  10. Bigerelle M, Iost A (2000) Fractal dimension and classification of music. Chaos Solitons Fract 11(14):2179–2192CrossRefGoogle Scholar
  11. Bornas X, Fiol-Veny A, Balle M, Morillas-Romero A, Tortella-Feliu M (2015) Long range temporal correlations in EEG oscillations of subclinically depressed individuals: their association with brooding and suppression. Cogn Neurodyn 9(1):53–62CrossRefGoogle Scholar
  12. Braeunig M, Sengupta R, Patranabis A (2012) On tanpura drone and brain electrical correlates. In: Ystad S, Aramaki M, Kronland-Martinet R, Jensen K, Mohanty S (eds) Speech, sound and music processing: embracing research in India. pp 53–65Google Scholar
  13. Chordia P, Rae A (2007) Understanding emotion in raag: an empirical study of listener responses. In: International symposium on computer music modeling and retrieval. Springer, Berlin, pp 110–124Google Scholar
  14. Dutta S, Ghosh D, Chatterjee S (2013) Multifractal detrended fluctuation analysis of human gait diseases. Fron Physiol 4:274Google Scholar
  15. Dutta S, Ghosh D, Samanta S, Dey S (2014) Multifractal parameters as an indication of different physiological and pathological states of the human brain. Phys A: Stat Mech Appl 396:155–163CrossRefGoogle Scholar
  16. Elgendi M, Rebsamen B, Cichocki A, Vialatte F, Dauwels J (2013) Real-time wireless sonification of brain signals. In: Yamaguchi Y (ed) Advances in cognitive neurodynamics (III). Springer, Dordrecht, pp 175–181CrossRefGoogle Scholar
  17. Figliola A, Serrano E, Rosso OA (2007) Multifractal detrented fluctuation analysis of tonic-clonic epileptic seizures. European Phys J Spec Topics 143(1):117–123CrossRefGoogle Scholar
  18. Gao TT, Wu D, Huang YL, Yao DZ (2007) Detrended fluctuation analysis of the human EEG during listening to emotional music. J Electron Sci Tech 5(3):272–277Google Scholar
  19. Gao J, Hu J, Tung WW (2011) Complexity measures of brain wave dynamics. Cogn Neurodyn 5(2):171–182CrossRefGoogle Scholar
  20. Ghosh M (2002) Natyashastra (ascribed to Bharata Muni). Chowkhamba Sanskrit Series Office, VaranasiGoogle Scholar
  21. Ghosh D, Dutta S, Chakraborty S (2014) Multifractal detrended cross-correlation analysis for epileptic patient in seizure and seizure free status. Chaos Solitons Fractals 67:1–10CrossRefGoogle Scholar
  22. Ghosh D, Dutta S, Chakraborty S (2015) Multifractal detrended cross-correlation analysis of market clearing price of electricity and SENSEX in India. Physica A 434:52–59CrossRefGoogle Scholar
  23. Ghosh D, Sengupta R, Sanyal S, Banerjee A (2018a) Emotions from Hindustani classical music: an EEG based study including neural hysteresis. In: Baumann C (ed) Musicality of human brain through fractal analytics. Springer, Singapore, pp 49–72CrossRefGoogle Scholar
  24. Ghosh D, Sengupta R, Sanyal S, Banerjee A (2018b) Musical perception and visual imagery: do musicians visualize while performing?. In: Baumann C (ed) Musicality of human brain through fractal analytics. Springer, Singapore, pp 73–102CrossRefGoogle Scholar
  25. Glen J (2010) Use of audio signals derived from electroencephalographic recordings as a novel ‘depth of anaesthesia’monitor. Med Hypotheses 75(6):547–549CrossRefGoogle Scholar
  26. Hardstone R, Poil SS, Schiavone G, Jansen R, Nikulin VV, Mansvelder HD, Linkenkaer-Hansen K (2012) Detrended fluctuation analysis: a scale-free view on neuronal oscillations. Front Physiol 3:450CrossRefGoogle Scholar
  27. Hermann T (2008) Taxonomy and definitions for sonification and auditory display. In: Proceedings of the 14th international conference on auditory display (ICAD 2008)Google Scholar
  28. Hermann T, Meinicke P, Bekel H, Ritter H, Müller HM, Weiss S (2002) Sonification for eeg data analysis. In: Proceedings of the 2002 international conference on auditory displayGoogle Scholar
  29. Hinterberger T, Hill J, Birbaumer N (2013) An auditory brain-computer communication device. In: proceedings IEEE BIOCAS’04, 2004. The 19th international conference on auditory display (ICAD-2013), Lodz, PolandGoogle Scholar
  30. Hsü KJ, Hsü AJ (1990) Fractal geometry of music. Proc Natl Acad Sci 87(3):938–941CrossRefGoogle Scholar
  31. John TN, Puthankattil SD, Menon R (2018) Analysis of long range dependence in the EEG signals of Alzheimer patients. Cogn Neurodyn 12(2):183–199CrossRefGoogle Scholar
  32. Kantelhardt JW, Zschiegner SA, Koscielny-Bunde E, Havlin S, Bunde A, Stanley HE (2002) Multifractal detrended fluctuation analysis of nonstationary time series. Physica A 316(1):87–114CrossRefGoogle Scholar
  33. Kantelhardt JW, Rybski D, Zschiegner SA, Braun P, Bunde EK, Livina V et al (2003) Multifractality of river runoff and precipitation: comparison of fluctuation analysis and wavelet methods. Phys A 330:240–245.  https://doi.org/10.1016/j.physa.2003.08.019 CrossRefGoogle Scholar
  34. Karkare S, Saha G, Bhattacharya J (2009) Investigating long-range correlation properties in EEG during complex cognitive tasks. Chaos Solitons Fract 42(4):2067–2073CrossRefGoogle Scholar
  35. Khamis H, Mohamed A, Simpson S, McEwan A (2012) Detection of temporal lobe seizures and identification of lateralisation from audified EEG. Clin Neurophysiol 123(9):1714–1720CrossRefGoogle Scholar
  36. Koelsch S, Fritz T, Müller K, Friederici AD (2006) Investigating emotion with music: an fMRI study. Hum Brain Mapp 27(3):239–250CrossRefGoogle Scholar
  37. Kramer G (ed) (1994) Some organizing principles for representing data with sound. In: Auditory display-sonification, audification, and auditory interfaces. Reading, MA, Addison-Wesley, pp 185–221Google Scholar
  38. Kramer G, Walker B, Bonebright T, Cook P, Flowers JH, Miner N, Neuhoff J (2010) Sonification report: status of the field and research agendaGoogle Scholar
  39. Kumar A, Mullick SK (1996) Nonlinear dynamical analysis of speech. J Acoust Soc Am 100(1):615–629CrossRefGoogle Scholar
  40. Lin YP, Wang CH, Jung TP, Wu TL, Jeng SK, Duann JR, Chen JH (2010) EEG-based emotion recognition in music listening. IEEE Trans Biomed Eng 57(7):1798–1806CrossRefGoogle Scholar
  41. Lu J, Wu D, Yang H, Luo C, Li C, Yao D (2012) Scale-free brain-wave music from simultaneously EEG and fMRI recordings. PLoS ONE 7(11):e49773CrossRefGoogle Scholar
  42. Lu J, Guo S, Chen M, Wang W, Yang H, Guo D, Yao D (2018) Generate the scale-free brain music from BOLD signals. Medicine 97(2):e9628CrossRefGoogle Scholar
  43. Maity AK, Pratihar R, Mitra A, Dey S, Agrawal V, Sanyal S, Banerjee A, Ghosh D, Sengupta R (2015) Multifractal detrended fluctuation analysis of alpha and theta EEG rhythms with musical stimuli. Chaos Solitons Fract 81:52–67CrossRefGoogle Scholar
  44. Mathur A, Vijayakumar SH, Chakrabarti B, Singh NC (2015) Emotional responses to Hindustani raga music: the role of musical structure. Front Psychol 6:513Google Scholar
  45. McCreadie KA, Coyle DH, Prasad G (2013) Sensorimotor learning with stereo auditory feedback for a brain–computer interface. Med Biol Eng Compu 51(3):285–293CrossRefGoogle Scholar
  46. Meinicke P, Hermann T, Bekel H, Müller HM, Weiss S, Ritter H (2004) Identification of discriminative features in the EEG. Intell Data Anal 8(1):97–107CrossRefGoogle Scholar
  47. Miranda ER, Brouse A (2005) Interfacing the brain directly with musical systems: on developing systems for making music with brain signals. Leonardo 38(4):331–336CrossRefGoogle Scholar
  48. Miranda ER, Castet J (eds) (2014) Guide to brain-computer music interfacing. SpringerGoogle Scholar
  49. Miranda ER, Magee WL, Wilson JJ, Eaton J, Palaniappan R (2011) Brain-computer music interfacing (BCMI): from basic research to the real world of special needs. Music Med 3(3):134–140CrossRefGoogle Scholar
  50. Movahed MS, Hermanis E (2008) Fractal analysis of river flow fluctuations. Physica A 387(4):915–932CrossRefGoogle Scholar
  51. Olivan J, Kemp B, Roessen M (2004) Easy listening to sleep recordings: tools and examples. Sleep Med 5(6):601–603CrossRefGoogle Scholar
  52. Podobnik B, Stanley HE (2008) Detrended cross-correlation analysis: a new method for analyzing two nonstationary time series. Phys Rev Lett 100(8):084102CrossRefGoogle Scholar
  53. Podobnik B, Jiang ZQ, Zhou WX, Stanley HE (2011) Statistical tests for power-law cross-correlated processes. Phys Rev E 84(6):066118CrossRefGoogle Scholar
  54. Pressing Jeff (1988) Nonlinear maps as generators of musical design. Comput Music J 12(2):35–46CrossRefGoogle Scholar
  55. Pu J, Xu H, Wang Y, Cui H, Hu Y (2016) Combined nonlinear metrics to evaluate spontaneous EEG recordings from chronic spinal cord injury in a rat model: a pilot study. Cogn Neurodyn 10(5):367–373CrossRefGoogle Scholar
  56. Rosenboom D (1999) Extended musical interface with the human nervous system: assessment and prospectus. Leonardo 32(4):257–257.  https://doi.org/10.1162/002409499553398 CrossRefGoogle Scholar
  57. Sadegh Movahed M, Jafari GR, Ghasemi F, Rahvar S, Reza Rahimi TM (2006) Multifractal detrended fluctuation analysis of sunspot time series. J Stat Mech 0602:P02003.  https://doi.org/10.1088/1742-5468/2006/02/P0200 Google Scholar
  58. Sammler D, Grigutsch M, Fritz T, Koelsch S (2007) Music and emotion: electrophysiological correlates of the processing of pleasant and unpleasant music. Psychophysiology 44(2):293–304CrossRefGoogle Scholar
  59. Sengupta R, Dey N, Nag D, Datta AK (2001) Comparative study of fractal behavior in quasi-random and quasi-periodic speech wave map. Fractals 9(04):403–414CrossRefGoogle Scholar
  60. Sengupta R, Dey N, Datta AK, Ghosh D (2005) Assessment of musical quality of tanpura by fractal-dimensional analysis. Fractals 13(03):245–252CrossRefGoogle Scholar
  61. Sengupta R, Dey N, Datta AK, Ghosh D, Patranabis A (2010) Analysis of the signal complexity in sitar performances. Fractals 18(02):265–270CrossRefGoogle Scholar
  62. Sjőlander K, Beskow J (2009) Wavesurfer. Computer program, version, 1(3)Google Scholar
  63. Supper A (2012) The search for the “killer application”: drawing the boundaries around the sonification of scientific data. In: The oxford handbook of sound studiesGoogle Scholar
  64. Telesca L, Lapenna V, Macchiato M (2004) Mono- and multi-fractal investigation of scaling properties in temporal patterns of seismic sequences. Chaos Soliton Fract 19:1–15.  https://doi.org/10.1016/S0960-0779(03)00188-7 CrossRefGoogle Scholar
  65. Truax, B (1990) Chaotic non-linear systems and digital synthesis: an exploratory study. In: International computer music conference (ICMC). Glasgow, Scotland, pp 100–103Google Scholar
  66. Väljamäe A, Steffert T, Holland S, Marimon X, Benitez R, Mealla S et al (2013). A review of real-time EEG sonification research. In: Proceddings of the interntional conference on auditory display, 2013, pp 85–93Google Scholar
  67. Van Leeuwen WS, Bekkering ID (1958) Some results obtained with the EEG-spectrograph. Electroencephalogr Clin Neurophysiol 10(3):563–570CrossRefGoogle Scholar
  68. Wang J, Shang P, Ge W (2012) Multifractal cross-correlation analysis based on statistical moments. Fractals 20(03n04):271–279CrossRefGoogle Scholar
  69. Wang F, Liao GP, Zhou XY, Shi W (2013) Multifractal detrended cross-correlation analysis for power markets. Nonlinear Dyn 72(1–2):353–363CrossRefGoogle Scholar
  70. Wieczorkowska AA, Datta AK, Sengupta R, Dey N, Mukherjee B (2010) On search for emotion in Hindusthani vocal music. In: Raś ZW, Wieczorkowska AA (eds) Advances in music information retrieval. Springer, Berlin, pp 285–304CrossRefGoogle Scholar
  71. Wu D, Li CY, Yao DZ (2009) Scale-free music of the brain. PLoS ONE 4(6):e5915CrossRefGoogle Scholar
  72. Yuvaraj R, Murugappan M (2016) Hemispheric asymmetry non-linear analysis of EEG during emotional responses from idiopathic Parkinson’s disease patients. Cogn Neurodyn 10(3):225–234CrossRefGoogle Scholar
  73. Zhou WX (2008) Multifractal detrended cross-correlation analysis for two non-stationary signals. Phys Rev E 77(6):066211CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  • Shankha Sanyal
    • 1
    • 2
    Email author
  • Sayan Nag
    • 3
  • Archi Banerjee
    • 1
    • 2
  • Ranjan Sengupta
    • 1
  • Dipak Ghosh
    • 1
  1. 1.Sir C.V. Raman Centre for Physics and MusicJadavpur UniversityKolkataIndia
  2. 2.Department of PhysicsJadavpur UniversityKolkataIndia
  3. 3.Department of Electrical EngineeringJadavpur UniversityKolkataIndia

Personalised recommendations