Psychophysiology-Informed (Multimodal) Imaging

  • Nikolaj Bak
  • Bob OranjeEmail author
Part of the Current Topics in Behavioral Neurosciences book series (CTBN, volume 21)


Electroencephalography (EEG) and magnetic resonance imaging are two popular methodologies for brain research. While EEG has a high temporal resolution, yet a low spatial resolution, MRI has the complete opposite, a high spatial resolution, yet a low temporal resolution. Obviously therefore, researchers have been searching for ways combining the two methodologies, for more than two decades. However, there are many issues that have to be solved before the methodologies can be successfully and, more importantly reliably, combined. Here, we give an overview of these issues, and present strategies that have been used over the past two decades to overcome them. We start with a general description of EEG and (f)MRI methodology, then present the difficulties involved in combining both methodologies, and lastly present and discuss the most popular strategies that have been used over the past two decades to solve these problems. We conclude that in spite of the many issues, the two methodologies can be combined successfully, provided that the correct procedures are followed.


EEG (f)MRI P50 suppression Schizophrenia 



The authors have no conflicts of interest.


  1. Aggernaes B, Glenthoj BY, Ebdrup BH, Rasmussen H, Lublin H, Oranje B (2010) Sensorimotor gating and habituation in antipsychotic-naïve, first-episode schizophrenia patients before and after six months treatment with quetiapine. Int J Neuropsychopharmacol 13(10):1383–1395PubMedCrossRefGoogle Scholar
  2. Allen PJ, Josephs O, Turner R (2000) A method for removing imaging artifact from continuous EEG recorded during functional MRI. Neuroimage 12(2):230–239PubMedCrossRefGoogle Scholar
  3. Allen PJ, Polizzi G, Krakow K, Fish DR, Lemieux L (1998) Identification of EEG events in the MR scanner: the problem of pulse artifact and a method for its subtraction. Neuroimage 8(3):229–239PubMedCrossRefGoogle Scholar
  4. Arciniegas DB, Topkoff JL, Rojas DC, Sheeder J, Teale P, Young DA, Sandberg E, Reite ML, Adler LE (2001) Reduced hippocampal volume in association with p50 nonsuppression following traumatic brain injury. J Neuropsychiatry Clin Neurosci 13(2):213–221PubMedCrossRefGoogle Scholar
  5. Bak N, Glenthoj BY, Rostrup E, Larsson H, Oranje B (2011) Source localization of sensory gating: a combined EEG and fMRI study in healthy volunteers. Neuroimage 54(4):2711–2718PubMedCrossRefGoogle Scholar
  6. Bak N, Rostrup E, Larsson HB, Glenthoj BY, Oranje B (2013) Concurrent functional magnetic resonance imaging and electroencephalography assessment of sensory gating in schizophrenia. Hum Brain MappGoogle Scholar
  7. Barry RJ, Clarke AR, Johnstone SJ (2003a) A review of electrophysiology in attention-deficit/hyperactivity disorder: I Qualitative and quantitative electroencephalography. Clin Neurophysiol 114(2):171–183PubMedCrossRefGoogle Scholar
  8. Barry RJ, Johnstone SJ, Clarke AR (2003b) A review of electrophysiology in attention-deficit/hyperactivity disorder: II. Event-related potentials. Clin Neurophysiol 114(2):184–198PubMedCrossRefGoogle Scholar
  9. Benar CG, Schon D, Grimault S, Nazarian B, Burle B, Roth M, Badier JM, Marquis P, Liegeois-Chauvel C, Anton JL (2007) Single-trial analysis of oddball event-related potentials in simultaneous EEG-fMRI. Hum Brain Mapp 28(7):602–613PubMedCrossRefGoogle Scholar
  10. Berger H (1929) Uber das Elektrenkephalogramm des Menschen. Archiv für Psychiatrie und Nervenkrankheiten 87(1):527–570CrossRefGoogle Scholar
  11. Bregadze N, Lavric A (2006) ERP differences with vs. without concurrent fMRI. Int J Psychophysiol 62(1):54–59PubMedCrossRefGoogle Scholar
  12. Coull JT (1998) Neural correlates of attention and arousal: insights from electrophysiology, functional neuroimaging and psychopharmacology. Prog Neurobiol 55(4):343–361PubMedCrossRefGoogle Scholar
  13. Counter SA, Olofsson A, Grahn HF, Borg E (1997) MRI acoustic noise: sound pressure and frequency analysis. J Magn Reson Imaging 7(3):606–611PubMedCrossRefGoogle Scholar
  14. Debener S, Ullsperger M, Siegel M, Engel AK (2006) Single-trial EEG-fMRI reveals the dynamics of cognitive function. Trends Cogn Sci 10(12):558–563PubMedCrossRefGoogle Scholar
  15. Debener S, Ullsperger M, Siegel M, Fiehler K, von Cramon DY, Engel AK (2005) Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J Neurosci 25(50):11730–11737PubMedCrossRefGoogle Scholar
  16. Egan MF, Duncan CC, Suddath RL, Kirch DG, Mirsky AF, Wyatt RJ (1994) Event-related potential abnormalities correlate with structural brain alterations and clinical features in patients with chronic schizophrenia. Schizophr Res 11(3):259–271PubMedCrossRefGoogle Scholar
  17. Eichele T, Calhoun VD, Debener S (2009) Mining EEG-fMRI using independent component analysis. Int J Psychophysiol 73(1):53–61PubMedCentralPubMedCrossRefGoogle Scholar
  18. Fabiani M, Gratton G, Coles MG (2000) Event-related brain potentials. In: Cacioppo JT (ed) Cambridge University Press, New York, pp 53–84Google Scholar
  19. Fuglo D, Pedersen H, Rostrup E, Hansen AE, Larsson HB (2012) Correlation between single-trial visual evoked potentials and the blood oxygenation level dependent response in simultaneously recorded electroencephalography-functional magnetic resonance imaging. Magn Reson Med 68(1):252–260PubMedCrossRefGoogle Scholar
  20. Fusar-Poli P, Crossley N, Woolley J, Carletti F, Perez-Iglesias R, Broome M, Johns L, Tabraham P, Bramon E, McGuire P (2011) White matter alterations related to P300 abnormalities in individuals at high risk for psychosis: an MRI-EEG study. J Psychiatry Neurosci 36(4):239–248PubMedCentralPubMedCrossRefGoogle Scholar
  21. Hammer TB, Oranje B, Skimminge A, Aggernaes B, Ebdrup BH, Glenthoj BY, Baare W (2012). Structural brain correlates of sensorimotor gating in antipsychotic-naïve men with first-episode schizophrenia. J Psychiatry Neurosci 37(4):8Google Scholar
  22. Hazlett EA, Buchsbaum MS, Tang CY, Fleischman MB, Wei TC, Byne W, Haznedar MM (2001) Thalamic activation during an attention-to-prepulse startle modification paradigm: a functional MRI study. Biol Psychiatry 50(4):281–291PubMedCrossRefGoogle Scholar
  23. Hazlett EA, Buchsbaum MS, Zhang J, Newmark RE, Glanton CF, Zelmanova Y, Haznedar MM, Chu KW, Nenadic I, Kemether EM, Tang CY, New AS, Siever LJ (2008) Frontal-striatal-thalamic mediodorsal nucleus dysfunction in schizophrenia-spectrum patients during sensorimotor gating. Neuroimage 42(3):1164–1177PubMedCentralPubMedCrossRefGoogle Scholar
  24. Herrmann CS, Debener S (2008) Simultaneous recording of EEG and BOLD responses: a historical perspective. Int J Psychophysiol 67(3):161–168PubMedCrossRefGoogle Scholar
  25. Huster RJ, Debener S, Eichele T, Herrmann CS (2012) Methods for simultaneous EEG-fMRI: an introductory review. J Neurosci 32(18):6053–6060PubMedCrossRefGoogle Scholar
  26. Im CH, Liu Z, Zhang N, Chen W, He B (2006) Functional cortical source imaging from simultaneously recorded ERP and fMRI. J Neurosci Methods 157(1):118–123PubMedCentralPubMedCrossRefGoogle Scholar
  27. Jezzard P, Matthews PM, Smith SM (2008) Functional MRI. In: Jezzard P, Matthews PM, Smith SM (eds) Oxford University Press, New YorkGoogle Scholar
  28. Jung R, Berger W (1979) Fiftieth anniversary of Hans Berger’s publication of the electroencephalogram. His first records in 1924–1931 (author’s transl). Arch Psychiatr Nervenkr 227(4):279–300PubMedCrossRefGoogle Scholar
  29. Keshavan MS, Reynolds CF, Kupfer DJ (1990) Electroencephalographic sleep in schizophrenia: a critical review. Compr Psychiatry 31(1):34–47PubMedCrossRefGoogle Scholar
  30. Kiloh LG, McComas AJ, Osselton JW (1972) Clinical electroencephalography. In: Kiloh LG, McComas AJ, Osselton JW (eds) vol 3. Butterworth & co, LondonGoogle Scholar
  31. Kooi KA, Tucker RP, Marshall RE (1978) Basic electrical principles-genesis and regulation of electrical activity of the brain. Fundam Electroencephal 2(1–4):3–48Google Scholar
  32. Krakow K, Allen PJ, Symms MR, Lemieux L, Josephs O, Fish DR (2000) EEG recording during fMRI experiments: image quality. Hum Brain Mapp 10(1):10–15PubMedCrossRefGoogle Scholar
  33. Kumari V, Antonova E, Zachariah E, Galea A, Aasen I, Ettinger U, Mitterschiffthaler MT, Sharma T (2005) Structural brain correlates of prepulse inhibition of the acoustic startle response in healthy humans. Neuroimage 26(4):1052–1058PubMedCrossRefGoogle Scholar
  34. Kumari V, Fannon D, Geyer MA, Premkumar P, Antonova E, Simmons A, Kuipers E (2008) Cortical grey matter volume and sensorimotor gating in schizophrenia. Cortex 44(9):1206–1214PubMedCentralPubMedCrossRefGoogle Scholar
  35. Kumari V, Gray JA, Geyer MA, Ffytche D, Soni W, Mitterschiffthaler MT, Vythelingum GN, Simmons A, Williams SC, Sharma T (2003) Neural correlates of tactile prepulse inhibition: a functional MRI study in normal and schizophrenic subjects. Psychiatry Res 122(2):99–113Google Scholar
  36. Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A (2001) Neurophysiological investigation of the basis of the fMRI signal. Nature 412(6843):150–157PubMedCrossRefGoogle Scholar
  37. Logothetis NK, Wandell BA (2004) Interpreting the BOLD signal. Annu Rev Physiol 66:735–769PubMedCrossRefGoogle Scholar
  38. Madsen GF, Bilenberg N, Cantio C, Oranje B (2013) Increased prepulse inhibition and sensitization of the startle reflex in autistic children. Autism Res (Accepted Epub)Google Scholar
  39. Markand ON (1994) Brainstem auditory evoked potentials. J Clin Neurophysiol 11(3):319–342PubMedCrossRefGoogle Scholar
  40. Mathiak K, Ackermann H, Rapp A, Mathiak KA, Shergill S, Riecker A, Kircher TT (2011) Neuromagnetic oscillations and hemodynamic correlates of P50 suppression in schizophrenia. Psychiatry Res 194(1):95–104PubMedCrossRefGoogle Scholar
  41. Mayer AR, Hanlon FM, Franco AR, Teshiba TM, Thoma RJ, Clark VP, Canive JM (2009) The neural networks underlying auditory sensory gating. Neuroimage 44(1):182–189PubMedCentralPubMedCrossRefGoogle Scholar
  42. McCarley RW, Salisbury DF, Hirayasu Y, Yurgelun-Todd DA, Tohen M, Zarate C, Kikinis R, Jolesz FA, Shenton ME (2002) Association between smaller left posterior superior temporal gyrus volume on magnetic resonance imaging and smaller left temporal P300 amplitude in first-episode schizophrenia. Arch Gen Psychiatry 59(4):321–331PubMedCrossRefGoogle Scholar
  43. Mobascher A, Warbrick T, Brinkmeyer J, Musso F, Stoecker T, Jon SN, Winterer G (2012) Nicotine effects on anterior cingulate cortex in schizophrenia and healthy smokers as revealed by EEG-informed fMRI. Psychiatry Res 204(2–3):168–177PubMedCrossRefGoogle Scholar
  44. Mulert C, Jager L, Schmitt R, Bussfeld P, Pogarell O, Moller HJ, Juckel G, Hegerl U (2004) Integration of fMRI and simultaneous EEG: towards a comprehensive understanding of localization and time-course of brain activity in target detection. Neuroimage 22(1):83–94PubMedCrossRefGoogle Scholar
  45. Mullinger K, Debener S, Coxon R, Bowtell R (2008) Effects of simultaneous EEG recording on MRI data quality at 1.5, 3 and 7 tesla. Int J Psychophysiol 67(3):178–188PubMedCrossRefGoogle Scholar
  46. Näätänen R (1990) The role of attention in auditory information processing as revealed by event-related potentials and other brain measures of cognitive function. Behav and Brain Sci 13(2):201–288CrossRefGoogle Scholar
  47. Niedermeyer E, Lopes da Silva F (2005) Electroencephalography, basic principles, clinical applications, and related fields. In: Niedermeyer E, Lopes da Silva F (eds) vol 5. Lippincott WIlliams & Wilkins, PhiladelphiaGoogle Scholar
  48. Ogawa S, Lee TM, Kay AR, Tank DW (1990) Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc Natl Acad Sci USA 87(24):9868–9872PubMedCentralPubMedCrossRefGoogle Scholar
  49. Oranje B, Aggernaes B, Rasmussen H, Ebdrup BH, Glenthoj BY (2013) P50 suppression and its neural generators in antipsychotic-naïve, first-episode schizophrenia before and after 6 months of quetiapine treatment. Schizophr Bull 39(2):472–480 Google Scholar
  50. Oranje B, Geyer MA, Kenemans JL, Verbaten MN (2006) Prepulse inhibition and P50 suppression: commonalities and dissociations. Psychiatry Res 143(2–3):147–158PubMedCrossRefGoogle Scholar
  51. Pae JS, Kwon JS, Youn T, Park HJ, Kim MS, Lee B, Park KS (2003) LORETA imaging of P300 in schizophrenia with individual MRI and 128-channel EEG. Neuroimage 20(3):1552–1560PubMedCrossRefGoogle Scholar
  52. Pfefferbaum A, Roth WT, Ford JM (1995) Event-related potentials in the study of psychiatric disorders. Arch Gen Psychiatry 52(7):559–563PubMedCrossRefGoogle Scholar
  53. Raichle ME, Mintun MA (2006) Brain work and brain imaging. Annu Rev Neurosci 29:449–476PubMedCrossRefGoogle Scholar
  54. Rasser PE, Schall U, Todd J, Michie PT, Ward PB, Johnston P, Helmbold K, Case V, Soyland A, Tooney PA, Thompson PM (2011) Gray matter deficits, mismatch negativity, and outcomes in schizophrenia. Schizophr Bull 37(1):131–140PubMedCentralPubMedCrossRefGoogle Scholar
  55. Ritter P, Villringer A (2006) Simultaneous EEG-fMRI. Neurosci Biobehav Rev 30(6):823–838PubMedCrossRefGoogle Scholar
  56. Scherg M (1990) Fundamentals of dipole source potential analysis. In: Grandori, F, Hoke M, Romani GL (eds) auditory evoked magnetic fields and electric potentials. Karger, Αdvanced Audiology 6, Basel, pp 40–69Google Scholar
  57. Scherg M, Ille N, Bornfleth H, Berg P (2002) Advanced tools for digital EEG review: virtual source montages, whole-head mapping, correlation, and phase analysis. J Clin Neurophysiol 19(2):91–112PubMedCrossRefGoogle Scholar
  58. Scherg M, Picton TW (1991) Separation and identification of event-related potential components by brain electric source analysis. Electroencephalogr Clin Neurophysiol Suppl 42:24–37Google Scholar
  59. Sumich AL, Kumari V, Heasman BC, Gordon E, Brammer M (2006) Abnormal asymmetry of N200 and P300 event-related potentials in subclinical depression. J Affect Disord 92(2–3):171–183PubMedCrossRefGoogle Scholar
  60. Tregellas JR, Davalos DB, Rojas DC, Waldo MC, Gibson L, Wylie K, Du YP, Freedman R (2007) Increased hemodynamic response in the hippocampus, thalamus and prefrontal cortex during abnormal sensory gating in schizophrenia. Schizophr Res 92(1–3):262–272PubMedCentralPubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Center for Clinical Intervention and Neuropsychiatric Schizophrenia Research (CINS)Copenhagen University Hospital, Psychiatric Center GlostrupGlostrupDenmark
  2. 2.Center for Neuropsychiatric Schizophrenia Research (CNSR)Copenhagen University Hospital, Psychiatric Center GlostrupGlostrupDenmark
  3. 3.Department of Neurology, Faculty of Health SciencesPsychiatry, and Sensory Sciences, University of CopenhagenCopenhagenDenmark
  4. 4.Department of Psychiatry, Brain Center Rudolf MagnusUniversity Medical Center UtrechtUtrechtThe Netherlands

Personalised recommendations