Skip to main content
SpringerLink
Log in

Multimodal Fusion of Structural and Functional Brain Imaging Data

  • Protocol
  • First Online:
Book cover fMRI Techniques and Protocols

Part of the book series: Neuromethods ((NM,volume 119))

Abstract

Recent years have witnessed a rapid growth of interest in moving functional magnetic resonance imaging (fMRI) beyond simple scan-length averages and into approaches that can integrate structural MRI measures and capture rich multimodal interactions. It is becoming increasingly clear that multimodal fusion is able to provide more information for individual subjects by exploiting covariation between modalities, rather an analysis of each modality alone. Multimodal fusion is a more complicated endeavor that must be approached carefully and efficient methods should be developed to draw generalized and valid conclusions out of high dimensional data with a limited number of subjects, such as patients with brain disorders. Numerous research efforts have been reported in the field based on various statistical models, including independent component analysis (ICA), canonical correlation analysis (CCA), and partial least squares (PLS). In this chapter, we survey a number of methods previously shown in multimodal fusion reports, performed with or without prior information, and with their possible strengths and limitations addressed. To examine the function–structure associations of the brain in a more comprehensive and integrated manner, we also reviewed a number of multimodal studies that combined fMRI and structural (sMRI and/or diffusion tensor MRI) measures, which could reveal important brain alterations that may not be fully detected by employing separate analysis of individual modalities, and also enable us to identify potential brain illness biomarkers.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Protocol
USD 49.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Skudlarski P, Jagannathan K, Calhoun VD et al (2008) Measuring brain connectivity: diffusion tensor imaging validates resting state temporal correlations. Neuroimage 43(3):554–561

    Article  PubMed  PubMed Central  Google Scholar 

  2. Savopol F, Armenakis C (2002) Mergine of heterogeneous data for emergency mapping: data integration or data fusion? Proc. ISPRS

    Google Scholar 

  3. Ardnt C, Loffeld O (1996) Information gained by data fusion, Proc. SPIE, vol. 2784

    Google Scholar 

  4. Sui J, Adali T, Yu Q et al (2012) A review of multivariate methods for multimodal fusion of brain imaging data. J Neurosci Methods 204(1):68–81

    Article  PubMed  Google Scholar 

  5. Sui J, Huster R, Yu Q et al (2014) Function-structure associations of the brain: evidence from multimodal connectivity and covariance studies. Neuroimage 102P1:11–23

    Article  Google Scholar 

  6. Schultz CC, Fusar-Poli P, Wagner G et al (2012) Multimodal functional and structural imaging investigations in psychosis research. Eur Arch Psychiatry Clin Neurosci 262(Suppl 2):S97–S106

    Article  PubMed  Google Scholar 

  7. Horwitz B, Poeppel D (2002) How can EEG/MEG and fMRI/PET data be combined? Hum Brain Mapp 17(1):1–3

    Article  PubMed  Google Scholar 

  8. Calhoun VD, Adali T (2009) Feature-based fusion of medical imaging data. IEEE Trans Inf Technol Biomed 13(5):711–720

    Article  PubMed  Google Scholar 

  9. Blum AL, Langley P (1997) Selection of relevant features and examples in machine learning. Artif Intell 97(1–2):245–271

    Article  Google Scholar 

  10. Smith SM, Fox PT, Miller KL et al (2009) Correspondence of the brain’s functional architecture during activation and rest. Proc Natl Acad Sci U S A 106(31):13040–13045

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Goldberg-Zimring D, Mewes AU, Maddah M et al (2005) Diffusion tensor magnetic resonance imaging in multiple sclerosis. J Neuroimaging 15(4 Suppl):68S–81S

    Article  PubMed  Google Scholar 

  12. Ramnani N, Lee L, Mechelli A et al (2002) Exploring brain connectivity: a new frontier in systems neuroscience. Functional Brain Connectivity, 4–6 April 2002, Dusseldorf, Germany. Trends Neurosci 25:496–497

    Article  PubMed  Google Scholar 

  13. Henson RN, Flandin G, Friston KJ et al (2010) A parametric empirical Bayesian framework for fMRI-constrained MEG/EEG source reconstruction. Hum Brain Mapp 31(10):1512–1531

    Article  PubMed  PubMed Central  Google Scholar 

  14. Lemieux L (2004) Electroencephalography-correlated functional MR imaging studies of epileptic activity. Neuroimaging Clin N Am 14(3):487–506

    Article  PubMed  Google Scholar 

  15. Friston KJ (2009) Modalities, modes, and models in functional neuroimaging. Science 326(5951):399–403

    Article  CAS  PubMed  Google Scholar 

  16. De Martino F, Valente G, de Borst AW et al (2010) Multimodal imaging: an evaluation of univariate and multivariate methods for simultaneous EEG/fMRI. Magn Reson Imaging 28(8):1104–1112

    Article  PubMed  Google Scholar 

  17. Eichele T, Specht K, Moosmann M et al (2005) Assessing the spatiotemporal evolution of neuronal activation with single-trial event-related potentials and functional MRI. Proc Natl Acad Sci U S A 102(49):17798–17803

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Astolfi L, Cincotti F, Mattia D et al (2004) Estimation of the effective and functional human cortical connectivity with structural equation modeling and directed transfer function applied to high-resolution EEG. Magn Reson Imaging 22(10):1457–1470

    Article  PubMed  Google Scholar 

  19. Rykhlevskaia E, Gratton G, Fabiani M (2008) Combining structural and functional neuroimaging data for studying brain connectivity: a review. Psychophysiology 45(2):173–187

    Article  PubMed  Google Scholar 

  20. Oakes TR, Fox AS, Johnstone T et al (2007) Integrating VBM into the General Linear Model with voxelwise anatomical covariates. Neuroimage 34(2):500–508

    Article  PubMed  Google Scholar 

  21. Schlosser R, Gesierich T, Kaufmann B et al (2003) Altered effective connectivity during working memory performance in schizophrenia: a study with fMRI and structural equation modeling. Neuroimage 19(3):751–763

    Article  PubMed  Google Scholar 

  22. Calhoun VD, Adali T, Kiehl KA et al (2006) A method for multitask fMRI data fusion applied to schizophrenia. Hum Brain Mapp 27(7):598–610

    Article  PubMed  PubMed Central  Google Scholar 

  23. Correa N, Adali T, Calhoun VD (2007) Performance of blind source separation algorithms for fMRI analysis using a group ICA method. Magn Reson Imaging 25(5):684–694

    Article  PubMed  Google Scholar 

  24. Groves AR, Beckmann CF, Smith SM et al (2011) Linked independent component analysis for multimodal data fusion. Neuroimage 54(3):2198–2217

    Article  PubMed  Google Scholar 

  25. Sui J, He H, Pearlson GD et al (2012) Three-way (N-way) fusion of brain imaging data based on mCCA + jICA and its application to discriminating schizophrenia. Neuroimage 2(66):119–132

    Google Scholar 

  26. Sui J, Pearlson G, Caprihan A et al (2011) Discriminating schizophrenia and bipolar disorder by fusing fMRI and DTI in a multimodal CCA+ joint ICA model. Neuroimage 57(3):839–855

    Article  PubMed  PubMed Central  Google Scholar 

  27. Chen K, Reiman EM, Huan Z et al (2009) Linking functional and structural brain images with multivariate network analyses: a novel application of the partial least square method. Neuroimage 47(2):602–610

    Article  PubMed  PubMed Central  Google Scholar 

  28. Martinez-Montes E, Valdes-Sosa PA, Miwakeichi F et al (2004) Concurrent EEG/fMRI analysis by multiway Partial Least Squares. Neuroimage 22(3):1023–1034

    Article  PubMed  Google Scholar 

  29. Correa NM, Eichele T, Adali T et al (2010) Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI. Neuroimage 50(4):1438–1445

    Article  PubMed  PubMed Central  Google Scholar 

  30. Liu J, Pearlson G, Windemuth A et al (2009) Combining fMRI and SNP data to investigate connections between brain function and genetics using parallel ICA. Hum Brain Mapp 30(1):241–255

    Article  PubMed  PubMed Central  Google Scholar 

  31. Sui J, Adali T, Pearlson GD et al (2009) A method for accurate group difference detection by constraining the mixing coefficients in an ICA framework. Hum Brain Mapp 30(9):2953–2970

    Article  PubMed  PubMed Central  Google Scholar 

  32. Sui J, Adali T, Pearlson GD et al (2009) An ICA-based method for the identification of optimal FMRI features and components using combined group-discriminative techniques. Neuroimage 46(1):73–86

    Article  PubMed  PubMed Central  Google Scholar 

  33. Caprihan A, Pearlson GD, Calhoun VD (2008) Application of principal component analysis to distinguish patients with schizophrenia from healthy controls based on fractional anisotropy measurements. Neuroimage 42(2):675–682

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Liu J, Xu L, Calhoun VD (2008). Extracting principle components for discriminant analysis of FMRI images. ICASSP 449–452

    Google Scholar 

  35. Ulloa AE, Chen J, Vergara VM et al (2014) Association between copy number variation losses and alcohol dependence across African American and European American ethnic groups. Alcohol Clin Exp Res 38(5):1266–1274

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Calhoun VD, Adali T, Giuliani NR et al (2006) Method for multimodal analysis of independent source differences in schizophrenia: combining gray matter structural and auditory oddball functional data. Hum Brain Mapp 27(1):47–62

    Article  CAS  PubMed  Google Scholar 

  37. Xu L, Pearlson G, Calhoun VD (2009) Joint source based morphometry identifies linked gray and white matter group differences. Neuroimage 44(3):777–789

    Article  PubMed  Google Scholar 

  38. Franco AR, Ling J, Caprihan A et al (2008) Multimodal and multi-tissue measures of connectivity revealed by joint independent component analysis. IEEE J Sel Top Signal Process 2(6):986–997

    Article  PubMed  PubMed Central  Google Scholar 

  39. Calhoun VD, Adali T, Liu J (2006) A feature-based approach to combine functional MRI, structural MRI and EEG brain imaging data. Conf Proc IEEE Eng Med Biol Soc 1:3672–3675

    CAS  PubMed  Google Scholar 

  40. Teipel SJ, Bokde AL, Meindl T et al (2010) White matter microstructure underlying default mode network connectivity in the human brain. Neuroimage 49(3):2021–2032

    Article  PubMed  Google Scholar 

  41. Eichele T, Calhoun VD, Debener S (2009) Mining EEG-fMRI using independent component analysis. Int J Psychophysiol 73(1):53–61

    Article  PubMed  PubMed Central  Google Scholar 

  42. Xu L, Groth KM, Pearlson G et al (2009) Source-based morphometry: the use of independent component analysis to identify gray matter differences with application to schizophrenia. Hum Brain Mapp 30(3):711–724

    Article  PubMed  PubMed Central  Google Scholar 

  43. Shenton ME, Dickey CC, Frumin M et al (2001) A review of MRI findings in schizophrenia. Schizophr Res 49(1–2):1–52

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Correa NM, Li YO, Adali T et al (2008) Canonical correlation analysis for feature-based fusion of biomedical imaging modalities and its application to detection of associative networks in schizophrenia. IEEE J Sel Top Signal Process 2(6):998–1007

    Article  PubMed  PubMed Central  Google Scholar 

  45. Li Y-O, Adali T, Wang W et al (2009) Joint blind source separation by multiset canonical correlation analysis. IEEE Trans Signal Process 57(10):3918–3929

    Article  PubMed  PubMed Central  Google Scholar 

  46. Lin FH, McIntosh AR, Agnew JA et al (2003) Multivariate analysis of neuronal interactions in the generalized partial least squares framework: simulations and empirical studies. Neuroimage 20(2):625–642

    Article  PubMed  Google Scholar 

  47. Krishnan A, Williams LJ, McIntosh AR et al (2010) Partial Least Squares (PLS) methods for neuroimaging: a tutorial and review. Neuroimage 56(2):455–475

    Article  PubMed  Google Scholar 

  48. Correa NM, Adali T, Li YO et al (2010) Canonical correlation analysis for data fusion and group inferences: examining applications of medical imaging data. IEEE Signal Process Mag 27(4):39–50

    Article  PubMed  PubMed Central  Google Scholar 

  49. McIntosh AR, Lobaugh NJ (2004) Partial least squares analysis of neuroimaging data: applications and advances. Neuroimage 23(Suppl 1):S250–S263

    Article  PubMed  Google Scholar 

  50. Camara E, Rodriguez-Fornells A, Munte TF (2010) Microstructural brain differences predict functional hemodynamic responses in a reward processing task. J Neurosci 30(34):11398–11402

    Article  CAS  PubMed  Google Scholar 

  51. Sui J, He H, Yu Q et al (2013) Combination of resting state fMRI, DTI, and sMRI data to discriminate schizophrenia by N-way MCCA + jICA. Front Hum Neurosci 7:235

    Article  PubMed  PubMed Central  Google Scholar 

  52. Beckmann CF, Smith SM (2005) Tensorial extensions of independent component analysis for multisubject FMRI analysis. Neuroimage 25(1):294–311

    Article  CAS  PubMed  Google Scholar 

  53. Hyvarinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13(4–5):411–430

    Article  CAS  PubMed  Google Scholar 

  54. Bell AJ, Sejnowski TJ (1995) An information-maximization approach to blind separation and blind deconvolution. Neural Comput 7(6):1129–1159

    Article  CAS  PubMed  Google Scholar 

  55. Bishop CM (1999) Variational principal components. Artif Neural Netw 7:509, Conference Publication No. 470

    Google Scholar 

  56. Liu J, Chen J, Ehrlich S et al (2014) Methylation patterns in whole blood correlate with symptoms in schizophrenia patients. Schizophr Bull 40(4):769–776

    Article  PubMed  Google Scholar 

  57. Segall JM, Allen EA, Jung RE et al (2012) Correspondence between structure and function in the human brain at rest. Front Neuroinform 6:10

    Article  PubMed  PubMed Central  Google Scholar 

  58. Khullar S, Michael AM, Cahill ND et al (2011) ICA-fNORM: spatial normalization of fMRI data using intrinsic group-ICA networks. Front Syst Neurosci 5:93

    Article  PubMed  PubMed Central  Google Scholar 

  59. Salgado-Pineda P, Junque C, Vendrell P et al (2004) Decreased cerebral activation during CPT performance: structural and functional deficits in schizophrenic patients. Neuroimage 21(3):840–847

    Article  PubMed  Google Scholar 

  60. Salgado-Pineda P, Fakra E, Delaveau P et al (2011) Correlated structural and functional brain abnormalities in the default mode network in schizophrenia patients. Schizophr Res 125(2–3):101–109

    Article  CAS  PubMed  Google Scholar 

  61. Skudlarski P, Jagannathan K, Anderson K et al (2010) Brain connectivity is not only lower but different in schizophrenia: a combined anatomical and functional approach. Biol Psychiatry 68(1):61–69

    Article  PubMed  PubMed Central  Google Scholar 

  62. Schlosser RG, Nenadic I, Wagner G et al (2007) White matter abnormalities and brain activation in schizophrenia: a combined DTI and fMRI study. Schizophr Res 89(1–3):1–11

    Article  PubMed  Google Scholar 

  63. Staempfli P, Reischauer C, Jaermann T et al (2008) Combining fMRI and DTI: a framework for exploring the limits of fMRI-guided DTI fiber tracking and for verifying DTI-based fiber tractography results. Neuroimage 39(1):119–126

    Article  CAS  PubMed  Google Scholar 

  64. Koch K, Wagner G, Schachtzabel C et al (2011) Neural activation and radial diffusivity in schizophrenia: combined fMRI and diffusion tensor imaging study. Br J Psychiatry 198(3):223–229

    Article  PubMed  Google Scholar 

  65. Kim J, Lee JH (2012) Integration of structural and functional magnetic resonance imaging improves mild cognitive impairment detection. Magn Reson Imaging 31:718–732

    Article  PubMed  Google Scholar 

  66. Fan Y, Resnick SM, Wu X et al (2008) Structural and functional biomarkers of prodromal Alzheimer’s disease: a high-dimensional pattern classification study. Neuroimage 41(2):277–285

    Article  PubMed  PubMed Central  Google Scholar 

  67. Wee CY, Yap PT, Zhang D et al (2011) Identification of MCI individuals using structural and functional connectivity networks. Neuroimage 59(3):2045–2056

    Article  PubMed  PubMed Central  Google Scholar 

  68. Westlye LT, Walhovd KB, Bjornerud A et al (2009) Error-related negativity is mediated by fractional anisotropy in the posterior cingulate gyrus—a study combining diffusion tensor imaging and electrophysiology in healthy adults. Cereb Cortex 19(2):293–304

    Article  PubMed  Google Scholar 

  69. Sui J, Yu Q, He H et al (2012) A selective review of multimodal fusion methods in schizophrenia. Front Hum Neurosci 6:27

    Article  PubMed  PubMed Central  Google Scholar 

  70. Zhang H, Liu L, Wu H, Fan Y (2012) Feature selection and SVM classification of multiple modality images for predicting MCI, in OHBM, Beijing, China.

    Google Scholar 

  71. Kim DI, Sui J, Rachakonda S et al (2010) Identification of imaging biomarkers in schizophrenia: a coefficient-constrained independent component analysis of the mind multi-site schizophrenia study. Neuroinformatics 8(4):213–229

    Article  PubMed  PubMed Central  Google Scholar 

  72. Tian L, Meng C, Yan H et al (2011) Convergent evidence from multimodal imaging reveals amygdala abnormalities in schizophrenic patients and their first-degree relatives. PLoS One 6(12):e28794

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  73. Casey BJ, Tottenham N, Liston C et al (2005) Imaging the developing brain: what have we learned about cognitive development? Trends Cogn Sci 9(3):104–110

    Article  CAS  PubMed  Google Scholar 

  74. Smieskova R, Allen P, Simon A et al (2012) Different duration of at-risk mental state associated with neurofunctional abnormalities. A multimodal imaging study. Hum Brain Mapp 33(10):2281–2294

    Article  PubMed  Google Scholar 

  75. Rasser PE, Johnston P, Lagopoulos J et al (2005) Functional MRI BOLD response to Tower of London performance of first-episode schizophrenia patients using cortical pattern matching. Neuroimage 26(3):941–951

    Article  PubMed  Google Scholar 

  76. Fusar-Poli P, Broome MR, Woolley JB et al (2011) Altered brain function directly related to structural abnormalities in people at ultra high risk of psychosis: longitudinal VBM-fMRI study. J Psychiatr Res 45(2):190–198

    Article  CAS  PubMed  Google Scholar 

  77. Michael AM, Baum SA, White T et al (2010) Does function follow form?: methods to fuse structural and functional brain images show decreased linkage in schizophrenia. Neuroimage 49(3):2626–2637

    Article  PubMed  Google Scholar 

  78. Michael AM, King MD, Ehrlich S et al (2011) A data-driven investigation of gray matter-function correlations in schizophrenia during a working memory task. Front Hum Neurosci 5:71

    PubMed  PubMed Central  Google Scholar 

  79. Rektorova I, Mikl M, Barrett J et al (2012) Functional neuroanatomy of vocalization in patients with Parkinson’s disease. J Neurol Sci 313(1–2):7–12

    Article  CAS  PubMed  Google Scholar 

  80. Harms MP, Wang L, Csernansky JG et al (2012) Structure-function relationship of working memory activity with hippocampal and prefrontal cortex volumes. Brain Struct Funct 218(1):173–186

    Article  PubMed  Google Scholar 

  81. Choi K, Yang Z, Hu X et al (2008) A combined functional-structural connectivity analysis of major depression using joint independent components analysis. Psychiatric MRI/MRS 3555

    Google Scholar 

  82. Camchong J, MacDonald AW III, Bell C et al (2011) Altered functional and anatomical connectivity in schizophrenia. Schizophr Bull 37(3):640–650

    Article  PubMed  Google Scholar 

  83. Kim DJ, Park B, Park HJ (2012) Functional connectivity-based identification of subdivisions of the basal ganglia and thalamus using multilevel independent component analysis of resting state fMRI. Hum Brain Mapp 34:1371–1385

    Article  PubMed  Google Scholar 

  84. Olesen PJ, Nagy Z, Westerberg H et al (2003) Combined analysis of DTI and fMRI data reveals a joint maturation of white and grey matter in a fronto-parietal network. Brain Res Cogn Brain Res 18(1):48–57

    Article  PubMed  Google Scholar 

  85. Matthews SC, Strigo IA, Simmons AN et al (2011) A multimodal imaging study in U.S. veterans of Operations Iraqi and Enduring Freedom with and without major depression after blast-related concussion. Neuroimage 54(Suppl 1):S69–S75

    Article  PubMed  Google Scholar 

  86. Zhou Y, Shu N, Liu Y et al (2008) Altered resting-state functional connectivity and anatomical connectivity of hippocampus in schizophrenia. Schizophr Res 100(1–3):120–132

    Article  PubMed  Google Scholar 

  87. Yan H, Tian L, Yan J et al (2012) Functional and anatomical connectivity abnormalities in cognitive division of anterior cingulate cortex in schizophrenia. PLoS One 7(9):e45659

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  88. Soldner J, Meindl T, Koch W et al (2011) Structural and functional neuronal connectivity in Alzheimer’s disease: a combined DTI and fMRI study. Nervenarzt 83(7):878–887

    Article  Google Scholar 

  89. Wang F, Kalmar JH, He Y et al (2009) Functional and structural connectivity between the perigenual anterior cingulate and amygdala in bipolar disorder. Biol Psychiatry 66(5):516–521

    Article  PubMed  PubMed Central  Google Scholar 

  90. Schonberg T, Pianka P, Hendler T et al (2006) Characterization of displaced white matter by brain tumors using combined DTI and fMRI. Neuroimage 30(4):1100–1111

    Article  PubMed  Google Scholar 

  91. Voss HU, Schiff ND (2009) MRI of neuronal network structure, function, and plasticity. Prog Brain Res 175:483–496

    Article  PubMed  Google Scholar 

  92. Palacios EM, Sala-Llonch R, Junque C et al (2012) White matter integrity related to functional working memory networks in traumatic brain injury. Neurology 78(12):852–860

    Article  CAS  PubMed  Google Scholar 

  93. Jacobson S, Kelleher I, Harley M et al (2009) Structural and functional brain correlates of subclinical psychotic symptoms in 11–13 year old schoolchildren. Neuroimage 49(2):1875–1885

    Article  PubMed  Google Scholar 

  94. Supekar K, Uddin LQ, Prater K et al (2010) Development of functional and structural connectivity within the default mode network in young children. Neuroimage 52(1):290–301

    Article  PubMed  PubMed Central  Google Scholar 

  95. Pomarol-Clotet E, Canales-Rodriguez EJ, Salvador R et al (2010) Medial prefrontal cortex pathology in schizophrenia as revealed by convergent findings from multimodal imaging. Mol Psychiatry 15(8):823–830

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  96. Sexton CE, Allan CL, Le Masurier M et al (2012) Magnetic resonance imaging in late-life depression: multimodal examination of network disruption. Arch Gen Psychiatry 69(7):680–689

    Article  PubMed  Google Scholar 

  97. Qiu MG, Ye Z, Li QY et al (2011) Changes of brain structure and function in ADHD children. Brain Topogr 24(3–4):243–252

    Article  PubMed  Google Scholar 

  98. Sui J, He H, Liu J et al (2012) Three-way FMRI-DTI-methylation data fusion based on mCCA + jICA and its application to schizophrenia. Conf Proc IEEE Eng Med Biol Soc 2012:2692–2695

    PubMed  Google Scholar 

  99. Zhang D, Wang Y, Zhou L et al (2011) Multimodal classification of Alzheimer’s disease and mild cognitive impairment. Neuroimage 55(3):856–867

    Article  PubMed  PubMed Central  Google Scholar 

  100. Groves AR, Smith SM, Fjell AM et al (2012) Benefits of multi-modal fusion analysis on a large-scale dataset: life-span patterns of inter-subject variability in cortical morphometry and white matter microstructure. Neuroimage 63(1):365–380

    Article  PubMed  Google Scholar 

  101. Sui J, Castro E, Hao H et al (2014) Combination of FMRI-SMRI-EEG data improves discrimination of schizophrenia patients by ensemble feature selection. The 36th Annual International Conference of the IEEE engineering in medicine and biology society (EMBC‘14) Chicago, Illinois, USA, no. August 26–30

    Google Scholar 

  102. Eichele T, Calhoun VD, Moosmann M et al (2008) Unmixing concurrent EEG-fMRI with parallel independent component analysis. Int J Psychophysiol 67(3):222–234

    Article  PubMed  Google Scholar 

  103. Haller S, Xekardaki A, Delaloye C et al (2011) Combined analysis of grey matter voxel-based morphometry and white matter tract-based spatial statistics in late-life bipolar disorder. J Psychiatry Neurosci 36(6):391–401

    Article  PubMed  PubMed Central  Google Scholar 

  104. Chen Z, Cui L, Li M et al (2011) Voxel based morphometric and diffusion tensor imaging analysis in male bipolar patients with first-episode mania. Prog Neuropsychopharmacol Biol Psychiatry 36(2):231–238

    Article  PubMed  Google Scholar 

  105. Meda SA, Jagannathan K, Gelernter J et al (2010) A pilot multivariate parallel ICA study to investigate differential linkage between neural networks and genetic profiles in schizophrenia. Neuroimage 53(3):1007–1015

    Article  PubMed  Google Scholar 

  106. Jagannathan K, Calhoun VD, Gelernter J et al (2010) Genetic associations of brain structural networks in schizophrenia: a preliminary study. Biol Psychiatry 68(7):657–666

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  107. Jamadar S, Powers NR, Meda SA et al (2010) Genetic influences of cortical gray matter in language-related regions in healthy controls and schizophrenia. Schizophr Res 129:141–148

    Article  Google Scholar 

  108. Hao X, Xu D, Bansal R et al (2011) Multimodal magnetic resonance imaging: the coordinated use of multiple, mutually informative probes to understand brain structure and function. Hum Brain Mapp 34(2):253–271

    Article  PubMed  PubMed Central  Google Scholar 

  109. Fusar-Poli P, McGuire P, Borgwardt S (2011) Mapping prodromal psychosis: a critical review of neuroimaging studies. Eur Psychiatry 27(3):181–191

    Article  PubMed  Google Scholar 

  110. Meda SA, Gill A, Stevens MC et al (2012) Differences in resting-state functional magnetic resonance imaging functional network connectivity between schizophrenia and psychotic bipolar probands and their unaffected first-degree relatives. Biol Psychiatry 71(10):881–889

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Brainnetome Center and National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Jing Sui

  2. The Mind Research Network and Lovelace Biomedical and Environmental Research Institute, Albuquerque, NM, 87106, USA

    Jing Sui & Vince D. Calhoun

  3. Electrical and Computer, University of New Mexico, Albuquerque, NM, 87131, USA

    Vince D. Calhoun

  4. Departments and Neurosciences, University of New Mexico, Albuquerque, NM, 87131, USA

    Vince D. Calhoun

Authors
  1. Jing Sui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vince D. Calhoun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vince D. Calhoun .

Editor information

Editors and Affiliations

  1. Neuroimaging Research Unit, INSPE, Division of Neuroscience, San Raffaele Scientific Institute and Vita-Salute San Raffaele University, Milan, Italy

    Massimo Filippi

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Science+Business Media New York

About this protocol

Cite this protocol

Sui, J., Calhoun, V.D. (2016). Multimodal Fusion of Structural and Functional Brain Imaging Data. In: Filippi, M. (eds) fMRI Techniques and Protocols. Neuromethods, vol 119. Humana Press, New York, NY. https://doi.org/10.1007/978-1-4939-5611-1_28

Download citation

  • DOI: https://doi.org/10.1007/978-1-4939-5611-1_28

  • Published:

  • Publisher Name: Humana Press, New York, NY

  • Print ISBN: 978-1-4939-5609-8

  • Online ISBN: 978-1-4939-5611-1

  • eBook Packages: Springer Protocols

Publish with us

Policies and ethics

Search

Navigation