Abstract
The connectional brain template (CBT) captures the shared traits across all individuals of a given population of brain connectomes, thereby acting as a fingerprint. Estimating a CBT from a population where brain graphs are derived from diverse neuroimaging modalities (e.g., functional and structural) and at different resolutions (i.e., number of nodes) remains a formidable challenge to solve. Such network integration task allows for learning a rich and universal representation of the brain connectivity across varying modalities and resolutions. The resulting CBT can be substantially used to generate entirely new multimodal brain connectomes, which can boost the learning of the downs-stream tasks such as brain state classification. Here, we propose the Multimodal Multiresolution Brain Graph Integrator Network (i.e., M2GraphIntegrator), the first multimodal multiresolution graph integration framework that maps a given connectomic population into a well-centered CBT. M2GraphIntegrator first unifies brain graph resolutions by utilizing resolution-specific graph autoencoders. Next, it integrates the resulting fixed-size brain graphs into a universal CBT lying at the center of its population. To preserve the population diversity, we further design a novel clustering-based training sample selection strategy which leverages the most heterogeneous training samples. To ensure the biological soundness of the learned CBT, we propose a topological loss that minimizes the topological gap between the ground-truth brain graphs and the learned CBT. Our experiments show that from a single CBT, one can generate realistic connectomic datasets including brain graphs of varying resolutions and modalities. We further demonstrate that our framework significantly outperforms benchmarks in reconstruction quality, augmentation task, centeredness and topological soundness.
E. Cinar and S.E. Haseki—Co-first authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ideker, T., Galitski, T., Hood, L.: A new approach to decoding life: systems biology. Annu. Rev. Genomics Hum. Genet. 2, 343–372 (2001)
Essen, D., et al.: The human connectome project: a data acquisition perspective. Neuroimage 62, 2222–31 (2012)
Qiu, J., Qinglin, Z., Bi, T., Wu, G., Wei, D., Yang, W.: (Southwest university longitudinal imaging multimodal (SLIM) brain data repository: a long-term test-retest sample of young healthy adults in southwest china)
Biobank, U.: About UK biobank (2014). https://www.ukbiobank.ac.uk/about-biobank-uk
Fornito, A., Zalesky, A., Breakspear, M.: The connectomics of brain disorders. Nat. Rev. Neurosci. 16, 159–172 (2015)
Fornito, A., Zalesky, A., Bullmore, E.: Fundamentals of Brain Network Analysis. Academic Press (2016)
van den Heuvel, M.P., Sporns, O.: A cross-disorder connectome landscape of brain dysconnectivity. Nat. Rev. Neurosci. 20, 435–446 (2019)
Seidlitz, J., et al.: Morphometric similarity networks detect microscale cortical organization and predict inter-individual cognitive variation. Neuron 97, 231–247 (2018)
Holmes, A.J., et al.: Brain genomics superstruct project initial data release with structural, functional, and behavioral measures. Sci. Data 2, 1–16 (2015)
Rekik, I., Li, G., Lin, W., Shen, D.: Estimation of brain network atlases using diffusive-shrinking graphs: application to developing brains. In: Niethammer, M., Styner, M., Aylward, S., Zhu, H., Oguz, I., Yap, P.-T., Shen, D. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 385–397. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59050-9_31
Dhifallah, S., Rekik, I., Initiative, A.D.N., et al.: Estimation of connectional brain templates using selective multi-view network normalization. Med. Image Anal. 59, 101567 (2020)
Gurbuz, M.B., Rekik, I.: Deep graph normalizer: a geometric deep learning approach for estimating connectional brain templates. In: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (eds.) MICCAI 2020. LNCS, vol. 12267, pp. 155–165. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_16
Bessadok, A., Mahjoub, M.A., Rekik, I.: Graph neural networks in network neuroscience. arXiv preprint arXiv:2106.03535 (2021)
Nalepa, J., Marcinkiewicz, M., Kawulok, M.: Data augmentation for brain-tumor segmentation: a review. Front. Comput. Neurosci. 13, 83 (2019)
Perl, Y.S., et al.: Data augmentation based on dynamical systems for the classification of brain states. Chaos, Solitons Fractals 139, 110069 (2020)
Sserwadda, A., Rekik, I.: Topology-guided cyclic brain connectivity generation using geometric deep learning. J. Neurosci. Methods 353, 108988 (2021)
Khan, A., Fraz, K.: Post-training iterative hierarchical data augmentation for deep networks. Advances in Neural Information Processing Systems 33 (2020)
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. Adv. Neural. Inf. Process. Syst. 33, 5812–5823 (2020)
Perez, L., Wang, J.: The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621 (2017)
Mikołajczyk, A., Grochowski, M.:Data augmentation for improving deep learning in image classification problem. In: 2018 International Interdisciplinary PhD Workshop (IIPhDW). IEEE (2018) 117–122
Graa, O., Rekik, I.: Multi-view learning-based data proliferator for boosting classification using highly imbalanced classes. J. Neurosci. Methods 327, 108344 (2019)
Wong, S.C., Gatt, A., Stamatescu, V., McDonnell, M.D.: Understanding data augmentation for classification: when to warp? In: 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–6. IEEE (2016)
Du, Y., Fu, Z., Calhoun, V.D.: Classification and prediction of brain disorders using functional connectivity: promising but challenging. Front. Neurosci. 12, 525 (2018)
Wang, B., et al.: Similarity network fusion for aggregating data types on a genomic scale. Nat. Methods 11, 333 (2014)
Demir, U., Gharsallaoui, M.A., Rekik, I.: Clustering-based deep brain multigraph integrator network for learning connectional brain templates. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis, pp. 109–120 (2020)
Sağlam, M., Rekik, I.: Multi-scale profiling of brain multigraphs by eigen-based cross-diffusion and heat tracing for brain state profiling. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis, pp. 142–151 (2020)
Isallari, M., Rekik, I.: Gsr-net: graph super-resolution network for predicting high-resolution from low-resolution functional brain connectomes. In: International Workshop on Machine Learning in Medical Imaging, pp. 139–149 (2020)
Mhiri, I., Mahjoub, M.A., Rekik, I.: Stairwaygraphnet for inter-and intra-modality multi-resolution brain graph alignment and synthesis. In: International Workshop on Machine Learning in Medical Imaging, pp. 140–150 (2021)
Mhiri, I., Khalifa, A.B., Mahjoub, M.A., Rekik, I.: Brain graph super-resolution for boosting neurological disorder diagnosis using unsupervised multi-topology connectional brain template learning. Med. Image Anal. 65, 101768 (2020)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks (2017)
Shen, X., et al.: Using connectome-based predictive modeling to predict individual behavior from brain connectivity. Nature Protocols 12, 506–518 (2017)
Kullback, S.: Information theory and statistics. Courier Corporation (1997)
Acknowledgements
This work was funded by generous grants from the European H2020 Marie Sklodowska-Curie action (grant no. 101003403, http://basira-lab.com/normnets/) to I.R. and the Scientific and Technological Research Council of Turkey to I.R. under the TUBITAK 2232 Fellowship for Outstanding Researchers (no. 118C288, http://basira-lab.com/reprime/). However, all scientific contributions made in this project are owned and approved solely by the authors.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cinar, E., Haseki, S.E., Bessadok, A., Rekik, I. (2022). Deep Cross-Modality and Resolution Graph Integration for Universal Brain Connectivity Mapping and Augmentation. In: Manfredi, L., et al. Imaging Systems for GI Endoscopy, and Graphs in Biomedical Image Analysis. ISGIE GRAIL 2022 2022. Lecture Notes in Computer Science, vol 13754. Springer, Cham. https://doi.org/10.1007/978-3-031-21083-9_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-21083-9_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21082-2
Online ISBN: 978-3-031-21083-9
eBook Packages: Computer ScienceComputer Science (R0)
