Graph-based network analysis of resting-state fMRI: test-retest reliability of binarized and weighted networks

  • Jie Xiang
  • Jiayue Xue
  • Hao Guo
  • Dandan LiEmail author
  • Xiaohong Cui
  • Yan Niu
  • Ting Yan
  • Rui Cao
  • Yao Ma
  • Yanli Yang
  • Bin WangEmail author


In the past decade, resting-state functional magnetic resonance imaging (rs-fMRI) and graph-based measures have been widely used to quantitatively characterize the architectures of brain functional networks in healthy individuals and in patients with abnormalities related to psychopathic and neurological disorders. To accurately evaluate the topological organization of brain functional networks, the definition of the nodes and edges for the construction of functional networks is critical. Furthermore, both types of brain functional networks (binarized networks and weighted networks) are widely used to analyze topological organization. However, how to best select the network type is still debated. Consequently, we investigated the test-retest reliability of brain functional networks with binarized and weighted edges using two independent datasets and four strategies for defining nodes. We revealed fair to good reliability for a majority of network topological attributes and overall higher reliabilities for weighted networks than for binarized networks. For regional nodal efficiency, weighted networks also showed higher reliability across nodes. Thus, our findings imply that weighted networks contain more information, leading to more stable results. In addition, we found that the reliability of brain functional networks was influenced by the node definition strategy and that more precise of nodal definition were associated with higher reliability. The time effect of reliability was restricted, as no differences between long-term and short-term reliability were observed. In conclusion, our results suggest that weighted networks have better reliability because they reflect more topological information, implying broader applications of weighted networks related to normal and disordered function of the human brain.


Resting-state fMRI Graph-based measures Binarized and weighted edges Test-retest reliability 



This project is supported by the National Natural Science Foundation of China (61503272, 61873178 and 61876124), the Natural Science Foundation of Shanxi (201801D121135), the International Science and Technology Cooperation Project of Shanxi (201803D421047), and the Youth Science and Technology Research Fund (201701D221119). Also, we would like to thank PhD Lynne Hyman for the professional language editing services.


This project is supported by the National Natural Science Foundation of China (61503272, 61873178 and 61876124), the Natural Science Foundation of Shanxi (201801D121135), the International Science and Technology Cooperation Project of Shanxi (201803D421047), and the Youth Science and Technology Research Fund (201701D221119).

Compliance with ethical standards

Conflict of interest

Jie Xiang, Jiayue Xue, Hao Guo, Dandan Li, Xiaohong Cui, Yan Niu, Ting Yan, Rui Cao, Yao Ma, Yanli Yang and Bin Wang declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics approval

The dataset of IPCAS_1 and NYU CSC are used in the study. The IPCAS_1 dataset was approved by the Institute of Psychology, Chinese Academy of Sciences, the NYU CSC dataset was approved by the New York University, Child Study Center.

Supplementary material

11682_2019_42_MOESM1_ESM.pdf (1.5 mb)
ESM 1 (PDF 1583 kb)


  1. Achard, S., & Bullmore, E. (2007). Efficiency and cost of economical brain functional networks. PLoS Computational Biology, 3(2), e17.CrossRefGoogle Scholar
  2. Achard, S., Salvador, R., Whitcher, B., Suckling, J., & Bullmore, E. (2006). A resilient, low-frequency, small-world human brain functional network with highly connected association cortical hubs. Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 26(1), 63–72.CrossRefGoogle Scholar
  3. Aso, T., Okamura, S., Matsuguchi, T., Sakamoto, N., Sata, T., & Niho, Y. (2011). Rich-Club Organization of the Human Connectome. Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 31(44), 15775–15786.CrossRefGoogle Scholar
  4. Bassett, D. S., & Bullmore, E. T. (2016). Small-world brain networks revisited. Neuroscientist A Review Journal Bringing Neurobiology Neurology & Psychiatry, 23(5).Google Scholar
  5. Bassett, D. S., Brown, J. A., Deshpande, V., Carlson, J. M., & Grafton, S. T. (2011a). Conserved and variable architecture of human white matter connectivity. Neuroimage, 54(2), 1262–1279.CrossRefGoogle Scholar
  6. Bassett, D. S., Wymbs, N. F., Porter, M. A., Mucha, P. J., Carlson, J. M., & Grafton, S. T. (2011b). Dynamic reconfiguration of human brain networks during learning. Proceedings of the National Academy of Sciences of the United States of America, 108(18), 7641–7646.CrossRefGoogle Scholar
  7. Bassett, D. S., Nelson, B. G., Mueller, B. A., Camchong, J., & Lim, K. O. (2012). Altered resting state complexity in schizophrenia. Neuroimage, 59(3), 2196–2207.CrossRefGoogle Scholar
  8. Biswal, B. B., Mennes, M., Zuo, X. N., Gohel, S., Kelly, C., Smith, S. M., Beckmann, C. F., Adelstein, J. S., Buckner, R. L., & Colcombe, S. (2009). Toward discovery science of human brain function. Proceedings of the National Academy of Sciences of the United States of America, 71(10), 4734–4739.CrossRefGoogle Scholar
  9. Braun, U., Plichta, M. M., Esslinger, C., Sauer, C., Haddad, L., Grimm, O., Mier, D., Mohnke, S., Heinz, A., Erk, S., Walter, H., Seiferth, N., Kirsch, P., & Meyer-Lindenberg, A. (2012). Test–retest reliability of resting-state connectivity network characteristics using fMRI and graph theoretical measures. NeuroImage, 59(2), 1404–1412.CrossRefGoogle Scholar
  10. Buckner, R. L., Andrewshanna, J. R., & Schacter, D. L. (2008). The brain's default network: Anatomy, function, and relevance to disease. Annals of the New York Academy of Sciences, 1124(1), 1–38.CrossRefGoogle Scholar
  11. Butts, C. T. (2009). Revisiting the foundations of network analysis. Science, 325(5939), 414–416.CrossRefGoogle Scholar
  12. Cao, H., Plichta, M. M., Schäfer, A., Haddad, L., Grimm, O., Schneider, M., Esslinger, C., Kirsch, P., Meyer Lindenberg, A., & Tost, H. (2014). Test-retest reliability of fMRI-based graph theoretical properties during working memory, emotion processing, and resting state. Neuroimage, 84(1), 888–900.CrossRefGoogle Scholar
  13. Charles, L. (2014). Resolving structure in human brain organization: Identifying mesoscale Organization in Weighted Network Representations. PLoS Computational Biology, 10(10), e1003712.CrossRefGoogle Scholar
  14. Cocito, C., Vanlinden, F., & Branlant, C. (2012). Exposure to subliminal arousing stimuli induces robust activation in the amygdala, hippocampus, anterior cingulate, insular cortex and primary visual cortex: A systematic meta-analysis of fMRI studies. Neuroimage, 59(3), 2962–2973.CrossRefGoogle Scholar
  15. Cole, M. W., Pathak, S., & Schneider, W. (2010). Identifying the brain's most globally connected regions. Neuroimage, 49(4), 3132–3148.CrossRefGoogle Scholar
  16. Doria, V., Beckmann, C. F., Arichi, T., Merchant, N., Groppo, M., Turkheimer, F. E., Counsell, S. J., Murgasova, M., Aljabar, P., & Nunes, R. G. (2010). Emergence of resting state networks in the preterm human brain. Proceedings of the National Academy of Sciences of the United States of America, 107(46), 20015–20020.CrossRefGoogle Scholar
  17. Dosenbach, N. U. F., Nardos, B., Cohen, A. L., Fair, D. A., Power, J. D., Church, J. A., Nelson, S. M., Wig, G. S., Vogel, A. C., & Lessovschlaggar, C. N. (2010). Prediction of individual brain maturity using fMRI. Science, 329(5997), 1358–1361.CrossRefGoogle Scholar
  18. Fan, L., Li, H., Zhuo, J., Zhang, Y., Wang, J., Chen, L., Yang, Z., Chu, C., Xie, S., & Laird, A. R. (2016). The human Brainnetome atlas: A new brain atlas based on connectional architecture. Cerebral Cortex, 26(8), 3508–3526.CrossRefGoogle Scholar
  19. Farine, D. R. (2014). Measuring phenotypic assortment in animal social networks: Weighted associations are more robust than binary edges. Animal Behaviour, 89(3), 141–153.CrossRefGoogle Scholar
  20. Finn, E. S., Shen, X., Scheinost, D., Rosenberg, M. D., Huang, J., Chun, M. M., Papademetris, X., & Constable, R. T. (2015). Functional connectome fingerprinting: Identifying individuals based on patterns of brain connectivity. Nature Neuroscience, 18(11), 1664–1671.CrossRefGoogle Scholar
  21. Gilman, S. R., Iossifov, I., Levy, D., Ronemus, M., Wigler, M., & Vitkup, D. (2011). Rare de novo variants associated with autism implicate a large functional network of genes involved in formation and function of synapses. Neuron, 70(5), 898–907.CrossRefGoogle Scholar
  22. Glasser, M. F., Smith, S. M., Marcus, D. S., Andersson, J. L. R., Auerbach, E. J., Behrens, T. E. J., Coalson, T. S., Harms, M. P., Jenkinson, M., & Moeller, S. (2016). The human connectome Project's neuroimaging approach. Nature Neuroscience, 19(9), 1175–1187.CrossRefGoogle Scholar
  23. Gong, G., He, Y., Concha, L., Lebel, C., Gross, D. W., Evans, A. C., & Beaulieu, C. (2009). Mapping anatomical connectivity patterns of human cerebral cortex using in vivo diffusion tensor imaging Tractography. Cerebral Cortex, 19(3), 524–536.CrossRefGoogle Scholar
  24. Guo, C. C., Kurth, F., Zhou, J., Mayer, E. A., Eickhoff, S. B., Kramer, J. H., & Seeley, W. W. (2012). One-year test-retest reliability of intrinsic connectivity network fMRI in older adults. Neuroimage, 61(4), 1471–1483.CrossRefGoogle Scholar
  25. Hayasaka, S., & Laurienti, P. J. (2010). Comparison of characteristics between region-and voxel-based network analyses in resting-state fMRI data. Neuroimage, 50(2), 499–508.CrossRefGoogle Scholar
  26. He, Y., Chen, Z., & Evans, A. (2008). Structural insights into aberrant topological patterns of large-scale cortical networks in Alzheimer's disease. Journal of Neuroscience, 4(4), T284–T285.Google Scholar
  27. He, Y., Dagher, A., Chen, Z., Charil, A., Zijdenbos, A., Worsley, K., & Evans, A. (2009). Impaired small-world efficiency in structural cortical networks in multiple sclerosis associated with white matter lesion load. Brain: A Journal of Neurology, 132(Pt 12), 3366–3379.CrossRefGoogle Scholar
  28. Heuvel, M. P. V. D., & Sporns, O. (2013). Network hubs in the human brain. Trends in Cognitive Sciences, 17(12), 683–696.CrossRefGoogle Scholar
  29. Horn, A., Ostwald, D., Reisert, M., & Blankenburg, F. (2014). The structural-functional connectome and the default mode network of the human brain. Neuroimage, 102142–102151.Google Scholar
  30. Kim, J., Chey, J., Kim, S. E., & Kim, H. (2015). The effect of education on regional brain metabolism and its functional connectivity in an aged population utilizing positron emission tomography. Neuroscience Research, 94(3), 50–61.CrossRefGoogle Scholar
  31. Klimm, F., Bassett, D.S., Carlson, J.M., Mucha, P.J., (2014). Resolving Structural Variability in Network Models and the Brain. PLoS Computational Biology, 10,3(2014-3-27) 10(3), e1003491.Google Scholar
  32. Koch, W., Teipel, S., Mueller, S., Buerger, K., Bokde, A. L. W., Hampel, H., Coates, U., Reiser, M., & Meindl, T. (2010). Effects of aging on default mode network activity in resting state fMRI: Does the method of analysis matter? Neuroimage, 51(1), 280–287.CrossRefGoogle Scholar
  33. Latora, V., & Marchiori, M. (2001). Efficient behavior of small-world networks. Physical Review Letters, 87(19), 198701.CrossRefGoogle Scholar
  34. Mazoyer, B., Zago, L., Mellet, E., Bricogne, S., Etard, O., Houdé, O., Crivello, F., Joliot, M., Petit, L., & Tzourio-Mazoyer, N. (2001). Cortical networks for working memory and executive functions sustain the conscious resting state in man. Brain Research Bulletin, 54(3), 287–298.CrossRefGoogle Scholar
  35. Meunier, D., Achard, S., Morcom, A., & Bullmore, E. (2009). Age-related changes in modular organization of human brain functional networks. Neuroimage, 44(3), 715–723.CrossRefGoogle Scholar
  36. Motter, A. E., Changsong, Z., & Jürgen, K. (2005). Network synchronization, diffusion, and the paradox of heterogeneity. Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics, 71(2), 016116.CrossRefGoogle Scholar
  37. Newman, M. E. (2002). Assortative mixing in networks. Physical Review Letters, 89(20), 208701.CrossRefGoogle Scholar
  38. Newman, M. E. (2006). Modularity and community structure in networks. APS March Meeting, pp., 8577–8582.Google Scholar
  39. Plichta, M. M., Schwarz, A. J., Grimm, O., Morgen, K., Mier, D., Haddad, L., Gerdes, A. B., Sauer, C., Tost, H., & Esslinger, C. (2012). Test-retest reliability of evoked BOLD signals from a cognitive-emotive fMRI test battery. Neuroimage, 60(3), 1746–1758.CrossRefGoogle Scholar
  40. Power, J. D., Cohen, A. L., Nelson, S. M., Wig, G. S., Barnes, K. A., Church, J. A., Vogel, A. C., Laumann, T. O., Miezin, F. M., & Schlaggar, B. L. (2011). Functional network Organization of the Human Brain. Neuron, 72(4), 665–678.CrossRefGoogle Scholar
  41. Raichle, M. E., Macleod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., & Shulman, G. L. (2001). A default mode of brain function. Proceedings of the National Academy of Sciences of the United States of America, 98(2), 676–682.CrossRefGoogle Scholar
  42. Ravasz, E., & Barabási, A. L. (2003). Hierarchical organization in complex networks. Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics, 67(2), 026112.CrossRefGoogle Scholar
  43. Rubinov, M., & Sporns, O. (2010). Complex network measures of brain connectivity: Uses and interpretations. Neuroimage, 52(3), 1059–1069.CrossRefGoogle Scholar
  44. Rubinov, M., & Sporns, O. (2011). Weight-conserving characterization of complex functional brain networks. Neuroimage, 56(4), 2068–2079.CrossRefGoogle Scholar
  45. Sampat, M. P., Whitman, G. J., Stephens, T. W., Broemeling, L. D., Heger, N. A., Bovik, A. C., & Markey, M. K. (2006). The reliability of measuring physical characteristics of spiculated masses on mammography. British Journal of Radiology 79 Spec No, 2(special_issue_2), S134.CrossRefGoogle Scholar
  46. Sanabriadiaz, G., Meliegarcía, L., Iturriamedina, Y., Alemángómez, Y., Hernándezgonzález, G., Valdésurrutia, L., Galán, L., & Valdéssosa, P. (2010). Surface area and cortical thickness descriptors reveal different attributes of the structural human brain networks. Neuroimage, 50(4), 1497–1510.CrossRefGoogle Scholar
  47. Shrout, P. E., & Fleiss, J. L. (2015). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420.CrossRefGoogle Scholar
  48. Shulman, G. L., Fiez, J. A., Corbetta, M., Buckner, R. L., Miezin, F. M., Raichle, M. E., & Petersen, S. E. (1997). Common blood flow changes across visual tasks: II. Decreases in cerebral cortex. Journal of Cognitive Neuroscience, 9(5), 648–663.CrossRefGoogle Scholar
  49. Smith, S. M., Miller, K. L., Salimikhorshidi, G., Webster, M., Beckmann, C. F., Nichols, T. E., Ramsey, J. D., & Woolrich, M. W. (2011). Network modelling methods for FMRI. Neuroimage, 54(2), 875–891.CrossRefGoogle Scholar
  50. Smith, S. M., Vidaurre, D., Beckmann, C. F., Glasser, M. F., Jenkinson, M., Miller, K. L., Nichols, T. E., Robinson, E., Salimikhorshidi, G., & Woolrich, M. W. (2013). Functional connectomics from resting-state fMRI. Trends in Cognitive Sciences, 17(12), 666–682.CrossRefGoogle Scholar
  51. Sporns, O. (2002). Network analysis, complexity, and brain function. Complexity, 8(1), 56–60.CrossRefGoogle Scholar
  52. Sporns, O. (2013). Making sense of brain network data. Nature Methods, 10(6), 491–493.CrossRefGoogle Scholar
  53. Sporns, O., Tononi, G., & Kötter, R. (2005). The human connectome: A structural description of the human brain. PLoS Computational Biology, 1(4), e42.CrossRefGoogle Scholar
  54. Tian, L., Wang, J., Yan, C., & He, Y. (2011). Hemisphere- and gender-related differences in small-world brain networks: A resting-state functional MRI study. Neuroimage, 54(1), 191–202.CrossRefGoogle Scholar
  55. Tijms, B. M., Wink, A. M., De, H. W., Wm, V. D. F., Stam, C. J., Scheltens, P., & Barkhof, F. (2013). Alzheimer's disease: Connecting findings from graph theoretical studies of brain networks. Neurobiology of Aging, 34(8), 2023–2036.CrossRefGoogle Scholar
  56. Tzouriomazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., & Joliot, M. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage, 15(1), 273–289.CrossRefGoogle Scholar
  57. Van Den Heuvel, M., Stam, C. J., Kahn, R. S., & Hulshoff Pol, H. E. (2009). Efficiency of functional brain networks and intellectual performance. Journal of Neuroscience, 29(23), 7619–7624.CrossRefGoogle Scholar
  58. Van Wijk, B. C., Stam, C. J., & Daffertshofer, A. (2010). Comparing brain networks of different size and connectivity density using graph theory. PLoS One, 5(10), e13701.CrossRefGoogle Scholar
  59. Wandell, B. A., Dumoulin, S. O., & Brewer, A. A. (2007). Visual field maps in human cortex. Neuron, 56(2), 366–383.CrossRefGoogle Scholar
  60. Wang, J., Zuo, X., & He, Y. (2010a). Graph-based network analysis of resting-state functional MRI. Frontiers in Systems Neuroscience, 4(16), 16.Google Scholar
  61. Wang, L., Li, Y., Metzak, P., He, Y., & Woodward, T. S. (2010b). Age-related changes in topological patterns of large-scale brain functional networks during memory encoding and recognition. Neuroimage, 50(3), 862–872.CrossRefGoogle Scholar
  62. Wang, L., Zhu, C., He, Y., Zang, Y., Cao, Q., Zhang, H., Zhong, Q., & Wang, Y. (2010c). Altered small-world brain functional networks in children with attention-deficit/hyperactivity disorder. Human Brain Mapping, 30(2), 638–649.CrossRefGoogle Scholar
  63. Wang, J. H., Zuo, X. N., Gohel, S., Milham, M. P., Biswal, B. B., & He, Y. (2011). Graph theoretical analysis of functional brain networks: Test-retest evaluation on short- and long-term resting-state functional MRI data. PLoS One, 6(7), e21976.CrossRefGoogle Scholar
  64. Watts, D.J., Strogatz, S.H., (1998). Collective dynamics of 'small-world' networks, Collective dynamics of ‘small-world’ networks.Google Scholar
  65. Weber, M. J., Detre, J. A., Thompsonschill, S. L., & Avants, B. B. (2013). Reproducibility of functional network metrics and network structure: A comparison of task-related BOLD, resting ASL with BOLD contrast, and resting cerebral blood flow. Cognitive, Affective, & Behavioral Neuroscience, 13(3), 627–640.CrossRefGoogle Scholar
  66. Wig, G. S., Schlaggar, B. L., & Petersen, S. E. (2011). Concepts and principles in the analysis of brain networks. Annals of the New York Academy of Sciences, 1224(1), 126–146.CrossRefGoogle Scholar
  67. Wijk, B. C. M. V., Stam, C. J., & Daffertshofer, A. (2010). Comparing brain networks of different size and connectivity density using graph theory. PLoS One, 5(10), e13701.CrossRefGoogle Scholar
  68. Winer, B. J. (1962). Statistical principles in experimental design. International Student Edition, 29, 7304–7309.Google Scholar
  69. Zalesky, A., Fornito, A., Harding, I. H., Cocchi, L., Yücel, M., Pantelis, C., & Bullmore, E. T. (2010). Whole-brain anatomical networks: Does the choice of nodes matter? Neuroimage, 50(3), 970–983.CrossRefGoogle Scholar
  70. Zhao, X., Liu, Y., Wang, X., Liu, B., Xi, Q., Guo, Q., Jiang, H., Jiang, T., & Wang, P. (2012). Disrupted small-world brain networks in moderate Alzheimer's disease: A resting-state FMRI study. PLoS One, 7(3), e33540.CrossRefGoogle Scholar
  71. Zhao, K., Yan, W. J., Chen, Y. H., Zuo, X. N., & Fu, X. (2013). Amygdala volume predicts inter-individual differences in fearful face recognition. PLoS One, 8(8), e74096.CrossRefGoogle Scholar
  72. Zhou, J., Gennatas, E. D., Kramer, J. H., Miller, B. L., & Seeley, W. W. (2012). Predicting regional neurodegeneration from the healthy brain functional connectome. Neuron, 73(6), 1216–1227.CrossRefGoogle Scholar
  73. Zuo, X. N., & Xing, X. X. (2014). Test-retest reliabilities of resting-state FMRI measurements in human brain functional connectomics: A systems neuroscience perspective. Neuroscience and Biobehavioral Reviews, 45, 100–118.CrossRefGoogle Scholar
  74. Zuo, X. N., Ehmke, R., Mennes, M., Imperati, D., Castellanos, F. X., Sporns, O., & Milham, M. P. (2012). Network centrality in the human functional connectome. Cerebral Cortex, 22(8), 1862–1875.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Information and ComputerTaiyuan University of TechnologyTaiyuanChina
  2. 2.Translational Medicine Research CenterShanxi Medical UniversityTaiyuanChina

Personalised recommendations