Connecting Neural Reconstruction Integrity (NRI) to Graph Metrics and Biological Priors

  • Elizabeth P. ReillyEmail author
  • Erik C. Johnson
  • Marisa J. Hughes
  • Devin RamsdenEmail author
  • Laurent Park
  • Brock Wester
  • Will Gray-Roncal
Conference paper
Part of the Springer Proceedings in Complexity book series (SPCOM)


We previously introduced the Neural Reconstruction Integrity (NRI) metric as a measure of how well the connectivity of the brain is measured in a neural circuit reconstruction, which can be represented as a graph or network. While powerful, NRI requires ground truth data for evaluation, which is conventionally obtained through time-intensive human annotation. NRI is a proxy for graph-based metrics since it focuses on the pre- and post-synaptic connections (or in and out edges) at a single neuron or vertex rather than overall graph structure in order to satisfy the format of available ground truth and provide rapid assessments. In this paper, we study the relationship between the NRI and graph theoretic metrics in order to understand the relationship of NRI to small world properties, centrality measures, and cost of information flow, as well as minimize our dependence on ground truth. The common errors under evaluation are synapse insertions and deletions and neuron splits and merges. We also elucidate the connection between graph metrics and biological priors for more meaningful interpretation of our results. We identified the most useful local metric to be local clustering coefficient, while the most useful global metrics are characteristic path length, rich-club coefficient, and density due to their strong correlations with NRI and perturbation errors.


Neural Reconstruction Integrity Graph metrics Connectomics Evaluation 


  1. 1.
    Albert, R., Barabási, A.L.: Statistical mechanics of complex networks. Rev. Mod. Phys. 74(1), 47 (2002)ADSMathSciNetCrossRefGoogle Scholar
  2. 2.
    Arganda-Carreras, I., Turaga, S.C., Berger, D.R., Cireşan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J., Laptev, D., Dwivedi, S., Buhmann, J.M., et al.: Crowdsourcing the creation of image segmentation algorithms for connectomics. Front. Neuroanat. 9, 142 (2015)CrossRefGoogle Scholar
  3. 3.
    Bassett, D.S., Bullmore, E.: Small-world brain networks. Neuroscientist 12(6), 512–523 (2006)CrossRefGoogle Scholar
  4. 4.
    Binnewijzend, M.A., Adriaanse, S.M., Van der Flier, W.M., Teunissen, C.E., de Munck, J.C., Stam, C.J., Scheltens, P., van Berckel, B.N., Barkhof, F., Wink, A.M.: Brain network alterations in Alzheimer’s disease measured by eigenvector centrality in fMRI are related to cognition and CSF biomarkers. Hum. Brain Mapp. 35(5), 2383–2393 (2014)CrossRefGoogle Scholar
  5. 5.
    Brin, S., Page, L.: The anatomy of a large-scale hypertextual web search engine. Comput. Netw. ISDN Syst. 30(1–7), 107–117 (1998)CrossRefGoogle Scholar
  6. 6.
    Bullmore, E., Sporns, O.: Complex brain networks: graph theoretical analysis of structural and functional systems. Nat. Rev. Neurosci. 10(3), 186 (2009)CrossRefGoogle Scholar
  7. 7.
    Denk, W., Horstmann, H.: Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure. PLoS Biol. 2(11), e329 (2004)CrossRefGoogle Scholar
  8. 8.
    Eberhard, J.P., Wanner, A., Wittum, G.: Neugen: a tool for the generation of realistic morphology of cortical neurons and neural networks in 3D. Neurocomputing 70(1–3), 327–342 (2006)CrossRefGoogle Scholar
  9. 9.
    Erdos, P., Rényi, A.: On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci 5(1), 17–60 (1960)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Fletcher, J.M., Wennekers, T.: From structure to activity: using centrality measures to predict neuronal activity. Int. J. Neural Syst. 28(02), 1750013 (2018)CrossRefGoogle Scholar
  11. 11.
    Fornito, A., Zalesky, A., Breakspear, M.: Graph analysis of the human connectome: promise, progress, and pitfalls. Neuroimage 80, 426–444 (2013)CrossRefGoogle Scholar
  12. 12.
    Funke, J., Klein, J., Moreno-Noguer, F., Cardona, A., Cook, M.: TED: a tolerant edit distance for segmentation evaluation. Methods 115, 119–127 (2017)CrossRefGoogle Scholar
  13. 13.
    Gong, G., He, Y., Concha, L., Lebel, C., Gross, D.W., Evans, A.C., Beaulieu, C.: Mapping anatomical connectivity patterns of human cerebral cortex using in vivo diffusion tensor imaging tractography. Cereb. Cortex 19(3), 524–536 (2008)CrossRefGoogle Scholar
  14. 14.
    Gray Roncal, W.R., Kleissas, D.M., Vogelstein, J.T., Manavalan, P., Lillaney, K., Pekala, M., Burns, R., Vogelstein, R.J., Priebe, C.E., Chevillet, M.A., et al.: An automated images-to-graphs framework for high resolution connectomics. Fronti. Neuroinform. 9, 20 (2015)Google Scholar
  15. 15.
    Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 2(1), 193–218 (1985)CrossRefGoogle Scholar
  16. 16.
    Jain, V., Bollmann, B., Richardson, M., Berger, D.R., Helmstaedter, M.N., Briggman, K.L., Denk, W., Bowden, J.B., Mendenhall, J.M., Abraham, W.C., et al.: Boundary learning by optimization with topological constraints. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2488–2495. IEEE (2010)Google Scholar
  17. 17.
    Januszewski, M., Kornfeld, J., Li, P.H., Pope, A., Blakely, T., Lindsey, L., Maitin-Shepard, J., Tyka, M., Denk, W., Jain, V.: High-precision automated reconstruction of neurons with flood-filling networks. Nat. Methods 15(8), 605 (2018)CrossRefGoogle Scholar
  18. 18.
    Johnson, E.C., Wilt, M., Rodriguez, L.M., Norman-Tenazas, R., Rivera, C., Drenkow, N., Kleissas, D., LaGrow, T.J., Cowley, H., Downs, J., et al.: Toward a reproducible, scalable framework for processing large neuroimaging datasets. BioRxiv, p. 615161 (2019)Google Scholar
  19. 19.
    Knott, G., Marchman, H., Wall, D., Lich, B.: Serial section scanning electron microscopy of adult brain tissue using focused ion beam milling. J. Neurosci. 28(12), 2959–2964 (2008)CrossRefGoogle Scholar
  20. 20.
    Lohmann, G., Margulies, D.S., Horstmann, A., Pleger, B., Lepsien, J., Goldhahn, D., Schloegl, H., Stumvoll, M., Villringer, A., Turner, R.: Eigenvector centrality mapping for analyzing connectivity patterns in fMRI data of the human brain. PLoS ONE 5(4), e10232 (2010)ADSCrossRefGoogle Scholar
  21. 21.
    Nunez-Iglesias, J., Kennedy, R., Parag, T., Shi, J., Chklovskii, D.B.: Machine learning of hierarchical clustering to segment 2D and 3D images. PLoS ONE 8(8), e71715 (2013)ADSCrossRefGoogle Scholar
  22. 22.
    Plaza, S.M.: Focused proofreading to reconstruct neural connectomes from EM images at scale. In: Deep Learning and Data Labeling for Medical Applications, pp. 249–258. Springer (2016)Google Scholar
  23. 23.
    Rand, W.M.: Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66(336), 846–850 (1971)CrossRefGoogle Scholar
  24. 24.
    Reilly, E.P., Garretson, J.S., Gray Roncal, W.R., Kleissas, D.M., Wester, B.A., Chevillet, M.A., Roos, M.J.: Neural reconstruction integrity: a metric for assessing the connectivity accuracy of reconstructed neural networks. Front. Neuroinformatics 12, 74 (2018)CrossRefGoogle Scholar
  25. 25.
    Toga, A.W., Clark, K.A., Thompson, P.M., Shattuck, D.W., Van Horn, J.D.: Mapping the human connectome. Neurosurgery 71(1), 1–5 (2012)CrossRefGoogle Scholar
  26. 26.
    Van Den Heuvel, M.P., Sporns, O.: Rich-club organization of the human connectome. J. Neurosci. 31(44), 15775–15786 (2011)CrossRefGoogle Scholar
  27. 27.
    Wasserman, S., Faust, K.: Social Network Analysis: Methods and Applications, vol. 8. Cambridge University Press, Cambridge (1994)CrossRefGoogle Scholar
  28. 28.
    Watts, D.J., Strogatz, S.H.: Collective dynamics of ‘small-world’ networks. Nature 393(6684), 440 (1998)ADSCrossRefGoogle Scholar
  29. 29.
    Zheng, Z., Lauritzen, J.S., Perlman, E., Robinson, C.G., Nichols, M., Milkie, D., Torrens, O., Price, J., Fisher, C.B., Sharifi, N., et al.: A complete electron microscopy volume of the brain of adult drosophila melanogaster. Cell 174(3), 730–743 (2018)CrossRefGoogle Scholar

Copyright information

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Elizabeth P. Reilly
    • 1
    Email author
  • Erik C. Johnson
    • 1
  • Marisa J. Hughes
    • 1
  • Devin Ramsden
    • 2
    Email author
  • Laurent Park
    • 2
  • Brock Wester
    • 1
  • Will Gray-Roncal
    • 1
  1. 1.Johns Hopkins University Applied Physics LaboratoryLaurelUSA
  2. 2.Johns Hopkins UniversityBaltimoreUSA

Personalised recommendations