Skip to main content
SpringerLink
Log in

Evidence Graphs: Supporting Transparent and FAIR Computation, with Defeasible Reasoning on Data, Methods, and Results

  • Conference paper
  • First Online:
Provenance and Annotation of Data and Processes (IPAW 2020, IPAW 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12839))

Abstract

Introduction: Transparency of computation is a requirement for assessing the validity of computed results and research claims based upon them; and it is essential for access to, assessment, and reuse of computational components. These components may be subject to methodological or other challenges over time. While reference to archived software and/or data is increasingly common in publications, a single machine-interpretable, integrative representation of how results were derived, that supports defeasible reasoning, has been absent.

Methods: We developed the Evidence Graph Ontology, EVI, in OWL 2, with a set of inference rules, to provide deep representations of supporting and challenging evidence for computations, services, software, data, and results, across arbitrarily deep networks of computations, in connected or fully distinct processes.

EVI integrates FAIR practices on data and software, with important concepts from provenance models, and argumentation theory. It extends PROV for additional expressiveness, with support for defeasible reasoning. EVI treats any computational result or component of evidence as a defeasible assertion, supported by a DAG of the computations, software, data, and agents that produced it.

Results: We have successfully deployed EVI for large-scale predictive analytics on clinical time-series data. Every result may reference its evidence graph as metadata, which can be extended when subsequent computations are executed.

Discussion: Evidence graphs support transparency and defeasible reasoning on results. They are first-class computational objects and reference the datasets and software from which they are derived. They support fully transparent computation, with challenge and support propagation. The EVI approach may be extended to include instruments, animal models, and critical experimental reagents.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cousijn, H., et al.: A data citation roadmap for scientific publishers. Sci. Data 5, 180259 (2018). https://doi.org/10.1038/sdata.2018.259

    Article  Google Scholar 

  2. Data Citation Synthesis Group: Joint declaration of data citation principles. In: Future of Research Communication and e-Scholarship (FORCE11), San Diego (2014)

    Google Scholar 

  3. Fenner, M., et al.: A data citation roadmap for scholarly data repositories. Sci. Data 6, 28 (2019). https://doi.org/10.1038/s41597-019-0031-8

    Article  Google Scholar 

  4. Groth, P., Cousijn, H., Clark, T., Goble, C.: FAIR data reuse—the path through data citation. Data Intell. 2, 78–86 (2020). https://doi.org/10.1162/dint_a_00030

    Article  Google Scholar 

  5. Juty, N., Wimalaratne, S.M., Soiland-Reyes, S., Kunze, J., Goble, C.A., Clark, T.: Unique, persistent, resolvable: identifiers as the foundation of FAIR. Data Intell. 2, 30–39 (2020). https://doi.org/10.5281/zenodo.3267434

    Article  Google Scholar 

  6. Katz, D.S., et al.: Recognizing the value of software: a software citation guide. F1000Research 9, 1257 (2021). https://doi.org/10.12688/f1000research.26932.2

    Article  Google Scholar 

  7. Katz, D.S., Gruenpeter, M., Honeyman, T.: Taking a fresh look at FAIR for research software. Patterns 2(3), 100222 (2021). https://doi.org/10.1016/j.patter.2021.100222

    Article  Google Scholar 

  8. Smith, A.M., Katz, D.S., Niemeyer, K.E.: FORCE11 Software Citation Working Group: software citation principles. PeerJ Comput. Sci. 2, e86 (2016). https://doi.org/10.7717/peerj-cs.86

    Article  Google Scholar 

  9. Starr, J., et al.: Achieving human and machine accessibility of cited data in scholarly publications. PeerJ Comput. Sci. 1, e1 (2015). https://doi.org/10.7717/peerj-cs.1

    Article  Google Scholar 

  10. Wilkinson, M.D., et al.: The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data 3, 160018 (2016)

    Article  Google Scholar 

  11. Wimalaratne, S.M., et al.: Uniform resolution of compact identifiers for biomedical data. Sci. Data 5, 180029 (2018). https://doi.org/10.1038/sdata.2018.29

    Article  Google Scholar 

  12. Dear, P.: Revolutionizing the Sciences: European Knowledge and Its Ambitions, 1500–1700. Princeton University Press, Princeton and Oxford (2009)

    Book  Google Scholar 

  13. Holmes, F.L.: Argument and narrative in scientific writing. In: Dear, P. (ed.) The Literary Structure of Scientific Argument: Historical Studies, p. 224. University of Pennsylvania Press, Philadelphia (1991)

    Google Scholar 

  14. Rossi, P.: Philosophy, Technology, and the Arts in the Early Modern Era. Harper & Row, New York (1970)

    Google Scholar 

  15. Shapin, S.: Pump and circumstance: Robert Boyle’s literary technology. In: Hellyer, M. (ed.) The Scientific Revolution. Blackwell, Oxford (2003)

    Google Scholar 

  16. Committee on Science: Engineering, and Public Policy of the National Academies: On Being a Scientist: Responsible Conduct in Research. National Academies Press, Washington (1995)

    Google Scholar 

  17. Lakatos, I.: Proofs and Refutations. Cambridge University Press, Cambridge (1976)

    Book  Google Scholar 

  18. Maxwell, E.A.: Fallacies in Mathematics. Cambridge University Press, Cambridge (1959)

    Book  Google Scholar 

  19. Krabbe, E.C.W.: Strategic maneuvering in mathematical proofs. Argumentation 22, 453–468 (2008). https://doi.org/10.1007/s10503-008-9098-7

    Article  Google Scholar 

  20. Ioannidis, J.P.A.: Why most published research findings are false. PLoS Med. 2, e124 (2005). https://doi.org/10.1371/journal.pmed.0020124

    Article  Google Scholar 

  21. Ioannidis, J.A.: Contradicted and initially stronger effects in highly cited clinical research. JAMA 294, 218–228 (2005). https://doi.org/10.1001/jama.294.2.218

    Article  MathSciNet  Google Scholar 

  22. Koons, R.: Defeasible Reasoning (2013). http://plato.stanford.edu/archives/spr2014/entries/reasoning-defeasible/

  23. Clark, T., Ciccarese, P.N., Goble, C.A.: Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. J. Biomed. Semant. 5, 28 (2014). https://doi.org/10.1186/2041-1480-5-28

    Article  Google Scholar 

  24. Greenberg, S.A.: Understanding belief using citation networks. J. Eval. Clin. Pract. 17, 389–393 (2011). https://doi.org/10.1111/j.1365-2753.2011.01646.x

    Article  Google Scholar 

  25. Greenberg, S.A.: How citation distortions create unfounded authority: analysis of a citation network. BMJ 339, b2680 (2009). https://doi.org/10.1136/bmj.b2680

    Article  Google Scholar 

  26. Bench-Capon, T.J.M., Dunne, P.E.: Argumentation in artificial intelligence. Artif. Intell. 171, 619–641 (2007). https://doi.org/10.1016/j.artint.2007.05.001

    Article  MathSciNet  MATH  Google Scholar 

  27. Besnard, P., Hunter, A.: Elements of Argumentation. MIT Press, Cambridge (2008)

    Book  Google Scholar 

  28. Boella, G., Gabbay, D.M., Van Der Torre, L., Villata, S.: Support in abstract argumentation. In: Baroni, P., et al. (eds.) Computational Models of Argument. IOS Press, Amsterdam (2010)

    Google Scholar 

  29. Brewka, G., Polberg, S., Woltran, S.: Generalizations of dung frameworks and their role in formal argumentation. IEEE Intell. Syst. 29, 30–38 (2014). https://doi.org/10.1109/MIS.2013.122

    Article  Google Scholar 

  30. Carrera, Á., Iglesias, C.A.: A systematic review of argumentation techniques for multi-agent systems research. Artif. Intell. Rev. 44(4), 509–535 (2015). https://doi.org/10.1007/s10462-015-9435-9

    Article  Google Scholar 

  31. Cayrol, C., Lagasquie-Schiex, M.C.: Bipolar abstract argumentation systems. In: Rahwan, I., Simari, G.R. (eds.) Argumentation in Artificial Intelligence. Springer, Dordrecht (2009). https://doi.org/10.1007/978-0-387-98197-0_4

    Chapter  MATH  Google Scholar 

  32. Cohen, A., Gottifredi, S., García, A.J., Simari, G.R.: An approach to abstract argumentation with recursive attack and support. J. Appl. Log. 13, 509–533 (2015). https://doi.org/10.1016/j.jal.2014.12.001

    Article  MathSciNet  MATH  Google Scholar 

  33. Dung, P.M.: On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artif. Intell. 77, 321–357 (1995). https://doi.org/10.1016/0004-3702(94)00041-x

    Article  MathSciNet  MATH  Google Scholar 

  34. Oren, N., Norman, T.J.: Semantics for Evidence-Based Argumentation, p. 9. IOS Press, Amsterdam (2003)

    Google Scholar 

  35. Brewka, G., Woltran, S.: Abstract Dialectical Frameworks, p. 10 (2010)

    Google Scholar 

  36. Dung, P.M., Thang, P.M.: Representing the semantics of abstract dialectical frameworks based on arguments and attacks. Argum. Comput. 9, 249–267 (2018). https://doi.org/10.3233/AAC-180427

    Article  Google Scholar 

  37. Cayrol, C., Lagasquie-Schiex, M.-C.: Coalitions of arguments: a tool for handling bipolar argumentation frameworks. Int. J. Intell. Syst. 25, 83–109 (2010). https://doi.org/10.1002/int.20389

    Article  MATH  Google Scholar 

  38. Cayrol, C., Lagasquie-Schiex, M.-C.: Bipolarity in argumentation graphs: towards a better understanding. Int. J. Approximate Reasoning 54, 876–899 (2013). https://doi.org/10.1016/j.ijar.2013.03.001

    Article  MathSciNet  MATH  Google Scholar 

  39. Gil, Y., et al.: PROV Model Primer: W3C Working Group Note 30 April 2013 (2013). https://www.w3.org/TR/prov-primer/

  40. Lebo, T., et al.: PROV-O: The PROV Ontology W3C Recommendation 30 April 2013 (2013)

    Google Scholar 

  41. Moreau, L., et al.: PROV-DM: The PROV Data Model: W3C Recommendation 30 April 2013. World Wide Web Consortium (2013)

    Google Scholar 

  42. Soergel, D.A.W.: Rampant software errors may undermine scientific results. F1000Research 3, 303 (2015). https://doi.org/10.12688/f1000research.5930.2

    Article  Google Scholar 

  43. Neupane, J.B., Neupane, R.P., Luo, Y., Yoshida, W.Y., Sun, R., Williams, P.G.: Characterization of leptazolines A–D, polar oxazolines from the cyanobacterium Leptolyngbya sp., reveals a glitch with the “Willoughby–Hoye” scripts for calculating NMR chemical shifts. Org. Lett. 21(20), 8449–8453 (2019). https://doi.org/10.1021/acs.orglett.9b03216

    Article  Google Scholar 

  44. Miller, G.: A scientist’s nightmare: software problem leads to five retractions. Science 314, 1856–1857 (2006). https://doi.org/10.1126/science.314.5807.1856

    Article  Google Scholar 

  45. Axelrod, V.: Minimizing bugs in cognitive neuroscience programming. Front. Psychol. 5, 1435 (2014). https://doi.org/10.3389/fpsyg.2014.01435

    Article  Google Scholar 

  46. Brown, A.W., Kaiser, K.A., Allison, D.B.: Issues with data and analyses: errors, underlying themes, and potential solutions. Proc. Natl. Acad. Sci. USA 115, 2563–2570 (2018). https://doi.org/10.1073/pnas.1708279115

    Article  Google Scholar 

  47. Goldberg, S.I., Niemierko, A., Turchin, A.: Analysis of Data Errors in Clinical Research Databases. 5

    Google Scholar 

  48. Giglio, M., et al.: ECO, the evidence and conclusion ontology: community standard for evidence information. Nucleic Acids Res. 47, D1186–D1194 (2019). https://doi.org/10.1093/nar/gky1036

    Article  Google Scholar 

  49. Rocca-Serra, P., et al.: ISA software suite: supporting standards-compliant experimental annotation and enabling curation at the community level. Bioinformatics 26, 2354–2356 (2010). https://doi.org/10.1093/bioinformatics/btq415

    Article  Google Scholar 

  50. Bandrowski, A., et al.: The ontology for biomedical investigations. PLoS ONE 11, e0154556 (2016). https://doi.org/10.1371/journal.pone.0154556

    Article  Google Scholar 

  51. Velterop, J.: Nanopublications: the future of coping with information overload. LOGOS 21, 119–122 (2010). https://doi.org/10.1163/095796511X560006

    Article  Google Scholar 

  52. Gibson, A., van Dam, J., Schultes, E., Roos, M., Mons, B.: Towards computational evaluation of evidence for scientific assertions with nanopublications and cardinal assertions. In: Proceedings of the 5th International Workshop on Semantic Web Applications and Tools for Life Sciences (SWAT4LS), Paris, pp. 28–30 (2012)

    Google Scholar 

  53. Groth, P., Gibson, A., Velterop, J.: The anatomy of a nano-publication. Inf. Serv. Use 30, 51–56 (2010). https://doi.org/10.3233/ISU-2010-0613

    Article  Google Scholar 

  54. Schultes, E., et al.: The Open PHACTS Nanopublication Guidelines V1.8. EU Innovative Medicines Initiative—Open PHACTS Project RDF/Nanopublication Working Group (2012)

    Google Scholar 

  55. DeRoure, D., Goble, C.: Lessons from myExperiment: Research Objects for Data Intensive Research. Presented at the eScience Workshop (2009)

    Google Scholar 

  56. Bechhofer, S., Roure, D.D., Gamble, M., Goble, C., Buchan, I.: Research objects: towards exchange and reuse of digital knowledge. Presented at the Future of the Web for Collaborative Science (FWCS), 19th International World Wide Web Conference (WWW 2010) 26 April (2010)

    Google Scholar 

  57. Belhajjame, K., et al.: Using a suite of ontologies for preserving workflow-centric research objects. J. Web Semant. 32, 16–42 (2015). https://doi.org/10.1016/j.websem.2015.01.003

    Article  Google Scholar 

  58. Carragáin, E.Ó., Goble, C., Sefton, P., Soiland-Reyes, S.: A lightweight approach to research object data packaging (2019). https://doi.org/10.5281/ZENODO.3250687

  59. Toulmin, S.E.: The Uses of Argument. Cambridge University Press, Cambridge (2003)

    Book  Google Scholar 

  60. Verheij, B.: Evaluating arguments based on Toulmin’s scheme. Argumentation 19, 347–371 (2005). https://doi.org/10.1007/s10503-005-4421-z

    Article  Google Scholar 

  61. Verheij, B.: The Toulmin argument model in artificial intelligence. Or: how semi-formal, defeasible argumentation schemes creep into logic. In: Rahwan, I., Simari, G. (eds.) Argumentation in Artificial Intellgence. Springer, Dordrecht (2009). https://doi.org/10.1007/978-0-387-98197-0_11

    Chapter  Google Scholar 

  62. Aristotle: Rhetoric. Dover Publications, Mineola (2004)

    Google Scholar 

  63. Austin, J.L.: How to Do Things with Words. Harvard University Press, Cambridge (1962)

    Google Scholar 

  64. Levinson, M.A., et al.: FAIRSCAPE: a framework for FAIR and reproducible biomedical analytics. 2020.08.10.244947 (2020). https://doi.org/10.1101/2020.08.10.244947

  65. OWL 2 Working Group: OWL 2 Web Ontology Language: W3C Recommendation 27 October 2009. World Wide Web Consortium, Cambridge (2009)

    Google Scholar 

  66. Al Manir, S., Niestroy, J., Levinson, M., Clark, T.: EVI: The Evidence Graph Ontology, OWL 2 Vocabulary, Zenodo (2021)

    Google Scholar 

  67. Guha, R.V., Brickley, D., Macbeth, S.: Schema.org: evolution of structured data on the web. Commun. ACM 59(2), 44–51 (2016). https://doi.org/10.1145/2844544

    Article  Google Scholar 

  68. Troupin, C., Muñoz, C., Fernández, J.G.: Scientific results traceability: software citation using GitHub and Zenodo. 4 (2018)

    Google Scholar 

  69. Niestroy, J., et al.: Discovery of signatures of fatal neonatal illness in vital signs using highly comparative time-series analysis. BioRXiv. 2021.03.26.437138 (2021). https://doi.org/10.1101/2021.03.26.437138

  70. Niestroy, J., Levinson, M.A., Al Manir, S., Clark, T.: Evidence graph for: discovery of signatures of fatal neonatal illness in vital signs using highly comparative time-series analysis (2021). https://doi.org/10.18130/V3/HHTAYI

  71. Niestroy, J., et al.: Replication data for: discovery of signatures of fatal neonatal illness in vital signs using highly comparative time-series analysis, V2 (2021). https://doi.org/10.18130/V3/VJXODP

  72. Kunze, J., Rodgers, R.: The ARK Identifier Scheme (2008). https://escholarship.org/uc/item/9p9863nc

  73. Bandrowski, A.E., Martone, M.E.: RRIDs: a simple step toward improving reproducibility through rigor and transparency of experimental methods. Neuron 90, 434–436 (2016). https://doi.org/10.1016/j.neuron.2016.04.030

    Article  Google Scholar 

Download references

Acknowledgements

We thank Chris Baker (University of New Brunswick), Carole Goble (University of Manchester), and John Kunze (California Digital Library) for helpful discussions. This work was supported in part by the U.S. National Institutes of Health, grant NIH 1U01HG009452; and by a grant from the Coulter Foundation.

Author information

Authors and Affiliations

  1. Department of Public Health Sciences, University of Virginia, Charlottesville, VA, USA

    Sadnan Al Manir, Justin Niestroy, Maxwell Adam Levinson & Timothy Clark

  2. School of Data Science, University of Virginia, Charlottesville, VA, USA

    Timothy Clark

Authors
  1. Sadnan Al Manir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Justin Niestroy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maxwell Adam Levinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Timothy Clark
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Timothy Clark .

Editor information

Editors and Affiliations

  1. Illinois Institute of Technology, Chicago, IL, USA

    Boris Glavic

  2. Fluminense Federal University, Niterói, Brazil

    Vanessa Braganholo

  3. Northern Illinois University, DeKalb, IL, USA

    David Koop

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Al Manir, S., Niestroy, J., Levinson, M.A., Clark, T. (2021). Evidence Graphs: Supporting Transparent and FAIR Computation, with Defeasible Reasoning on Data, Methods, and Results. In: Glavic, B., Braganholo, V., Koop, D. (eds) Provenance and Annotation of Data and Processes. IPAW IPAW 2020 2021. Lecture Notes in Computer Science(), vol 12839. Springer, Cham. https://doi.org/10.1007/978-3-030-80960-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-80960-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-80959-1

  • Online ISBN: 978-3-030-80960-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation