Abstract
Traditionally, peer-review focuses on the evaluation of scientific publications, literature products that describe the research process and its final results in natural language. The adoption of ICT technologies in support of science introduces new opportunities to support transparent evaluation, thanks to the possibility of sharing research products, even inputs, intermediate and negative results, repetition and reproduction of the research activities conducted in a digital laboratory. Such innovative shift also sets the condition for novel peer review methodologies, as well as scientific reward policies, where scientific results can be transparently and objectively assessed via machine-assisted processes. This paper presents the foundations of a framework for the representation of a peer-reviewable research flow for a given discipline of science. Such a framework may become the scaffolding enabling the development of tools for supporting ongoing peer review of research flows. Such tools could be “hooked”, in real time, to the underlying digital laboratory, where scientists are carrying out their research flow, and they would abstract over the complexity of the research activity and offer user-friendly dashboards.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
RStudio, https://www.rstudio.com/.
- 3.
Jupyter Notebook, http://jupyter.org/.
- 4.
Taverna workbench, https://taverna.incubator.apache.org/.
- 5.
Examples are research funders like the European Commission [2], Wellcome Trust and funders of the cOAlition-S (https://www.scienceeurope.org/coalition-s/).
- 6.
BlueBridge, http://www.bluebridge-vres.eu/.
- 7.
PANGAEA: https://www.pangaea.de/.
- 8.
UNIPROT: https://www.ebi.ac.uk/uniprot.
References
European Commission: Validation of the results of the public consultation on Science 2.0: Science in Transition [report]. Brussels: European Commission, Directorate-General for Research and Innovation (2015). http://ec.europa.eu/research/consultations/science-2.0/science_2_0_final_report.pdf
European Commission’s Directorate-General for Research & Innovation (RTD): Open Innovation, Open Science and Open to the World (2016). https://ec.europa.eu/digital-single-market/en/news/open-innovation-open-science-open-world-vision-europe
FOSTER: Open Science Definition. https://www.fosteropenscience.eu/foster-taxonomy/open-science-definition
Bechhofer, S., et al.: Why linked data is not enough for scientists. Future Gener. Comput. Syst. 29(2), 99–611 (2013). https://doi.org/10.1016/j.future.2011.08.004. ISSN 0167-739X
Smagorinsky, P.: The method section as conceptual epicenter in constructing social science research reports. Writ. Commun. 25, 389–411 (2008). http://journals.sagepub.com/doi/pdf/10.1177/0741088308317815
Teytelman, L.: We’ve been itching to share this! Integration of GigaScience and protocols.io is an example of how science publishing should work. Protocols.io news (2016). https://www.protocols.io/groups/protocolsio/news/weve-been-itching-to-share-this-integration-of-gigascience
Cotos, E., Huffman, S., Link, S.: A move/step model for methods sections: demonstrating rigour and credibility. Engl. Specif. Purp. 46, 90–106 (2017). https://doi.org/10.1016/j.esp.2017.01.001. ISSN 0889-4906
Center for Open Science: Registered Reports: peer review before results are known to align scientific values and practices. https://cos.io/rr/
FORCE11: Guiding Principles for Findable, Accessible, Interoperable and Re-usable Data Publishing Version B1.0 (2014). https://www.force11.org/fairprinciples
Wilkinson, M.D., et al.: The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data 3, 160018 (2016). https://doi.org/10.1038/sdata.2016.18
Candela, L., Castelli, D., Manghi, P., Tani, A.: Data journals: a survey. J. Assoc. Inf. Sci. Technol. 66, 1747–1762 (2015). https://doi.org/10.1002/asi.23358
Assante, M., Candela, L., Castelli, D., Tani, A.: Are scientific data repositories coping with research data publishing? Data Sci. J. 15, 6 (2016). https://doi.org/10.5334/dsj-2016-006
Mayernik, M.S., Callaghan, S., Leigh, R., Tedds, J., Worley, S.: Peer review of datasets: when, why, and how. Bull. Amer. Meteor. Soc. 96, 191–201 (2015). https://doi.org/10.1175/BAMS-D-13-00083.1
Carpenter, T.A.: What Constitutes Peer Review of Data: a survey of published peer review guidelines (2017). arXiv preprint arXiv:1704.02236. https://arxiv.org/pdf/1704.02236.pdf
Protocols.io team: How to make your protocol more reproducible, discoverable, and user-friendly (2017). http://dx.doi.org/10.17504/protocols.io.g7vbzn6
Tang, A.: ArrayExpress at EMBL-EBI - quality first! Repositive blog (2017). https://blog.repositive.io/arrayexpress-at-embl-ebi-quality-first/
De Roure, D., Goble, C., Stevens, R.: The design and realisation of the myExperiment Virtual Research Environment for social sharing of workflows. Future Gener. Comput. Syst. 25(5), 561–567 (2009). https://doi.org/10.1016/j.future.2008.06.010
Shanahan, D.: A peerless review? Automating methodological and statistical review (2016). https://blogs.biomedcentral.com/bmcblog/2016/05/23/peerless-review-automating-methodological-statistical-review/
Di Leo, A., Risi, E., Biganzoli, L.: No pain, no gain… What we can learn from a trial reporting negative results. Ann. Oncol. 28(4), 678–680 (2017). https://doi.org/10.1093/annonc/mdx065
Borgman, C.L.: Big Data, Little Data, No Data: Scholarship in the Networked World. MIT Press, Cambridge (2015)
Kraker, P., Bachleitner, R., et al.: Deliverable D4.1 – Practices evaluation and mapping: methods, tools and user needs (2017). http://openup-h2020.eu/wp-content/uploads/2017/01/OpenUP_D4.1_Practices-evaluation-and-mapping.-Methods-tools-and-user-needs.pdf
Acknowledgement
This work is partially funded by the EC project OpenUP (H2020-GARRI-2015-1, Grant Agreement: 710722). The content of this work reflects the views of the author(s). The European Commission is not responsible for any use that may be made of the information it contains.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Bardi, A., Casarosa, V., Manghi, P. (2019). Foundations of a Framework for Peer-Reviewing the Research Flow. In: Manghi, P., Candela, L., Silvello, G. (eds) Digital Libraries: Supporting Open Science. IRCDL 2019. Communications in Computer and Information Science, vol 988. Springer, Cham. https://doi.org/10.1007/978-3-030-11226-4_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-11226-4_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-11225-7
Online ISBN: 978-3-030-11226-4
eBook Packages: Computer ScienceComputer Science (R0)