Assessing Review Reports of Scientific Articles: A Literature Review
Computational support has been applied in different stages for automation of the peer review process, such as reviewer assignment to the article, review of content of the scientific article, detection of plagiarism and bias, all applying Machine Learning (ML) techniques. However, there is a lack of studies which identify the instruments used to evaluate the reviewers’ reports. This systematic literature review aims to find evidence about which techniques have been applied in the assessment of the reviewers’ reports. Therefore, six online databases were evaluated, in which 55 articles were identified, all published since 2000, meeting the inclusion criteria of this review. The result shows 6 relevant studies, which address models of assessment of scientific article reviews. Nevertheless, the use of ML was not identified in any case. Therefore, our findings demonstrate that there are a few instruments used to assess the reviewers’ reports and furthermore, they cannot be reliably used to extensively automate the review process.
KeywordsSystematic literature review Peer review Assessment Reviewers’ report
We appreciate the financial support of AISTI (Iberian Association for Information Systems and Technologies), which permitted the registration in the WorldCIST’18 (6th World Conference on Information Systems and Technologies), held in Naples, Italy, 27–29 March 2018, and consequently this publication.
- 1.Chauvin, A., Moher, D., Altman, D., Schriger, D.L., Alam, S., Hopewell, S., Shanahan, D.R., Recchioni, A., Ravaud, P., Boutron, I.: A protocol of a cross-sectional study evaluating an online tool for early career peer reviewers assessing reports of randomised controlled trials. BMJ Open 7, 10 (2017)CrossRefGoogle Scholar
- 7.Burley, R., Moylan, E.: What might peer review look like in 2030? (2017)Google Scholar
- 18.Tennant, J.P., Dugan, J.M., Graziotin, D., Jacques, D.C., Waldner, F., Mietchen, D., Elkhatib, Y., B. Collister, L., Pikas, C.K., Crick, T., Masuzzo, P., Caravaggi, A., Berg, D.R., Niemeyer, K.E., Ross-Hellauer, T., Mannheimer, S., Rigling, L., Katz, D.S., Greshake Tzovaras, B., Pacheco-Mendoza, J., Fatima, N., Poblet, M., Isaakidis, M., Irawan, D.E., Renaut, S., Madan, C.R., Matthias, L., Nørgaard Kjær, J., O’Donnell, D.P., Neylon, C., Kearns, S., Selvaraju, M., Colomb, J.: A multi-disciplinary perspective on emergent and future innovations in peer review. F1000Research 6, 64 (2017)CrossRefGoogle Scholar