Costantino et al. [1] analyzed 125 original articles published in the three most important and influential medical journals. The Authors find that 102 of them (82 %) contained some kind of errors, even multiple and serious in five cases (4 %). This paper has indeed some weaknesses. First, the Authors considered only three medical journals, as representative of all medical journals. Second, although they state that the criteria used to identify the errors were objective, of necessity there is some amount of subjectivity in the detection of errors. To overcome this problem the Authors flagged an article only in case of agreement between two reviewers. For this reason, subjectivity leads to the underestimation of the errors. However, except for these considerations, the Costantino et al. paper is the first work that critically analyzes a substantial amount of relevant literature to check for the presence of errors. The reported weaknesses of this paper do not affect the importance of the results obtained through this research. What shall we do now? Throw away all the medical literature and replace evidence based medicine with evidence biased medicine? Are the errors retrieved so trivial so that they do not reduce the validity of the studies’ results so that we could ignore this study?

Let’s start from the beginning. The main aim of the scientific literature should be at the improvement of medical knowledge, the provision of information to clinicians to support their clinical practice, and to help produce a better management of their patients. Both Editors and the peer review system have a very important role in this context. They should protect clinicians, and consequently patients, against unproven information.

The scientific literature plays a fundamental role in medical education. During their continuing medical education and professional development, clinicians should also rely on the information published in scientific journals. Before deciding whether to give a new drug to their patients, clinicians should critically appraise the published articles, assessing both the internal and the external validity of the studies. This assessment should be based on reliable information: the results (i.e., figures) reported in the published papers should be reliable and consistent. What happens if the editors and the peer review system do not protect clinicians from unproven information, as the data reported by Costantino et al. seem to suggest? I don’t think that the message to take home is that the results from the 125 papers analyzed are to be considered false or misleading. The focus is not on the single study analyzed, but rather on the process of publication as a whole. The errors reported by Costantino et al. should be considered as near miss. The errors retrieved do not affect the study conclusions of the original papers. However, the large amount of errors seems to underscore a system frailty that has to be dealt with.

Some Authors (namely G.C. and P.D.) are active members of the Cochrane Collaboration or are well known in the academic world, but some others, who probably contributed to check the studies for errors, are young doctors with few publications. The question is: if a young doctor is able to identify at least one error in 80 % of published articles why is the peer review system not able to do the same? The authors find that 4 % of the articles reported some numbers in the abstract different from those reported in the text. What level of expertise is needed to recognize this kind of error?

We see the problem, but what is the solution? I think the time has come for a discussion about the editorial and peer review system. The results of a scientific study should be published to improve the patients’ condition. Unfortunately, too often economic or academic interests seem to lead in the decision to publish a paper. Moreover, even the most important journals seem to be hostage to this kind of system. There are many ways to check for errors before a paper is published. One possible solution is to increase the number of filters during the editorial process (see Reason’s Swiss cheese theory [2]). For such a purpose, there are probably plenty of young doctors who would be happy to assist in the editorial process of many journals. Another possible action would be to report the names of the editors and the reviewers of a paper, to augment their responsibility. Finally, the transparency of the whole reviewing process is open to improvement, as many open access journals are trying to do. [3].

When aiming at complete transparency, a relevant point has got to be kept in mind. As per certain information obtained from the Authors themselves, during the peer reviewing process of this paper when the Authors sent a pre-submission inquiry of the manuscript to one of the selected journals, the editorial board asked them for a detailed list of the retrieved errors. After including the amendments indicated, without asking for permission, the board decided not to publish this work, justifying this decision as it would have been “not right for the Journal”.

The paper by Costantino shows that there seems to be a lack of internal coherence in the studies analyzed. I’m aware that fraud [4] is much more important than errors, and that there are no methodologies used to identify fraudulent studies. Nevertheless, we should try our best to publish scientific reports with at least a high degree of internal coherence.

In conclusion, I think that the study by Costantino et al. does not undermine the foundation of the scientific literature: none of the errors they retrieved would have changed the study results. The authors addressed only objective errors, while the most important errors are often “subjective”, depending on the reader’s expertise, and not easily detectable and as a consequence any possible analysis can only be too superficial to be completely reliable. I do not know if the articles with more errors are those that are also more prone to give wrong information and results, but they certainly underline a weakness of the editorial and peer review system. The solution, if there be any, is not simple. However, I believe that recognizing the problem is the beginning of the solution.

One should note that, on the basis of the considerations by Costantino et al. there may be at least three errors in their own very paper!