Methodological quality in clinical trials and bibliometric indicators: no evidence of correlations
- 346 Downloads
Citation frequencies and journal impact factors (JIFs) are being used more and more to assess the quality of research and allocate research resources. If these bibliometric indicators are not an adequate predictor of research quality, there could be severe negative consequences for research. To analyse to which extent citation frequencies and journal impact factors correlate with the methodological quality of clinical research articles included in an SBU systematic review of antibiotic prophylaxis in surgery. All 212 eligible original articles were extracted from the SBU systematic review “Antibiotic Prophylaxis in Surgery” and categorized according to their methodological rigourness as high, moderate or low quality articles. Median of citation frequencies and JIFs were compared between the methodological quality groups using Kruskal–Wallis non-parametric test. An in-depth study of low-quality studies with higher citation frequencies/JIFs was also conducted. No significant differences were found in median citation frequencies (p = 0.453) or JIFs (p = 0.185) between the three quality groups. Studies that had high citation frequencies/JIFs but were assessed as low-quality lacked control groups, had high dropout rates or low internal validity. This study of antibiotic prophylaxis in surgery does not support the hypothesis that bibliometric indicators are a valid instrument for assessing methodological quality in clinical trials. This is a worrying observation, since bibliometric indicators have a major influence on research funding. However, further studies in other areas are needed.
KeywordsBibliometric analysis Citations Impact factor Study quality HTA
Conflict of Interest
All authors declare no conflict of interest.
- Akre, O., Barone-Adesi, F., Pettersson, A., Pearce, N., Merletti, F., & Richiardi, L. (2011). Differences in citation rates by country of origin for papers published in top-ranked medical journals: do they reflect inequalities in access to publication? Journal of Epidemiology and Community Health, 65(2), 119–123.CrossRefGoogle Scholar
- Andersen, J., Belmont, J., & Cho, C. T. (2006). Journal impact factor in the era of expanding literature. Journal of Microbiology, Immunology, and Infection, 39(6), 436–443.Google Scholar
- Bhandari, M., Busse, J., Devereaux, P. J., Montori, V. M., Swiontkowski, M., Tornetta Iii, P., et al. (2007). Factors associated with citation rates in the orthopedic literature. Canadian Journal of Surgery, 50(2), 119–123.Google Scholar
- Cho, J. E., Cho, C. T., & Belmont, J. M. (1998). Learning to assess the value of infectious disease journals. Journal of Microbiology, Immunology, and Infection, 31, 1–4.Google Scholar
- du Prel, J. B., Rohrig, B., & Blettner, M. (2009). Critical appraisal of scientific articles: part 1 of a series on evaluation of scientific publications. Deutsches Ärzteblatt International, 106(7), 100–105.Google Scholar
- Guyatt, G. H., Sackett, D. L., & Cook, D. J. (1993). Users’ guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. Journal of the American Medical Association, 270(21), 2598–2601.CrossRefGoogle Scholar
- Guyatt, G. H., Sackett, D. L., & Cook, D. J. (1994). Users’ guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. Journal of the American Medical Association, 271(1), 59–63.CrossRefGoogle Scholar
- SBU. (2010). Antibiotic prophylaxis for surgical procedures. A systematic review. Stockholm: Statens beredning för medicinsk utvärdering (SBU); SBU-rapport nr 200. ISBN 978-91-85413-36–2.Google Scholar