Advertisement

Can authorship bias be detected in meta-analysis?

  • Ahmed M. Abou-SettaEmail author
  • Rasheda Rabbani
  • Lisa M. Lix
  • Alexis F. Turgeon
  • Brett L. Houston
  • Dean A. Fergusson
  • Ryan Zarychanski
Reports of Original Investigations
First Online:
  • 17 Downloads

Abstract

Purpose

Statistical approaches have been developed to detect bias in individual trials, but guidance on how to detect systematic differences at a meta-analytical level is lacking. In this paper, we elucidate whether author bias can be detected in a cohort of randomized trials included in a meta-analysis.

Methods

We utilized mortality data from 35 trials (10,880 patients) included in our previously published meta-analysis. First, we linked each author with their trial (or trials). Then we calculated author-specific odds ratios using univariate cross table methods. Finally, we tested the effect of authorship by comparing each author’s estimated odds ratio with all other pooled estimated odds ratios using meta-regression.

Results

The median number of investigators named as authors on the primary trial reports was six (interquartile range: 5-8, range: 2-32). The results showed that the slope of author effect for mortality ranged from − 1.35 to 0.71. We identified only one author team showing a marginally significant effect (− 0.39; 95% confidence interval, − 0.78 to 0.00). This author team has a history of retractions due to data manipulations and ethical violations.

Conclusion

When combining trial-level data to produce a pooled effect estimate, investigators must consider sources of potential bias. Our results suggest that systematic errors can be detected using meta-regression, although further research is needed to examine the sensitivity of this model. Systematic reviewers will benefit from the availability of methods to guard against the dissemination of results with the potential to mislead decision-making.

Les biais liés aux auteurs peuvent-ils être détectés dans une méta-analyse?

Résumé

Objectif

Des approches statistiques ont été élaborées pour détecter les biais dans les essais individuels, mais nous manquons d’orientations sur les méthodes à utiliser pour les détecter au niveau des méta-analyses. Dans cet article, nous étudions s’il est possible de détecter des biais liés à l’auteur dans un ensemble d’essais randomisés inclus dans une méta-analyse.

Méthodes

Nous avons utilisé les données sur la mortalité tirées de 35 essais (10 880 patients) inclus dans notre méta-analyse publiée antérieurement. Nous avons tout d’abord lié chaque auteur à son étude (ou à ses études). Nous avons ensuite calculé des rapports de cotes (odds ratios) spécifiques utilisant des méthodes de tableaux unifactoriels croisés. Enfin, nous avons testé l’effet « auteur » en comparant les rapports de cotes estimés de chaque auteur avec le rapport de cotes groupé de tous les autres auteurs au moyen d’une métarégression.

Résultats

Le nombre médian d’investigateurs cité comme auteurs dans les publications principales des essais était de six (plage interquartile : 5 à 8, limites : 2 à 32). Les résultats ont montré que la pente de l’effet « auteur » pour la mortalité allait de -1,35 à 0,71. Nous n’avons identifié qu’une seule équipe d’auteurs ayant un effet peu à la limite de la significativité (-0,39; intervalle de confiance à 95 % : -0,78 à 0,00). Cette équipe a un historique de rétractions de publications en raison de manipulations des données et de violations de l’éthique.

Conclusion

Lorsqu’ils combinent les données des essais pour produire une estimation groupée de l’effet, les investigateurs doivent envisager les sources de biais potentiels. Nos résultats suggèrent que des erreurs systématiques peuvent être détectées en utilisant une métarégression bien qu’il soit nécessaire de poursuivre les recherches pour évaluer la sensibilité de ce modèle. Les réviseurs systématiques tireront parti de la disponibilité de méthodes les protégeant de la dissémination de résultats susceptibles de fausser des prises de décision potentielles.

This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

Ryan Zarychanski and Alexis F. Turgeon receive salary support and operating funds from the Canadian Institutes of Health Research (CIHR). Lisa Lix is supported by a Manitoba Research Chair from Research Manitoba and a Foundation Scheme grant from CIHR. These funding agencies had no role in the design or conduct of the study, including but not limited to, study identification, collection, management, analysis, and interpretation of the data, or preparation, review, or approval of the final report.

Conflicts of interest

None declared.

Editorial responsibility

This submission was handled by Dr. Gregory L. Bryson, Deputy Editor-in-Chief, Canadian Journal of Anesthesia.

Author contributions

Ahmed Abou-Setta, Rasheda Rabbani, Lisa Lix, and Ryan Zarychanski contributed substantially to all aspects of this manuscript, including study conception and design; acquisition, analysis, and interpretation of data; and drafting the article. Alexis Turgeon, Brett Houston, and Dean Fergusson contributed substantially to the interpretation of data.

References

  1. 1.
    Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions. Version 5.1. 0. The Cochrane Collaboration 2011. Available from URL: https://handbook-5-1.cochrane.org/ (accessed October 2018).
  2. 2.
    Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev 2012; 12: MR000033.Google Scholar
  3. 3.
    Bastian H. “They would say that, wouldn’t they?” A reader’s guide to author and sponsor biases in clinical research. J R Soc Med 2006; 99: 611-4.Google Scholar
  4. 4.
    Buchter RB, Pieper D. Most overviews of Cochrane reviews neglected potential biases from dual authorship. J Clin Epidemiol 2016; 77: 91-4.CrossRefGoogle Scholar
  5. 5.
    Singh JP, Grann M, Fazel S. Authorship bias in violence risk assessment? A systematic review and meta-analysis. PLoS One 2013; 8: e72484.CrossRefGoogle Scholar
  6. 6.
    Fanelli D. Do pressures to publish increase scientists’ bias? An empirical support from US States Data. PLoS One 2010; 5: e10271.CrossRefGoogle Scholar
  7. 7.
    Baerlocher MO, O’Brien J, Newton M, Gautam T, Noble J. Data integrity, reliability and fraud in medical research. Eur J Intern Med 2010; 21: 40-5.CrossRefGoogle Scholar
  8. 8.
    Carlisle JB. The analysis of 168 randomised controlled trials to test data integrity. Anaesthesia 2012; 67: 521-37.CrossRefGoogle Scholar
  9. 9.
    Pirracchio R, Resche-Rigon M, Chevret S, Journois D. Do simple screening statistical tools help to detect reporting bias? Ann Intensive Care 2013; 3: 29.CrossRefGoogle Scholar
  10. 10.
    Hein J, Zobrist R, Konrad C, Schuepfer G. Scientific fraud in 20 falsified anesthesia papers : detection using financial auditing methods. Anaesthesist 2012; 61: 543-9.CrossRefGoogle Scholar
  11. 11.
    Mendel R, Traut-Mattausch E, Jonas E, et al. Confirmation bias: why psychiatrists stick to wrong preliminary diagnoses. Psychol Med 2011; 41: 2651-9.CrossRefGoogle Scholar
  12. 12.
    Munro GD, Stansbury JA. The dark side of self-affirmation: confirmation bias and illusory correlation in response to threatening information. Pers Soc Psychol Bull 2009; 35: 1143-53.CrossRefGoogle Scholar
  13. 13.
    Jonas E, Schulz-Hardt S, Frey D, Thelen N. Confirmation bias in sequential information search after preliminary decisions: an expansion of dissonance theoretical research on selective exposure to information. J Pers Soc Psychol 2001; 80: 557-71.CrossRefGoogle Scholar
  14. 14.
    Goodyear-Smith FA, van Driel ML, Arroll B, Del Mar C. Analysis of decisions made in meta-analyses of depression screening and the risk of confirmation bias: a case study. BMC Med Res Methodol 2012; 12: 76.CrossRefGoogle Scholar
  15. 15.
    Pines JM. Profiles in patient safety: confirmation bias in emergency medicine. Acad Emerg Med 2006; 13: 90-4.CrossRefGoogle Scholar
  16. 16.
    Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 2003; 289: 454-65.CrossRefGoogle Scholar
  17. 17.
    Djulbegovic B, Lacevic M, Cantor A, et al. The uncertainty principle and industry-sponsored research. Lancet 2000; 356: 635-8.CrossRefGoogle Scholar
  18. 18.
    Trelle S, Reichenbach S, Wandel S, et al. Cardiovascular safety of non-steroidal anti-inflammatory drugs: network meta-analysis. BMJ 2011; 342: c7086.CrossRefGoogle Scholar
  19. 19.
    Ross JS, Hill KP, Egilman DS, Krumholz HM. Guest authorship and ghostwriting in publications related to rofecoxib: a case study of industry documents from rofecoxib litigation. JAMA 2008; 299: 1800-12.CrossRefGoogle Scholar
  20. 20.
    EIC Joint Statement. Editors-in-Chief Statement Regarding Published Clinical Trials Conducted without IRB Approval by Joachim Boldt - 2011. Available from URL: https://www.springer.com/cda/content/document/cda_downloaddocument/EIC+Joint+Statement+_20110210.pdf?SGWID=0-0-45-1077337-p173850204 (accessed October 2018).
  21. 21.
    Shafer SL. Shadow of doubt. Anesth Analg 2011; 112: 498-500.CrossRefGoogle Scholar
  22. 22.
    Abritis A. Retraction Watch - 2015. Available from URL: http://retractionwatch.com/2015/10/12/boldts-retraction-count-upped-to-94-co-author-takes-legal-action-to-prevent-95th/ (accessed October 2018).
  23. 23.
    Myburgh JA, Finfer S, Bellomo R, et al. Hydroxyethyl starch or saline for fluid resuscitation in intensive care. N Engl J Med 2012; 367: 1901-11.CrossRefGoogle Scholar
  24. 24.
    Perner A, Haase N, Guttormsen AB, et al. Hydroxyethyl starch 130/0.42 versus Ringer’s acetate in severe sepsis. N Engl J Med 2012; 367: 124-34.Google Scholar
  25. 25.
    Zarychanski R, Abou-Setta AM, Turgeon AF, et al. Association of hydroxyethyl starch administration with mortality and acute kidney injury in critically ill patients requiring volume resuscitation: a systematic review and meta-analysis. JAMA 2013; 309: 678-88.CrossRefGoogle Scholar
  26. 26.
    Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med 2002; 21: 1513-24.CrossRefGoogle Scholar
  27. 27.
    Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Meta-regression. In: Borenstein M, Hedges LV, Higgins JP, Rothstein HR (eds).Introduction to Meta-Analysis: John Wiley & Sons, Ltd; 2009: 187-203.Google Scholar
  28. 28.
    Guyatt G, Rennie D. Users’ Guides to the Medical Literature: a Manual for Evidence-Based Clinical Practice. AMA press Chicago; 2002.Google Scholar
  29. 29.
    Carlisle J, Pace N, Cracknell J, Moller A, Pedersen T, Zacharias M. What should the Cochrane Collaboration do about research that is, or might be, fraudulent? Cochrane Database Syst Rev 2013; 5: ED000060.Google Scholar
  30. 30.
    Mutter TC, Ruth CA, Dart AB. Hydroxyethyl starch (HES) versus other fluid therapies: effects on kidney function. Cochrane Database Syst Rev 2013; 7: CD007594.Google Scholar

Copyright information

© Canadian Anesthesiologists' Society 2019

Authors and Affiliations

  1. 1.George and Fay Yee Centre for Healthcare InnovationUniversity of Manitoba/Winnipeg Regional Health AuthorityWinnipegCanada
  2. 2.Department of Community Health Sciences, Max Rady College of Medicine, Rady Faculty of Health SciencesUniversity of ManitobaWinnipegCanada
  3. 3.Division of Critical Care Medicine, Department of Anesthesiology and Critical Care MedicineCentre de recherche CHU de Québec - Université Laval, Population Health and Optimal Health Practice Research Unit, Université LavalQuébec CityCanada
  4. 4.Department of Internal Medicine, Max Rady College of Medicine, Rady Faculty of Health SciencesUniversity of ManitobaWinnipegCanada
  5. 5.Clinical Epidemiology Program, Department of Medicine, Ottawa Hospital Research InstituteUniversity of OttawaOttawaCanada

Personalised recommendations

Cite article

Buy options