Advertisement

European Radiology

, Volume 28, Issue 9, pp 3632–3639 | Cite as

Reporting bias in imaging: higher accuracy is linked to faster publication

  • A. Dehmoobad Sharifabadi
  • D. A. Korevaar
  • T. A. McGrath
  • N. van Es
  • R. A. Frank
  • L. Cherpak
  • W. Dang
  • J. P. Salameh
  • F. Nguyen
  • C. Stanley
  • M. D. F. McInnes
Experimental
  • 131 Downloads

Abstract

Objectives

The objective of this study was to evaluate whether higher reported accuracy estimates are associated with shorter time to publication among imaging diagnostic accuracy studies.

Methods

We included primary imaging diagnostic accuracy studies, included in meta-analyses from systematic reviews published in 2015. For each primary study, we extracted accuracy estimates, participant recruitment periods and publication dates. Our primary outcome was the association between Youden’s index (sensitivity + specificity − 1, a single measure of diagnostic accuracy) and time to publication.

Results

We included 55 systematic reviews and 781 primary studies. Study completion dates were missing for 238 (30%) studies. The median time from completion to publication in the remaining 543 studies was 20 months (IQR 14–29). Youden’s index was negatively correlated with time from completion to publication (rho = −0.11, p = 0.009). This association remained significant in multivariable Cox regression analyses after adjusting for seven study characteristics: hazard ratio of publication was 1.09 (95% CI 1.03–1.16, p = 0.004) per unit increase for logit-transformed estimates of Youden’s index. When dichotomizing Youden’s index by a median split, time from completion to publication was 20 months (IQR 13–33) for studies with a Youden’s index below the median, and 19 months (14–27) for studies with a Youden’s index above the median (p = 0.104).

Conclusion

Imaging diagnostic accuracy studies with higher accuracy estimates were weakly associated with a shorter time to publication.

Key points

• Higher accuracy estimates are weakly associated with shorter time to publication.

• Lag in time to publication remained significant in multivariate Cox regression analyses.

• No correlation between accuracy and time from submission to publication was identified.

Keywords

Diagnostic test, routine Epidemiology Publication bias Meta-analysis Sensitivity and specificity 

Notes

Funding

This study has received funding by University of Ottawa Department of Radiology Research Stipend Program and University of Ottawa Summer Medical Student Research Program.

Compliance with ethical standards

Guarantor

The scientific guarantor of this publication is Matthew McInnes.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors has significant statistical expertise (Drs. Korevaar and McInnes).

Informed consent

Written informed consent was not required for this study because it evaluated published literature.

Ethical approval

Institutional review board approval was obtained.

Methodology

• retrospective

• cross-sectional

• multicentre study

Supplementary material

330_2018_5354_MOESM1_ESM.docx (24 kb)
ESM 1 (DOCX 23 kb)

References

  1. 1.
    Dwan K, Gamble C, Williamson PR, Kirkham JJ, Group RB (2013) Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS One 8:e66844CrossRefGoogle Scholar
  2. 2.
    de Barra M (2017) Reporting bias inflates the reputation of medical treatments: a comparison of outcomes in clinical trials and online product reviews. Soc Sci Med 2017.  https://doi.org/10.1016/j.socscimed.2017.01.033
  3. 3.
    Archer SW, Carlo WA, Truog WE et al (2016) Improving publication rates in a collaborative clinical trials research network. Semin Perinatol 40:410–417CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Toews I, Glenton C, Lewin S et al (2016) Extent, awareness and perception of dissemination bias in qualitative research: an explorative survey. PLoS One 11:e0159290CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Malički M, Marušić A, OPEN (to Overcome failure to Publish nEgative fiNdings) Consortium (2014) Is there a solution to publication bias? Researchers call for changes in dissemination of clinical research results. J Clin Epidemiol 67:1103–1110CrossRefPubMedGoogle Scholar
  6. 6.
    Page MJ, McKenzie JE, Kirkham J et al (2014) Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev 10:MR000035Google Scholar
  7. 7.
    Driessen E, Hollon SD, Bockting CL, Cuijpers P, Turner EH (2015) Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? A systematic review and meta-analysis of US National Institutes of Health-funded trials. PLoS One 10:e0137864CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Roest AM, de Jonge P, Williams CD, de Vries YA, Schoevers RA, Turner EH (2015) Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: a report of 2 meta-analyses. JAMA Psychiatry 72:500–510CrossRefPubMedGoogle Scholar
  9. 9.
    Song F, Loke Y, Hooper L (2014) Why are medical and health-related studies not being published? A systematic review of reasons given by investigators. PLoS One 9:e110418CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Scherer RW, Ugarte-Gil C, Schmucker C, Meerpohl JJ (2015) Authors report lack of time as main reason for unpublished research presented at biomedical conferences: a systematic review. J Clin Epidemiol 68:803–810CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L (2012) Industry sponsorship and research outcome. Cochrane Database Syst Rev 12:MR000033PubMedGoogle Scholar
  12. 12.
    de Vet H, Eisinga A, Riphagen I, Aertgeerts B, Pewsner D (2008) Chapter 7: Searching for studies. Cochrane handbook for systematic reviews of diagnostic test accuracy. Version 0.4 [updated September 2008]. The Cochrane CollaborationGoogle Scholar
  13. 13.
    Porté F, Uppara M, Malietzis G et al (2017) CT colonography for surveillance of patients with colorectal cancer: systematic review and meta-analysis of diagnostic efficacy. Eur Radiol 27:51–60CrossRefPubMedGoogle Scholar
  14. 14.
    Jarvis D, Mooney C, Cohen J et al (2017) A systematic review and meta-analysis to determine the contribution of MR imaging to the diagnosis of foetal brain abnormalities in utero. Eur Radiol 27:2367–2380CrossRefPubMedGoogle Scholar
  15. 15.
    Mauri G, Sconfienza LM, Pescatori LC et al (2017) Technical success, technique efficacy and complications of minimally-invasive imaging-guided percutaneous ablation procedures of breast cancer: a systematic review and meta-analysis. Eur Radiol 27:3199–3210CrossRefPubMedGoogle Scholar
  16. 16.
    Sevcenco S, Spick C, Helbich TH et al (2017) Malignancy rates and diagnostic performance of the Bosniak classification for the diagnosis of cystic renal lesions in computed tomography - a systematic review and meta-analysis. Eur Radiol 27:2239–2247CrossRefPubMedGoogle Scholar
  17. 17.
    van Dijken BRJ, van Laar PJ, Holtman GA, van der Hoorn A (2017) Diagnostic accuracy of magnetic resonance imaging techniques for treatment response evaluation in patients with high-grade glioma, a systematic review and meta-analysis. Eur Radiol 27:4129–4144CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Yunaga H, Ohta Y, Kaetsu Y et al (2017) Diagnostic performance of calcification-suppressed coronary CT angiography using rapid kilovolt-switching dual-energy CT. Eur Radiol 27:2794–2801CrossRefPubMedGoogle Scholar
  19. 19.
    Korevaar DA, van Es N, Zwinderman AH, Cohen JF, Bossuyt PM (2016a) Time to publication among completed diagnostic accuracy studies: associated with reported accuracy estimates. BMC Med Res Methodol 16:68CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    McInnes MD, Bossuyt PM (2015) Pitfalls of systematic reviews and meta-analyses in imaging research. Radiology 277:13–21CrossRefPubMedGoogle Scholar
  21. 21.
    McGrath TA, McInnes MD, Korevaar DA, Bossuyt PM (2016) Meta-analyses of diagnostic accuracy in imaging journals: analysis of pooling techniques and their effect on summary estimates of diagnostic accuracy. Radiology 281:78–85CrossRefPubMedGoogle Scholar
  22. 22.
    Dressler D, Leswick D (2015) Canadian Association of Radiologists annual scientific meetings: how many abstracts go on to publication? Can Assoc Radiol J 66:96–101CrossRefPubMedGoogle Scholar
  23. 23.
    Dangouloff-Ros V, Ronot M, Lagadec M, Vilgrain V (2015) Analysis of subsequent publication of scientific orally presented abstracts of the French national congress of radiology. Part I: general characteristics. Diagn Interv Imaging 96:461–466CrossRefPubMedGoogle Scholar
  24. 24.
    Loughborough W, Dale H, Wareham JH, Youssef AH, Rodrigues MA, Rodrigues JC (2016) Characteristics and trends in publication of scientific papers presented at the European Congress of Radiology: a comparison between 2000 and 2010. Insights Imaging 7:755–762CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    Shelmerdine SC, Lynch JO, Langan D, Arthurs OJ (2016) Presentation to publication: proportion of abstracts published for ESPR, SPR and IPR. Pediatr Radiol 46:1371–1377CrossRefPubMedGoogle Scholar
  26. 26.
    Korevaar DA, Ochodo EA, Bossuyt PM, Hooft L (2014) Publication and reporting of test accuracy studies registered in ClinicalTrials.gov. Clin Chem 60:651–659CrossRefPubMedGoogle Scholar
  27. 27.
    Core Team R (2017) R: a language and environment for statistical computing. R Foundation for Statistical Computing, ViennaGoogle Scholar
  28. 28.
    IBM (2016) IBM Statistics for Mac. 24 edGoogle Scholar
  29. 29.
    Suñé P, Suñé JM, Montoro JB (2013) Positive outcomes influence the rate and time to publication, but not the impact factor of publications of clinical trial results. PLoS One 8:e54583CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Stern JM, Simes RJ (1997) Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ 315:640–645CrossRefPubMedPubMedCentralGoogle Scholar
  31. 31.
    Song SY, Koo DH, Jung SY, Kang W, Kim EY (2017) The significance of the trial outcome was associated with publication rate and time to publication. J Clin Epidemiol 84:78–84CrossRefPubMedGoogle Scholar
  32. 32.
    Brazzelli M, Lewis SC, Deeks JJ, Sandercock PA (2009) No evidence of bias in the process of publication of diagnostic accuracy studies in stroke submitted as abstracts. J Clin Epidemiol 62:425–430CrossRefPubMedGoogle Scholar
  33. 33.
    Wilson C, Kerr D, Noel-Storr A, Quinn TJ (2015) Associations with publication and assessing publication bias in dementia diagnostic test accuracy studies. Int J Geriatr Psychiatry 30:1250–1256CrossRefPubMedGoogle Scholar
  34. 34.
    Korevaar DA, Cohen JF, Spijker R et al (2016b) Reported estimates of diagnostic accuracy in ophthalmology conference abstracts were not associated with full-text publication. J Clin Epidemiol 79:96–103CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Bossuyt PM, Reitsma JB, Bruns DE et al (2015) STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 351:h5527CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© European Society of Radiology 2018

Authors and Affiliations

  • A. Dehmoobad Sharifabadi
    • 1
  • D. A. Korevaar
    • 2
  • T. A. McGrath
    • 1
  • N. van Es
    • 3
  • R. A. Frank
    • 1
  • L. Cherpak
    • 1
  • W. Dang
    • 1
  • J. P. Salameh
    • 4
  • F. Nguyen
    • 1
  • C. Stanley
    • 5
  • M. D. F. McInnes
    • 6
  1. 1.Department of Radiology-Faculty of MedicineUniversity of OttawaOttawaCanada
  2. 2.Department of Clinical Epidemiology, Biostatistics and BioinformaticsAcademic Medical CenterAmsterdamThe Netherlands
  3. 3.Department of Vascular MedicineAcademic Medical CenterAmsterdamThe Netherlands
  4. 4.School of Epidemiology and Public HealthUniversity of OttawaOttawaCanada
  5. 5.Dalhousie UniversityHalifaxCanada
  6. 6.Department of Radiology, Clinical Epidemiology Program, Ottawa Hospital Research InstituteUniversity of OttawaOttawaCanada

Personalised recommendations