Citations in scientific articles: possibly biased reflections on the field of diagnostic imaging
This Editorial Comment refers to the article “Citation Bias in Imaging Research: Are studies with higher diagnostic accuracy estimates cited more often?” by Frank RA, Sharifabadi AD, Salameh JP et al, Eur Radiol. 2018 Nov 15. https://doi.org/10.1007/s00330-018-5801-8 .
Citation biases, a distortion of the scientific record by preferential citation mainly of “desirable” results, and other reporting biases  in the scientific and medical literature [3, 4] are very well recognised. Despite the general awareness of their potentially damaging effects, authors can be still incentivised to committing citation biases either intentionally or unintentionally. Incentives to practise preferential citation might be particularly imminent for busy (clinical) scientists under the constant pressure of impact-oriented publishing and the need to attract further research funding: negative results do not sell too well.
Frank RA, Sharifabadi AD, Salameh JP et al  have now analysed to what extent the field of diagnostic imaging research may be affected by citation biases. Based on a bibliometric analysis of 1016 primary studies, they observed moderate preferential citation rates of studies with higher diagnostic accuracies (particularly higher sensitivities). They conclude that such bias could result in possible harm to patients by overestimation of the performance of diagnostic imaging tests in clinical practice. There are, however, further potential effects related to citation habits beyond what could be assessed when adopting a straightforward definition of citation bias such as in this study .
On the side of authors, one might first think of early career scientists and risk of formal citation errors due to a lack of experience. However, I would like to focus on more experienced authors who are well aware of the protagonists in their fields and who have acquired skills for focused and goal-directed writing during their scientific careers. Although a goal-directed writing style has many benefits, it might also bear the risk of inducing relevant citation biases. For instance, when applying for grant funding, senior authors may be incentivised to refer to literature that has been frequently cited and thus appears more impactful. Additionally, we may sometimes be inclined to search specifically for the most convincing references supporting our own hypotheses and claims (either explicitly or as an implicit confirmation bias). Some scientists explicitly prioritise citations as a “vote” on published articles or define good storytelling (see  for a critical reflection) as a quality criterion for a high level of scientific writing, which may unintentionally impede an unbiased overview of a particular scientific field. Potential further biases are linked to the journals’ impact (including the potentially erroneous assumption to mistake journal reputation or impact factors as signs of methodological rigor of an individual article), the list of authors and institutional affiliations which might be used as indicators of authority and credibility of results. Further insights into potentially resulting biases can be gained from more advanced bibliometric studies of “citation networks” .
Further citation-related biases beyond preferential citation of positive findings as addressed by Frank RA, Sharifabadi AD, Salameh JP et al  may result from deviations from the original message, cascade citations (i.e. citation of a paper that only cites a previous one) as well as empty shell citations (when the cited paper does not contain at all the stated message). They may mutually increase biased views on the diagnostic performance of medical imaging techniques. My personal impression is that deviations from the original message (for example inadvertently citing a negative result as a positive finding, a minor or questionable result as a strong finding or a merely statistically significant result as clinically relevant) seem to be underappreciated in our field. The assumption carried by citations that diagnostic tests successful in a small, homogeneous, highly selected pilot sample might for instance translate to real world clinical populations is also of particular concern. Supposedly, studies serving certain “buzzwords” such as artificial intelligence may also be more highly cited, at least if they report positive results. It must be feared that other citation biases might also be increased in areas currently receiving particularly high levels of attention.
Journal and funding body policies may contribute to citation biases as well, e.g. by incentivising the submission of success stories, potentially in the legitimate interest of their own reputation, and more directly by including reference lists in word count limits or explicit limits on the maximum number of references (cited evidence in this comment can also only be exemplary). It has been observed that even policies such as an alphabetically ordered reference list can influence the number of future citations . Journal editors (and reviewers) may be tempted to ask authors for self-citations .
The increasing use of social media as an academic tool (as of now not as prevalent in radiology as in other disciplines), for instance when principal investigators or “influencers” highlight their labs’ or others’ work on Twitter, may impact citation rates in the “classical” peer-reviewed body of scientific literature  including the risk of “filter bubbles”.
Despite this mostly critical view, preferential citation of positive results may also serve a desirable role: highlighting successful methodological work in a certain sub-field may indeed help advance that field by guiding academic and technological workforce towards promising methods. Furthermore, preferential citation may simply reflect the fact that subsequent work follows a successful direction, in which case it would not be a true bias.
In conclusion, there is evidence of a citation bias in diagnostic imaging research . Preferential citation of studies with higher accuracies can additionally reflect consecutive work or can be sparingly used to highlight successful work. Suggestions on how we could avert the risk of citation-related biases are to incentivise brief systematic literature overviews within articles reporting original research or to preferentially cite balanced reviews in paper introductions (instead of selectively highlighting most impressive previous research), to trace back key messages to the original sources, to communicate the uncertainty of cited findings, to try and overcome questionable citation and literature research habits (as outlined above). Most importantly, however, readers and everybody involved in scientific publishing not only need to be aware of the different kinds of citation-related bias, but should try to keep them as low as possible so that they will not eventually result in harm to patients or the healthcare system.
I would like to thank Prof. Yves Menu (Paris), Prof. Bettina Pfleiderer (Münster), Dr. David Herr (Berlin) and Dr. David Mehler (Münster, Cardiff) for insightful discussions on citation-related biases from different perspectives and career stages. I hope that the thoughts presented here will therefore be less biased by my own perspective as a postdoctoral researcher and young radiologist.
The author states that this work has not received any funding.
Compliance with ethical standards
The scientific guarantor of this publication is Prof. Dr. Dr. Bettina Pfleiderer, Münster.
Conflict of interest
The author of this manuscript declares no relationships with any companies, whose products or services may be related to the subject matter of the article.
Statistics and biometry
No complex statistical methods were necessary for this paper.
Informed consent was not required for this Editorial Comment.
Institutional Review Board approval was not required for this Editorial Comment.
- 1.Frank RA, Sharifabadi AD, Salameh JP et al (2018) Citation Bias in Imaging Research: Are studies with higher diagnostic accuracy estimates cited more often? Eur Radiol. https://doi.org/10.1007/s00330-018-5801-8
- 2.Higgins JP, Green S (2008) Cochrane handbook for systematic reviews of interventions: Cochrane Book Series. Wiley-Blackwell, ChichesterGoogle Scholar
- 3.Jannot AS, Agoritsas T, Gayet-Ageron A, Perneger TV (2013) Citation bias favoring statistically significant studies was present in medical research. J Clin Epidemiol 66:296–301. https://doi.org/10.1016/j.jclinepi.2012.09.015
- 4.Fanelli D, Costas R, Ioannidis JP (2017) Meta-assessment of bias in science. Proc Natl Acad Sci U S A 114:3714–3719. https://doi.org/10.1073/pnas.1618569114
- 6.Greenberg SA (2009) How citation distortions create unfounded authority: analysis of a citation network. BMJ 339:b2680. https://doi.org/10.1136/bmj.b2680
- 7.Stevens JR, Duque JF (2018) Order matters: alphabetizing in-text citations biases citation rates. Psychon Bull Rev. https://doi.org/10.3758/s13423-018-1532-8
- 9.Peoples BK, Midway SR, Sackett D, Lynch A, Cooney PB (2016) Twitter predicts citation rates of ecological research. PLoS One. https://doi.org/10.1371/journal.pone.0166570