Citations and science
- 454 Downloads
The International Journal of Clinical Pharmacy is included in the Thomson Reuters Science Citation Index (SCI), part of the Thomson Reuters Scientific Databases and since 2016 managed by Clarivate Analytics. This SCI was first introduced in 1964 by Eugene Garfield as a means to evaluate the quality of research publications, by counting the frequency that specific articles are cited in a selection of journals. The journals are grouped by the field they cover, and citation frequencies are calculated within these groups. The more articles from a certain journal are cited amongst peers (divided by the number of articles appearing in the journal), the higher the impact of the journal will be. A number of measures have been taken to prevent the Impact Factors (IFs) from being influenced deliberately, such as limiting self-cites etc. Nevertheless, highly ranked journals also prove to be very selective , often looking for the newest and sometimes most controversial research, in the hope of high future citations. This boosts the impact factor, but in fact, highly ranked journals have also been observed to have lower power , lower power in turn meaning more potential for controversial false positives  and higher retraction rates .
The Science Citation Index contains a selection of the most highly cited, highest impact journals in each category and the journals selected in the SCI are considered the top journals in their subject category. So, it does not reflect every published citation in a subject group. Thomson Reuters/Clarivate decides which journals are selected for the Index, and for which category. Although it has never been designed as such, the annual reports of the SCI are used as tool to assign funding to study-groups or institutions . It also is used in scientific institutions to give a score to the research quality of researchers or research departments. This latter use is totally inappropriate and unacceptable, also because non-English language journals are not included.
Another limitation of the Impact Factor is that disciplines have different rates of citations and disciplines move faster or slower. Humanities journals in the Social Sciences Citation Index (SSCI) have some of the lowest citation rates, but some of the greatest longevity. However, even within disciplines, some sub-disciplines can move faster or slower. As an attempt to moderate this problem, ranking or quartile within a Subject Category can be used as a crude attempt to control different citation rates. That is, if a journal is top-ranked/in the first quartile of journals in a similar subject area, then journals in different disciplines, while having large discrepancies in Impact Factor, can be crudely compared (for example, first quartiles IF cut-offs for the following subject categories: Oncology 4.7; Pharmacology and Pharmacy 3.4; Plant Sciences 2.6; General and Internal Medicine 2.3; Cultural Studies 1.0).
Pharmacology papers will be cited by researchers on pharmacology and pharmacotherapy, as well as the medical community at large. This boosts the impact factors of such papers and journals.
Pharmacy practice is relatively underrepresented within the journal group, automatically leading to lower citation frequencies, and thus low impact factors for pharmacy practice related journals.
Traditionally, citation scores are calculated over a period of the past two full years, but the speed at which papers are cited differs between disciplines. For fast developing research (such as cancer research) this leads to an overestimation of impact. A high impact factor, based on many recent citations, is an indicator of the cutting edge, rather than long term impact . For pharmacy practice research, that almost per definition studies practice in a setting that is difficult to control, such a 2 year impact factor is not very significant because the time lapses between research that stands on the shoulders of previous research will be longer. A 5 year impact factor would do more justice to the authors and the journals.
In this issue, Minguet et al.  discuss a possible new grouping of the subject category of which our journal also is a part. This approach, based on MESH coding, could also be interesting for all other categories within the Index. According to their calculations, dividing the subject category ‘‘Pharmacology and Pharmacy’ into basic pharmacology, clinical pharmacology, and pharmacy would have an impact on the distribution of the journal IFs’ quartiles. This could have major implications for authors, and also on the ability to obtain research funding given that several funding agencies evaluate researchers almost solely based on journals’ IF and quartile distribution of their publications.
Academic institutions somehow seem to remain stuck in the old fashioned IFs of the SCI. The question is if this is going to last much longer. Many researchers and authors start moving away from the SCI, because Clarivate Analytics apparently does not want to change, and correct the misconceptions about their IFs in the scientific community. In the Excellence in Research for Australia (ERA) scheme (similar to Britain’s REF/RAE), journal ranking has been removed entirely, in favour of a descriptive approach . New, more flexible and responsive, bibliographic metrics have been developed such as Google Scholar Metrics which uses 5 years of citations, and has by the far the broadest scope in terms of citation sources (including mentions of papers in blogs, newspapers etc.). Scopus/Elsevier CiteScore is another new alternative with a better coverage of pharmacy journals, and a 3 year citation window. The CiteScore platform also includes several other metrics (Sci-Mago, SNIP) developed by external parties. The SNIP—Source Normalised Impact per Paper—is likely more accurate, because, as the name indicates, it attempts to more carefully account for citation differences between fields. That is, in a field with a low overall number of citations, a single citation would count for much more; and the corollary, in a field with high citations, a single citation counts for much less. The Elsevier platform also maintains separate Pharmacy and Pharmacology subject categories. IJCP is categorised in both, and fares much better with the SNIP than Elsevier’s CiteScore (similar to the JCR IF).
Article level metrics have also been proposed as a solution, in particular de-emphasising the selectivity of work. The most well-known of these is the Altmetric, which includes citations, but also news and blog sites, as well as social media mentions. However, the Altmetric can be quite influenced by Twitter, and many Twitter mentions seem to be either very mechanical, or even automated [9, 10].
None of these alternative metrics is yet well established as an alternative, but they have the potential to be more adaptive, accurate and inclusive than the Clarivate Analytic’s Impact Factor. Clarivate Analytics is a very closed organisation; their criteria for assigning groupings, or accepting journals into the Index are not always very clear, and even the way the IF is calculated is not fully transparent . In this case, however, we would be very interested in seeing a response to the paper of Minguet et al., because they discuss an issue that is very important for research level assessment and funding at large, and not only for practice related research in the field of pharmacy. Without adaption and change, Clarivate risks being left behind.
- 6.Leydesdorff L, Bornmann L, Comins JA, Milojević S. Citations: Indicators of quality? The impact fallacy. Front Res Metr Anal. 2016. doi: 10.3389/frma.2016.00001.
- 8.Creagh S. Journal rankings ditched: the experts respond 2011. https://theconversation.com/journal-rankings-ditched-the-experts-respond-1598.