, Volume 114, Issue 2, pp 593–603 | Cite as

Usage metrics vs classical metrics: analysis of Russia’s research output

  • Valentina MarkusovaEmail author
  • Valentin Bogorov
  • Alexander Libkind


This paper discusses the results of a pilot project investigating Russian scholarly publications using the altmetric indicators “Usage Count Last 180 days” (U1) and “Usage Count Since 2013” (U2) introduced by Web of Science. We explored the relationship between citation impact and both types of usage counts. The data set consisted of 37,281 records (publications) indexed by SCI-E in 2015. Seven broad research areas were selected to observe citation patterns and usage counts. A significant difference was found between mean citations and mean usage counts (U2) in a few research areas. We discovered a significant Kendall rank correlation between the citation metrics and usage metrics at the article level. This correlation is particularly strong for the longer period usage metric (U2). We also analyzed the relationship between usage metrics and traditional journal-level citation metrics. Very weak correlation was observed.


Citations Web of Science usage counts Kendall correlation Impact factor Russia 



The authors are very grateful to Prof. W. Glanzel for his support and feedback on how to improve our work. We express our gratitude to Diane Gal for her enormous editing work. This project was partly supported by the Russian Foundation for Basic Research (RFBR) (Grant: 17-02-00,157).


  1. Chi, P. S., & Glänzel, W. (2017a). An empirical investigation of the associations among usage, scientific collaboration and citation impact. Scientometrics, 112(7), 403–412. Scholar
  2. Chi, P. S., & Glänzel, W. (2017b). Impact and usage indicators for the assessment of research in scientific disciplines and journals. Scientometrics. In press.Google Scholar
  3. Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “Altmetric” correlate with citations? extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology., 66(10), 2003–2019. Scholar
  4. Glänzel, W. (2007). Characteristic scores and scales. A bibliometric analysis of subject characteristics based on long-term citation observation. Journal of Informetrics, 1(1), 92–102. Scholar
  5. Gonzalez-Valiente, C. L., Pacheco-Mendoza, J., & Arencibia-Jorge, R. (2016). A review of altmetric as an emerging discipline for research evaluation. Learned Publishing, 29(4), 229–238. Scholar
  6. Gumpenberger, C., Glanzel, W., & Gorraiz, J. (2016). The ecstasy and the agony of the altmetric score. Scientometrics, 108(2), 977–982. Scholar
  7. Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014). Coverage and adoption of altmetric sources in the bibliometric community. Scientometrics, 101(2), 1145–1163. Scholar
  8. Karaulova, M., Abdullah, G., Shackleton, O., & Shapira, P. (2016). Science system pass-dependencies and their influences: Nanotechnology research in Russia. Scientometrics, 100(3), 365–383. Scholar
  9. Lewison, G., & Markusova, V. (2010). The evaluation of Russian Cancer Research. Research Evaluation, 19(2), 129–144.CrossRefGoogle Scholar
  10. Markusova, V., & Wilson, C. (2002). From bioweapon to biodefence: The collaborative literature on biodefense in 1990s. Scientometrics, N1, 21–38.CrossRefGoogle Scholar
  11. Thelwall, M., & Wilson, P. (2016). Mendeley readership altmetrics for medical articles: An analysis of 45 fields. Journal of the Association for Information Science and Technology, 67(8), 1962–1972. Scholar
  12. Wang, X., Zhichao, F., & Sun, X. (2016). Usage patterns of scholarly articles on web of science. Scientometrics, 109(2), 917–926. Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2017

Authors and Affiliations

  1. 1.All Russian Institute for Scientific and Technical Information (VINITI) of the RASMoscowRussia
  2. 2.Customer Education Team, Clarivate AnalyticsMoscowRussia

Personalised recommendations