Are uncited papers necessarily all nonimpact papers? A quantitative analysis

Abstract

In traditional research, the application of quantitative measurement on the value and influence of uncited papers is not feasible owing to the lack of data. The rapid advancement of social media has offered a new path for the dissemination of scientific papers and gain influence. In this thesis, the uncited papers published during 2006–2014 with a 5-year window on PLOS ONE journals from the Web of Science core collection was selected. A measurement model was built for the influence of uncited papers on social media dimension based on its Discussed, Saved, and Viewed data to explore the dynamic evolution track of the influence and determine the patterns of interaction. We found that uncited papers have a universal and significant influence on social media platforms. Besides, the evolution pattern of influence of the uncited papers can be illustrated by the layer-to-layer aggregation model of “document properties → three indicators → influence” Overall, the findings of this thesis are referential to the quantitative measurement of the influence of uncited papers.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

References

  1. Bjork, B. C., & Solomon, D. (2013). The publishing delay in scholarly peer-reviewed journals. Journal of Informetrics,7(4), 914–923. https://doi.org/10.1016/j.joi.2013.09.001.

    Article  Google Scholar 

  2. Bornmann, L. (2014a). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics,8(4), 895–903. https://doi.org/10.1016/j.joi.2014.09.005.

    Article  Google Scholar 

  3. Bornmann, L. (2014b). Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime. Journal of Informetrics,8(4), 935–950. https://doi.org/10.1016/j.joi.2014.09.007.

    Article  Google Scholar 

  4. de Winter, J. C. F. (2015). The relationship between tweets, citations, and article views for PLOS ONE articles. Scientometrics,102(2), 1773–1779. https://doi.org/10.1007/s11192-014-1445-x.

    Article  Google Scholar 

  5. Didegah, F., & Thelwall, M. (2013). Which factors help authors produce the highest impact research? Collaboration, journal and document properties. Journal of Informetrics,7(4), 861–873. https://doi.org/10.1016/j.joi.2013.08.006.

    Article  Google Scholar 

  6. Egghe, L. (2010). The distribution of the uncitedness factor and its functional relation with the impact factor. Scientometrics,83(3), 689–695. https://doi.org/10.1007/s11192-009-0130-y.

    Article  Google Scholar 

  7. Egghe, L. (2013). The functional relation between the impact factor and the uncitedness factor revisited. Journal of Informetrics,7(1), 183–189. https://doi.org/10.1016/j.joi.2012.10.007.

    Article  Google Scholar 

  8. Egghe, L., Guns, R., & Rousseau, R. (2011). Thoughts on uncitedness: nobel laureates and fields medalists as case studies. Journal of the American Society for Information Science and Technology,62(8), 1637–1644. https://doi.org/10.1002/asi.21557.

    Article  Google Scholar 

  9. Fairclough, R., & Thelwall, M. (2015). National research impact indicators from Mendeley readers. Journal of Informetrics,9(4), 845–859. https://doi.org/10.1016/j.joi.2015.08.003.

    Article  Google Scholar 

  10. Florence, A. T. (2015). Cited but not read and read but not cited. International Journal of Pharmaceutics,492(1–2), 264–265. https://doi.org/10.1016/j.ijpharm.2015.07.038.

    Article  Google Scholar 

  11. Galligan, F., & Dyas-Correia, S. (2013). Altmetrics: Rethinking the way we measure. Serials Review,39(1), 56–61. https://doi.org/10.1016/j.serrev.2013.01.003.

    Article  Google Scholar 

  12. Gibbs, N. M. (2016). Taken as read but not cited. Anaesthesia and Intensive Care,44(6), 658–659. https://doi.org/10.1177/0310057x1604400628.

    Article  Google Scholar 

  13. Glanzel, W., Schlemmer, B., & Thijs, B. (2003). Better late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon. Scientometrics,58(3), 571–586. https://doi.org/10.1023/B:SCIE.0000006881.30700.ea.

    Article  Google Scholar 

  14. Gomes, J. A. N. F., & Vieira, E. S. (2009). How to improve the citation impact of a paper: Choice of journal, co-authors and institutional addresses. Paper presented at the 12th international conference on scientometrics and informetrics, ISSI 2009.

  15. Hammarfelt, B. (2014). Using altmetrics for assessing research impact in the humanities. Scientometrics,101(2), 1419–1430. https://doi.org/10.1007/s11192-014-1261-3.

    Article  Google Scholar 

  16. Haustein, S., Costas, R., & Lariviere, V. (2015). Characterizing social media metrics of scholarly papers: THE effect of document properties and collaboration patterns. PLoS ONE. https://doi.org/10.1371/journal.pone.0120495.

    Article  Google Scholar 

  17. Haustein, S., Lariviere, V., Thelwall, M., Amyot, D., & Peters, I. (2014a). Tweets vs. Mendeley readers: how do these two social media metrics differ? IT - Information Technology,56(5), 207–215. https://doi.org/10.1515/itit-2014-1048.

    Article  Google Scholar 

  18. Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M., & Lariviere, V. (2014b). Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Science and Technology,65(4), 656–669. https://doi.org/10.1002/asi.23101.

    Article  Google Scholar 

  19. Hoang, J. K., McCall, J., Dixon, A. F., Fitzgerald, R. T., & Gaillard, F. (2015). Using social media to share your radiology research: How effective is a blog post? Journal of the American College of Radiology,12(7), 760–765. https://doi.org/10.1016/j.jacr.2015.03.048.

    Article  Google Scholar 

  20. Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics,101(2), 1027–1042. https://doi.org/10.1007/s11192-014-1229-3.

    Article  Google Scholar 

  21. Hsu, J. W., & Huang, D. W. (2012). A scaling between impact factor and uncitedness. Physica A-Statistical Mechanics and Its Applications,391(5), 2129–2134. https://doi.org/10.1016/j.physa.2011.11.028.

    MathSciNet  Article  Google Scholar 

  22. Hu, Z. W., & Wu, Y. S. (2014). Regularity in the time-dependent distribution of the percentage of never-cited papers: An empirical pilot study based on the six journals. Journal of Informetrics,8(1), 136–146. https://doi.org/10.1016/j.joi.2013.11.002.

    Article  Google Scholar 

  23. Hu, Z. W., & Wu, Y. S. (2018). A probe into causes of non-citation based on survey data. Social Science Information Sur Les Sciences Sociales,57(1), 139–151. https://doi.org/10.1177/0539018417742537.

    Article  Google Scholar 

  24. Hu, Z. W., Wu, Y. S., & Sun, J. J. (2018a). A quantitative analysis of determinants of non-citation using a panel data model. Scientometrics,116(2), 843–861. https://doi.org/10.1007/s11192-018-2791-x.

    Article  Google Scholar 

  25. Hu, Z. W., Wu, Y. S., & Sun, J. J. (2018b). A survey-based structural equation model analysis on influencing factors of non-citation. Current Science,114(11), 2302–2312. https://doi.org/10.18520/cs/v114/i11/2302-2312.

    Article  Google Scholar 

  26. Kalita, D., Deka, D., & Hazarika, T. (2019). A 2D evaluation of altmetrics influence in citation growth: case study of indian research articles in PLoS Journals. Journal of Scientometric Research,8(1), 21–26. https://doi.org/10.5530/jscires.8.1.4.

    Article  Google Scholar 

  27. Lutz, C., & Hoffmann, C. P. (2018). Making academic social capital visible: relating SNS-based, alternative and traditional metrics of scientific impact. Social Science Computer Review,36(5), 632–643. https://doi.org/10.1177/0894439317721181.

    Article  Google Scholar 

  28. MacRoberts, M. H., & MacRoberts, B. R. (2010). Problems of citation analysis: a study of uncited and seldom-cited influences. Journal of the American Society for Information Science and Technology,61(1), 1–12. https://doi.org/10.1002/asi.21228.

    Article  Google Scholar 

  29. Melero, R. (2015). Altmetrics—A complement to conventional metrics. Biochemia Medica,25(2), 152–160. https://doi.org/10.11613/bm.2015.016.

    Article  Google Scholar 

  30. Nabout, J. C., Teresa, F. B., Machado, K. B., do Prado, V. H. M., Bini, L. M., & Diniz, J. A. F. (2018). Do traditional scientometric indicators predict social media activity on scientific knowledge? An analysis of the ecological literature. Scientometrics,115(2), 1007–1015. https://doi.org/10.1007/s11192-018-2678-x.

    Article  Google Scholar 

  31. Nicolaisen, J., & Frandsen, T. F. (2019). Zero impact: a large-scale study of uncitedness. Scientometrics,119(2), 1227–1254. https://doi.org/10.1007/s11192-019-03064-5.

    Article  Google Scholar 

  32. Nowroozzadeh, M. H., & Salehi-Marzijarani, M. (2019). Uncitedness in the Top General Medical Journals. Journal of General Internal Medicine,34(12), 2695–2696. https://doi.org/10.1007/s11606-019-05290-2.

    Article  Google Scholar 

  33. Onodera, N., & Yoshikane, F. (2015). Factors affecting citation rates of research articles. Journal of the Association for Information Science and Technology,66(4), 739–764. https://doi.org/10.1002/asi.23209.

    Article  Google Scholar 

  34. Ortega, J. L. (2018a). Disciplinary differences of the impact of altmetric. FEMS Microbiology Letters. https://doi.org/10.1093/femsle/fny049.

    Article  Google Scholar 

  35. Ortega, J. L. (2018b). The life cycle of altmetric impact: A longitudinal study of six metrics from PlumX. Journal of Informetrics,12(3), 579–589. https://doi.org/10.1016/j.joi.2018.06.001.

    Article  Google Scholar 

  36. Piwowar, H. (2013). Introduction altmetrics: What, why and where? Bulletin of the American Society for Information Science and Technology,39(4), 30–31.

    Google Scholar 

  37. Pulido, C. M., Redondo-Sama, G., Sorde-Marti, T., & Flecha, R. (2018). Social impact in social media: A new method to evaluate the social impact of research. PLos ONE. https://doi.org/10.1371/journal.pone.0203117.

    Article  Google Scholar 

  38. Schlogl, C., Gorraiz, J., Gumpenberger, C., Jack, K., & Kraker, P. (2014). Comparison of downloads, citations and readership data for two information systems journals. Scientometrics,101(2), 1113–1128. https://doi.org/10.1007/s11192-014-1365-9.

    Article  Google Scholar 

  39. Sud, P., & Thelwall, M. (2014). Evaluating altmetrics. Scientometrics,98(2), 1131–1143. https://doi.org/10.1007/s11192-013-1117-2.

    Article  Google Scholar 

  40. Sugimoto, C. R., Work, S., Lariviere, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology,68(9), 2037–2062. https://doi.org/10.1002/asi.23833.

    Article  Google Scholar 

  41. Thelwall, M., & Kousha, K. (2015). Web indicators for research evaluation. Part 2: Social media metrics. Profesional De La Informacion,24(5), 607–620. https://doi.org/10.3145/epi.2015.sep.09.

    Article  Google Scholar 

  42. Van Dalen, H. P., & Henkens, K. (2004). Demographers and their journals: Who remains uncited after ten years? Population and Development Review,30(3), 489–506.

    Article  Google Scholar 

  43. Van Noorden, R. (2017). The science that’s never been cited. Nature,552(7684), 162–164. https://doi.org/10.1038/d41586-017-08404-0.

    Article  Google Scholar 

  44. Wang, X. W., Fang, Z. C., & Guo, X. H. (2016a). Tracking the digital footprints to scholarly articles from social media. Scientometrics,109(2), 1365–1376. https://doi.org/10.1007/s11192-016-2086-z.

    Article  Google Scholar 

  45. Wang, X. W., Fang, Z. C., & Sun, X. L. (2016b). Usage patterns of scholarly articles on Web of Science: a study on Web of Science usage count. Scientometrics,109(2), 917–926. https://doi.org/10.1007/s11192-016-2093-0.

    Article  Google Scholar 

  46. Wang, X. W., Wang, Z., Mao, W. L., & Liu, C. (2014). How far does scientific community look back? Journal of Informetrics,8(3), 562–568. https://doi.org/10.1016/j.joi.2014.04.009.

    Article  Google Scholar 

  47. Zhang, X., Wang, X., Zhao, H., Ordonez de Pablos, P., Sun, Y., & Xiong, H. (2019). An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications. Scientometrics,119(3), 1311–1344. https://doi.org/10.1007/s11192-019-03088-x.

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the National Social Science Foundation of China under Grant 17BGL031.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jianhua Hou.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hou, J., Ye, J. Are uncited papers necessarily all nonimpact papers? A quantitative analysis. Scientometrics 124, 1631–1662 (2020). https://doi.org/10.1007/s11192-020-03539-w

Download citation

Keywords

  • Uncited paper
  • Social media
  • Dynamic evolution
  • Quantitative analysis