Advertisement

The correlation between paper length and citations: a meta-analysis

  • Juan Xie
  • Kaile Gong
  • Ying ChengEmail author
  • Qing Ke
Article

Abstract

Citation count is a widely used bibliometric indicator. It is influenced by many factors, some of which have been well investigated. This study investigated a more controversial relationship between paper length and citations. After systematic retrieval and selection of literature, we performed a random effects meta-analysis in which 24 effect sizes in 18 original studies were synthesized. The dataset included 1,548,088 papers. Meta-regression and subgroup analysis were used to identify factors moderating the correlation between paper length and citations. Tests of heterogeneity and publication bias were conducted to guarantee reliability, and statistical analysis via computer simulation was used to interpret the results of the heterogeneity test. We observed a moderate, positive correlation between paper length and citations (r = 0.310) from the dataset, and hence concluded that the longer a paper is, the more citations it receives. Citation windows and the perceived quality of journals were found to exert a moderating influence on the correlation.

Keywords

Paper length Citations Meta-analysis Correlation Meta regression 

Notes

Acknowledgements

The authors would like to thank Dr. Junwu Ye and Dr. Jiang Li for their advice on the English writing. We would also like to express our appreciation for the reviewers’ hard work and helpful suggestions. This research is financially supported by research Grants from the National Social Science Foundation of China (Grant No. 17BTQ014).

Supplementary material

11192_2019_3015_MOESM1_ESM.docx (36 kb)
Supplementary material 1 (DOCX 36 kb)

References

  1. Adair, J. G., & Vohra, N. (2003). The explosion of knowledge, references, and citations: Psychology’s unique response to a crisis. American Psychologist, 58(1), 15–23.CrossRefGoogle Scholar
  2. Agnieszka, B. K. (2012). Published, not perished, but has anybody read it? Citation success of finance research articles. Applied Financial Economics, 22(20), 1679–1695.CrossRefGoogle Scholar
  3. Antoniou, G. A., Antoniou, S. A., Georgakarakos, E. I., Sfyroeras, G. S., & Georgiadis, G. S. (2015). Bibliometric analysis of factors predicting increased citations in the vascular and endovascular literature. Annals of Vascular Surgery, 29, 286–292.CrossRefGoogle Scholar
  4. Arao, L. H. (2015). The half-life and obsolescence of the literature science area: A contribution to the understanding the chronology of citations in academic activity. Qualitative and Quantitative Methods in Libraries, 4, 603–610.Google Scholar
  5. Bornmann, L. (2015). Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics. Scientometrics, 103(3), 1123–1144.CrossRefGoogle Scholar
  6. Bornmann, L., & Leydesdorff, L. (2015). Does quality and content matter for citedness? A comparison with para-textual factors and over time. Journal of Informetrics, 9(3), 419–429.CrossRefGoogle Scholar
  7. Bornmann, L., & Leydesdorff, L. (2017). Skewness of citation impact data and covariates of citation distributions: A large-scale empirical analysis based on Web of Science data. Journal of Informetrics, 11(1), 164–175.CrossRefGoogle Scholar
  8. Cleveland, W. S. (1984). Graphs in scientific publications. American Statistician, 38(4), 261–269.Google Scholar
  9. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New York: Psychology Press.zbMATHGoogle Scholar
  10. Di Bitetti, M., & Ferreras, J. A. (2016). Publish (in English) or perish: The effect on citation rate of using languages other than English in scientific publications. Ambio, 46(1), 121–127.CrossRefGoogle Scholar
  11. Didegah, F., & Thelwall, M. (2013). Which factors help authors produce the highest impact research? Collaboration, journal and document properties. Journal of Informetrics, 7, 861–873.CrossRefGoogle Scholar
  12. Ellis, G., Whitehead, M. A., Robinson, D., O’Neill, D., & Langhorne, P. (2011). Comprehensive geriatric assessment for older adults admitted to hospital: Meta-analysis of randomized controlled trials. British Medical Journal, 343, d6553.CrossRefGoogle Scholar
  13. Falagas, M. E., Zarkali, A., Karageorgopoulos, D. E., Bardakas, V., & Mavros, M. N. (2013). The impact of article length on the number of future citations: A bibliometric analysis of general medicine journals. PLoS ONE.  https://doi.org/10.1371/journal.pone.0049476.Google Scholar
  14. Foley, J. A., & Della, S. S. (2011). Do shorter Cortex papers have greater impact? Cortex, 47(6), 635–642.CrossRefGoogle Scholar
  15. Fox, C. W., Paine, C. E. T., & Sauterey, B. (2016). Citations increase with manuscript length, author number, and references cited in ecology journals. Ecology and Evolution.  https://doi.org/10.1002/ece3.2505.Google Scholar
  16. Frogel, J. A. (2010). Astronomy’s greatest hits: The 100 most cited papers in each year of the first decade of the 21st century (2000–2009). Publications of the Astronomical Society of the Pacific, 122(896), 1214–1235.CrossRefGoogle Scholar
  17. Gross, P. L., & Gross, E. M. (1927). College libraries and chemical education. Bulletin of the American Association of University Professors, 66(1713), 385–389.Google Scholar
  18. Hartley, J., Sotto, E., & Pennebaker, J. (2002). Style and substance in psychology: Are influential articles more readable than less influential ones? Social Studies of Science, 32(2), 321–334.CrossRefGoogle Scholar
  19. Haslam, N., Ban, L., Kaufmann, L., Loughnan, S., Peters, K., Whelan, J., et al. (2008). What makes an article influential? Predicting impact in social and personality psychology. Scientometrics, 76(1), 169–185.CrossRefGoogle Scholar
  20. Haslam, N., & Koval, P. (2010). Predicting long-term citation impact of articles in social and personality psychology. Psychological Reports, 106(3), 891–900.CrossRefGoogle Scholar
  21. Haustein, S., Costas, R., & Larivière, V. (2015). Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PLoS ONE, 10(3), e0120495.CrossRefGoogle Scholar
  22. Hayashi, T., & Fujigaki, Y. (1999). Differences in knowledge production between disciplines based on analysis of paper styles and citation patterns. Scientometrics, 46(1), 73–86.CrossRefGoogle Scholar
  23. Hegarty, P., & Walton, Z. (2012). The consequences of predicting scientific impact in psychology using journal impact factors. Perspectives on Psychological Science, 7(1), 72.CrossRefGoogle Scholar
  24. Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2002). Statistical heterogeneity in systematic reviews of clinical trials: A critical appraisal of guidelines and practice. Journal of Health Services Research and Policy, 7(1), 51–61.CrossRefGoogle Scholar
  25. Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327, 557–560.CrossRefGoogle Scholar
  26. Hodge, D. R., Victor, B. G., Grogan-Kaylor, A., & Perron, B. E. (2017). Disseminating high-impact social work scholarship: A longitudinal examination of 5-year citation count correlates. Journal of the Society for Social Work and Research, 8(2), 211–231.CrossRefGoogle Scholar
  27. Hunter, J. E., & Schmidt, F. L. (2000). Fixed effects vs. random effects meta-analysis models: Implications for cumulative research knowledge. International Journal of Selection and Assessment, 8(4), 275–292.CrossRefGoogle Scholar
  28. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.CrossRefGoogle Scholar
  29. Kostoff, R. N. (2007). The difference between highly and poorly cited medical articles in the journal Lancet. Scientometrics, 72(3), 513–520.CrossRefGoogle Scholar
  30. Laperche, S., & Pillonel, J. (2005). Data envelopment analysis of OR/MS journals. Scientometrics, 64(2), 133–150.CrossRefGoogle Scholar
  31. Leimu, R., & Koricheva, J. (2005). What determines the citation frequency of ecological papers? Trends in Ecology and Evolution, 20(1), 28–32.CrossRefGoogle Scholar
  32. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.Google Scholar
  33. Lokker, C., McKibbon, K. A., McKinlay, R. J., Wilczynski, N. L., & Haynes, R. B. (2008). Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: Retrospective cohort study. British Medical Journal, 336, 655–657.CrossRefGoogle Scholar
  34. Mingers, J., & Xu, F. (2010). The drivers of citations in management science journals. European Journal of Operational Research, 205(2), 422–430.CrossRefzbMATHGoogle Scholar
  35. Nieri, M., Clauser, C., Franceschi, D., Pagliaro, U., Saletta, D., & Pini-Prato, G. (2007). Randomized clinical trials in implant therapy: Relationships among methodological, statistical, clinical, paratextual features and number of citations. Clinical Oral Implants Research, 18(4), 419–431.CrossRefGoogle Scholar
  36. O’Leary, D. E. (2008). The relationship between citations and number of downloads in decision support systems. Decision Support Systems, 45(4), 972–980.CrossRefGoogle Scholar
  37. Onodera, N., & Yoshikane, F. (2015). Factors affecting citation rates of research articles. Journal of the Association for Information Science and Technology, 66(4), 739–764.CrossRefGoogle Scholar
  38. Peters, H. P. F., & van Raan, A. F. J. (1994). On determinants of citation scores: A case study in chemical engineering. Journal of the American Society for Information Science, 45(1), 39–49.CrossRefGoogle Scholar
  39. Robson, B. J., & Mousquès, A. (2014). Predicting citation counts of environmental modelling papers. In B. P. Ames, N. W. T. Quinn, and A. E. Rizzoli (Eds.), Proceedings—7th international congress on environmental modelling and software: Bold visions for environmental modeling (iEMSs) (Vol. 3, pp. 1390–1396).Google Scholar
  40. Ronda-Pupo, G. A. (2017). The effect of document types and sizes on the scaling relationship between citations and co-authorship patterns in management journals. Scientometrics, 110, 1–17.CrossRefGoogle Scholar
  41. Rosenthal, R. (1991). Meta-analytic procedures for social research. Thousand Oaks, CA: Sage.CrossRefGoogle Scholar
  42. Royle, P., Kandala, N. B., Barnard, K., & Waugh, N. (2013). Bibliometrics of systematic reviews: Analysis of citation rates and journal impact factors. Systematic Reviews, 2, 74.CrossRefGoogle Scholar
  43. Shi, Y. L., Sanghack, L., & Sung, H. J. (2010). Author and article characteristics, journal quality and citation in economic research. Applied Economics Letters, 17, 1697–1701.CrossRefGoogle Scholar
  44. Simonton, D. K. (2006). Scientific status of disciplines, individuals, and ideas: Empirical analyses of the potential impact of theory. Review of General Psychology, 10(2), 98–112.CrossRefGoogle Scholar
  45. Slyder, J. B., Stein, B. R., Sams, B. S., Walker, D. M., Beale, B. J., Feldhaus, J. J., et al. (2011). Citation pattern and lifespan: A comparison of discipline, institution, and individual. Scientometrics, 89, 955–966.CrossRefGoogle Scholar
  46. So, M., Kim, J., Choi, S., & Park, H. W. (2015). Factors affecting citation networks in science and technology: Focused on non-quality factors. Quality and Quantity, 49, 1513–1530.CrossRefGoogle Scholar
  47. Stremersch, S., Verniers, I., & Verhoef, P. (2007). The quest for citations. Journal of Marketing, 71(3), 171–193.CrossRefGoogle Scholar
  48. Tahamtan, I., & Bornmann, L. (2018). Core elements in the process of citing publications: Conceptual overview of the literature. Journal of Informetrics, 12(1), 203–216.CrossRefGoogle Scholar
  49. Uthman, O. A., Okwundu, C. I., Wiysonge, C. S., Young, T., & Clarke, A. (2013). Citation classics in systematic reviews and meta-analyses: Who wrote the top 100 most cited articles? PLoS ONE.  https://doi.org/10.1371/journal.pone.0078517.Google Scholar
  50. Vanclay, J. K. (2013). Factors affecting citation rates in environmental science. Journal of Informetrics, 7(2), 265–271.CrossRefGoogle Scholar
  51. Vandermeulen, A. (1972). Manuscripts in the maelstrom: A theory of the editorial process. Public Choice, 13, 107–111.CrossRefGoogle Scholar
  52. Vieira, E. S., & Gomes, J. A. N. F. (2010). Citations to scientific articles: Its distribution and dependence on the article features. Journal of Informetrics, 4(1), 1–13.CrossRefGoogle Scholar
  53. Walters, G. D. (2006). Predicting subsequent citations to articles published in twelve crime-psychology journals: Author impact versus journal impact. Scientometrics, 69(3), 499–510.CrossRefGoogle Scholar
  54. Wesel, M. V., Wyatt, S., & Haaf, J. T. (2014). What a difference a colon makes: How superficial factors influence subsequent citation. Scientometrics, 98(3), 1601–1615.CrossRefGoogle Scholar
  55. Xiao, X. B., & Chai, Y. J. (2016). Properties of scholarly papers and number of citations. New Technology of Library and Information Service, 6, 46–52 (in Chinese).Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2019

Authors and Affiliations

  1. 1.School of Information ManagementNanjing UniversityNanjingChina

Personalised recommendations