Abstract
Getting cited is important for scholars and for the institutions in which they work. Whether because of the influence on scientific progress or because of the reputation of scholars and their institutions, understanding why some articles are cited more often than others can help scholars write more highly cited articles. This article builds upon earlier literature which identifies seemingly superficial factors that influence the citation rate of articles. Three Journal Citation Report subject categories are analyzed to identify these effects. From a set of 2,016 articles in Sociology, 6,957 articles in General & Internal Medicine, and 23,676 articles in Applied Physics, metadata from the Web of Knowledge was downloaded in addition to PDFs of the full articles. In this article number of words in title, number of pages, number of references, sentences in the abstract, sentences in the paper, number of authors and readability were identified as factors for analysis.
Similar content being viewed by others
References
Ball, R., Mittermaier, B., & Tunger, D. (2009). Creation of journal-based publication profiles of scientific institutions—A methodology for the interdisciplinary comparison of scientific research based on the J-factor. Scientometrics, 81(2), 381–392. doi:10.1007/s11192-009-2120-5.
Booth, W. C., Colomb, G. G., & Williams, J. M. (2003). The craft of research (2nd ed., Chicago guides to writing, editing, and publishing). Chicago: University of Chicago press.
Botton, A. D. (2001). The consolations of philosophy. Baltimore, MD: Penguin Books Ltd.
Collins, H. M. (1990). Artificial experts: Social knowledge and intelligent machines (inside technology). Cambridge, MA: MIT Press.
Crossley, S., Greenfield, J., & McNamara, D. (2008). Assessing text readability using cognitively based indices. Tesol Quarterly, 42(3), 475–493. doi:10.1002/j.1545-7249.2008.tb00142.x.
Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32(3), 221–233. doi:10.1037/h0057532.
Franceschet, M., & Costantini, A. (2010). The effect of scholar collaboration on impact and quality of academic papers. Journal of Informetrics, 4(4), 540–553. doi:http://dx.doi.org/10.1016/j.joi.2010.06.003.
Frenken, K., Hölzl, W., & Vor, F. D. (2005). The citation impact of research collaborations: the case of European biotechnology and applied microbiology (1988–2002). Journal of Engineering and Technology Management, 22, 9–30. doi:10.1111/j.1435-5957.2010.00309.x.
Friedman, D. B., Hoffman-Goetz, L., & Arocha, J. F. (2004). Readability of cancer information on the internet. Journal of Cancer Education, 19(2), 117–122. doi:10.1207/s15430154jce1902_13.
Fry, E. (1968). A readability formula that saves time. Journal of Reading, 11(7), 513–516, 575–578. doi:10.2307/40013635.
Gilbert, G. N. (1977). Referencing as persuasion. Social Studies of Science, 7(1), 113–122. doi:10.2307/284636.
Glänzel, W., & Thijs, B. (2004). Does co-authorship inflate the share of self-citations? Scientometrics, 61(3), 395–404. doi:10.1023/B:SCIE.0000045117.13348.b1.
Hartley, J., Trueman, M., & Meadows, A. (1988). Readability and prestige in scientific journals. Journal of Information Science, 14(2), 69–75. doi:10.1177/016555158901500209.
Haslam, N., Ban, L., Kaufmann, L., Loughnan, S., Peters, K., Whelan, J., et al. (2008). What makes an article influential? Predicting impact in social and personality psychology. Scientometrics, 76(1), 169–185. doi:10.1007/s11192-007-1892-8.
Hayden, J. D. (2008). Readability of the British Journal of Surgery. British Journal of Surgery, 95, 119–124. doi:10.1002/bjs.5994.
Hudson, J. (2007). Be known by the company you keep: Citations—quality or chance? Scientometrics, 71(2), 231–238. doi:10.1007/s11192-007-1671-6.
Jacques, T. S., & Sebire, N. J. (2010). The impact of article titles on citation hits: An analysis of general and specialist medical journals. JRSM Short Reports, 1(1). doi:10.1258/shorts.2009.100020.
Jamali, H., & Nikzad, M. (2011). Article title type and its relation with the number of downloads and citations. Scientometrics, 88(2), 653–661. doi:10.1007/s11192-011-0412-z.
Kincaid, J. P., Fishburne, R. P, Jr., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Springfield, VA: National Technical Information Service.
Larivière, V., Archambault, É., & Gingras, Y. (2008). Long-term variations in the aging of scientific literature: From exponential growth to steady-state science (1900–2004). Journal of the American Society for Information Science and Technology, 59(2), 288–296. doi:10.1002/asi.20744.
Latour, B. (1987). Science in action: How to follow scientists and engineers through society. Cambridge, MA: Harvard University Press.
Latour, B., & Woolgar, S. (1986). Laboratory life: The construction of scientific facts. Princeton, NJ: Princeton University Press.
Levitt, J. M., & Thelwall, M. (2009). Citation levels and collaboration within library and information science. Journal of the American Society for Information Science and Technology, 60(3), 434–442. doi:10.1002/asi.21000.
Lin, S.-Y., Su, C.-C., Lai, Y.-D., Yang, L.-C., & Hsieh, S.-K. (2009). Assessing text readability using hierarchical lexical relations retrieved from WordNet. Computational Linguistics and Chinese Language Processing, 14(1), 45–84.
Martin, B., & Groth, E. (1991). Scientific knowledge in controversy: The social dynamics of the fluoridation debate (SUNY series in science, technology, and society). Albany, NY: State University of New York Press.
Merton, R. (1968). The Matthew effect in science: The reward and communication systems of science are considered. Science, 159(3810), 56–63. doi:10.1126/science.159.3810.56.
Microsoft. (2003). Readability scores. http://office.microsoft.com/en-us/word/HP051863181033.aspx. Accessed 23 March 2013.
Neuman, W. L. (1991). Social research methods: Qualitative and quantitative approaches. Boston: Allyn and Bacon.
Price, D. J. D. S. (1963). Little science, big science (George B. Pegram lectures, Vol. 1962). New York: Columbia University Press.
Ramos, M. A., Melo, J. G., & Albuquerque, U. P. (2012). Citation behavior in popular scientific papers: What is behind obscure citations? The case of ethnobotany. Scientometrics, 92, 711–719. doi:10.1007/s11192-012-0662-4.
Roberts, J. C., Fletcher, R. H., & Fletcher, S. W. (1994). Effects of peer review and editing on the readability of articles published in annals of internal medicine. Journal of the American Medical Association, 272, 119–121. doi:10.1001/jama.1994.03520020045012.
Smart, J. C., & Bayer, A. E. (1986). Author collaboration and impact: a note on citation rates of single and multiple authored articles. Scientometrics, 10(5–6), 297–305. doi:10.1007/BF02016776.
Stremersch, S., Verniers, I., & Verhoef, P. (2007). The quest for citations: Drivers of article impact. Journal of Marketing, 71(3), 171–193. doi:10.1509/jmkg.71.3.171.
Vieira, E. S., & Gomes, J. A. N. F. (2010). Citations to scientific articles: Its distribution and dependence on the article features. Journal of Informetrics, 4, 1–13. doi:10.1016/j.joi.2009.06.002.
Villere, M. F., & Stearns, G. K. (1976). The readability of organizational behavior textbooks. The Academy of Management Journal, 19(1), 132–137. doi:10.2307/255455.
Wager, E., & Middleton, P. (2002). Effects of technical editing in biomedical journals: A systematic review. JAMA, 287(21), 2821–2824. doi:10.1001/jama.287.21.2821.
Wang, M., Yu, G., An, S., & Yu, D. (2012). Discovery of factors influencing citation impact based on a soft fuzzy rough set model. Scientometrics, 93, 635–644. doi:10.1007/s11192-012-0766-x.
Webster, G. D., Jonason, P. K., & Schember, T. O. (2009). Hot topics and popular papers in evolutionary psychology: Analyses of title words and citation counts in evolution and human behavior, 1979–2008. Evolutionary Psychology, 7(3), 348–362.
Weeks, W. B., & Wallace, A. E. (2002). Readability of British and American medical prose at the start of the 21st century. British Medical Journal, 325, 1451–1452. doi:10.1136/bmj.325. 7378.1451.
Acknowledgments
The authors would like to thank Loet Leydesdorff for his helpful comments. Furthermore we believe that additional comments from the two anonymous reviewers have increased the quality of this article, for which we are grateful.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix: limitations to the automatic processing of article text
The automatic processing of article text offers great advantages in terms of speed thus increasing the sheer number of articles that can be processed, but there are disadvantages in terms of accuracy. Some of the causes will be briefly discussed below. Even though the problems discussed below have an impact on the readability score of individual articles, our assessment is that, as all articles in a journal/year set seem to suffer the same problems, it reduces the inter-article differences per subset. Therefore we do not expect the problems to reduce the reliability of our analysis.
Continuous print
Many journals offer articles as a single unit in a PDF, but some journals print articles continuously, i.e. not always starting an article on a new page. Thus the PDF of an article may also contain pages of another article, most likely its reference list or first page. These pages are also included in the analysis of the target article.
OCR mistakes
As mentioned in the main text regarding titles of articles in the Web of Science, articles themselves also suffer from OCR problems. Unfortunately not all PDFs are created from their original source. One of the most common problems this respect is mistaking the letter “m” with the combination “rn”, and possibly also vice verse.
Footer/header
Depending on the way an article is created and presented, sentences across pages are read continuously (as a human reader would do) or are read as continuing in the footer and header, thus including items such as the page number, article or journal name.
Affiliations, addresses, and references
All text in the articles is extracted, icluding the affiliations and addresses. As these do not follow normal language conventions they will have an impact on readability.
Rights and permissions
About this article
Cite this article
van Wesel, M., Wyatt, S. & ten Haaf, J. What a difference a colon makes: how superficial factors influence subsequent citation. Scientometrics 98, 1601–1615 (2014). https://doi.org/10.1007/s11192-013-1154-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-013-1154-x