, Volume 116, Issue 3, pp 1771–1783 | Cite as

Temporal characteristics of retracted articles

  • Judit Bar-Ilan
  • Gali Halevi


There are three main reasons for retraction: (1) ethical misconduct (e.g. duplicate publication, plagiarism, missing credit, no IRB, ownership issues, authorship issues, interference in the review process, citation manipulation); (2) scientific distortion (e.g. data manipulation, fraudulent data, unsupported conclusions, questionable data validity, non-replicability, data errors—even if unintended); (3) administrative error (e.g. article published in wrong issue, not the final version published, publisher errors). The first category, although highly deplorable has almost no effect on the advancement of science, the third category is relatively minor. The papers belonging to the second category are most troublesome from the scientific point of view, as they are misleading and have serious negative implications not only on science but also on society. In this paper, we explore some temporal characteristics of retracted articles, including time of publication, years to retract, growth of post retraction citations over time and social media attention by the three major categories. The data set comprises 995 retracted articles retrieved in October 2014 from Elsevier’s ScienceDirect. Citations and Mendeley reader counts were retrieved four times within 4 years, which allowed us to examine post-retraction longitudinal trends not only for citations, but also for Mendeley reader counts. The major findings are that both citation counts and Mendeley reader counts continue to grow after retraction.


Scientific fraud and dishonesty Citation analysis Altmetrics 



The present study is an extended version of an article presented at the 16th International Conference on Scientometrics and Informetrics, Wuhan (China), 16–20 October 2017 (Bar-Ilan and Halevi 2017b). The first author was supported by EU COST Actions PEERE and KnowEscape. We thank Mike Thelwall for using his Webometric Analyst for data collection from the altmetric platforms. Funding was provided by European Cooperation in Science and Technology (Grant Nos. TD1210 - Knowescape, TD1306 - PEERE).


  1. Almeida, R. M. V., de Albuquerque Rocha, K., Catelani, F., Fontes-Pereira, A. J., & Vasconcelos, S. M. (2015). Plagiarism allegations account for most retractions in major Latin American/Caribbean databases. Science and Engineering Ethics, 22, 1447–1456.CrossRefGoogle Scholar
  2. Ataie-Ashtiani, B. (2017a). Chinese and Iranian scientific publications: Fast growth and poor ethics. Science and Engineering Ethics, 23(1), 317–319.CrossRefGoogle Scholar
  3. Ataie-Ashtiani, B. (2017b). World map of scientific misconduct. Science and Engineering Ethics. Scholar
  4. Bar-Ilan, J., & Halevi, G. (2017a). Post retraction citations in context: A case study. Scientometrics. Scholar
  5. Bar-Ilan, J., & Halevi, G. (2017b). Temporal characteristics of retracted articles—Research in progress. In Proceedings of ISSI 2017The 16th international conference on scientometrics and informetrics (pp. 664–669). Wuhan University, China.Google Scholar
  6. Bean, J. R. (2017). Truth or consequences: The growing trend of publication retraction. World Neurosurgery, 103, 917–918.CrossRefGoogle Scholar
  7. Bilbrey, E., O’Dell, N., & Creamer, J. (2014). A novel rubric for rating the quality of retraction notices. Publications, 2(1), 14–26.CrossRefGoogle Scholar
  8. Bornemann-Cimenti, H., & Sandner-Kiesling, A. (2015). Bringing retracted papers into focus. Clinical and Experimental Otorhinolaryngology, 8(1), 81.CrossRefGoogle Scholar
  9. Bozzo, A., Bali, K., Evaniew, N., & Ghert, M. (2017). Retractions in cancer research: A systematic survey. Research Integrity and Peer Review, 2(1), 5.CrossRefGoogle Scholar
  10. Budd, J. M., Sievert, M., & Schultz, T. R. (1998). Phenomena of retraction. Reasons for retraction and citations to the publications. Journal of the American Medical Association, 280(3), 296–297. Scholar
  11. Casadevall, A., Steen, R. G., & Fang, F. C. (2014). Sources of error in the retracted scientific literature. The FASEB Journal, 28(9), 3847–3855.CrossRefGoogle Scholar
  12. Chaddah, P. (2014). Not all plagiarism requires a retraction. Nature, 511(7508), 127.CrossRefGoogle Scholar
  13. Cyranoski, D. (2014). Stem-cell pioneer blamed media ‘bashing’ in suicide note. Nature News. Scholar
  14. da Silva, J. A. T., & Bornemann-Cimenti, H. (2017). Why do some retracted papers continue to be cited? Scientometrics, 110(1), 365–370.CrossRefGoogle Scholar
  15. Decullier, E., Huot, L., & Maisonneuve, H. (2014). What time-lag for a retraction search on PubMed? BMC Research Notes, 7(1), 395.CrossRefGoogle Scholar
  16. Decullier, E., Huot, L., Samson, G., & Maisonneuve, H. (2013). Visibility of retractions: A cross-sectional one-year study. BMC Research Notes, 6(1), 238.CrossRefGoogle Scholar
  17. Fanelli, D. (2013). Why growing retractions are (mostly) a good sign? PLoS Medicine, 10(12), e1001563. Scholar
  18. Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, 109(42), 17028–17033.CrossRefGoogle Scholar
  19. Foo, J. Y. A. (2011). A retrospective analysis of the trend of retracted publications in the field of biomedical and life sciences. Science and Engineering Ethics, 17(3), 459–468.CrossRefGoogle Scholar
  20. Furman, J. L., Jensen, K., & Murray, F. (2012). Governing knowledge in the scientific community: Exploring the role of retractions in biomedicine. Research Policy, 41(2), 276–290.CrossRefGoogle Scholar
  21. Garfield, E., & Welljams-Dorof, A. (1990). The impact of fraudulent research on the scientific literature: The Stephen E. Breuning case. JAMA, 263(10), 1424–1426.CrossRefGoogle Scholar
  22. Gasparyan, A. Y., Ayvazyan, L., Akazhanov, N. A., & Kitas, G. D. (2014). Self-correction in biomedical publications and the scientific impact. Croatian Medical Journal, 55(1), 61.CrossRefGoogle Scholar
  23. Grieneisen, M. L., & Zhang, M. (2012). A comprehensive survey of retracted articles from the scholarly literature. PLoS ONE, 7(10), e44118. Scholar
  24. Gunn, W. (2016). Comment #00632—Clarification/correction of Mendeley saves definition—NISO RP-25-201x-3, Altmetrics Data Quality Code of Conduct—Draft for public comment.pdf. Retrieved from
  25. Hesselmann, F., Graf, V., Schmidt, M., & Reinhart, M. (2017). The visibility of scientific misconduct: A review of the literature on retracted journal articles. Current Sociology, 65(6), 814–845.CrossRefGoogle Scholar
  26. Inoue, Y., & Muto, K. (2016). Noncompliance with Human Subjects’ Protection Requirements as a reason for retracting papers: Survey of retraction notices on medical papers published from 1981 to 2011. Accountability in Research, 23(2), 123–135. Scholar
  27. Jimenez, D. F., & Garza, D. N. (2017). Predatory publishing and academic integrity. World Neurosurgery, 105, 990–992.CrossRefGoogle Scholar
  28. King, E. G., Oransky, I., Sachs, T. E., Farber, A., Flynn, D. B., Abritis, A., et al. (2017). Analysis of retracted articles in the surgical literature. The American Journal of Surgery. Scholar
  29. Korpela, K. M. (2010). How long does it take for the scientific literature to purge itself of fraudulent material? The Breuning case revisited. Current Medical Research and Opinion, 26(4), 843–847.CrossRefGoogle Scholar
  30. Lu, S. F., Jin, G. Z., Uzzi, B., & Jones, B. (2013). The retraction penalty: Evidence from the Web of Science. Scientific Reports, 3, 3146.CrossRefGoogle Scholar
  31. Madlock-Brown, C. R., & Eichmann, D. (2014). The (lack of) impact of retraction on citation networks. Science and Engineering Ethics, 21(1), 127–137.CrossRefGoogle Scholar
  32. Marušić, A. (2018). Private communication.Google Scholar
  33. Merton, R. K. (1942). Science and technology in democratic order. Journal of Legal and Political Sociology, 1, 111–126. Reprinted in: The sociology of science: Theoretical and empirical investigations. Chicago: University of Chicago Press, 1973, pp. 286–324.Google Scholar
  34. Molckovsky, A., Vickers, M. M., & Tang, P. A. (2011). Characterization of published errors in high-impact oncology journals. Current Oncology, 18(1), 26–32.CrossRefGoogle Scholar
  35. Moradi, S. (2017). The altmetrics of retracted articles in biochemistry & molecular biology. Paper presented at: IFLA WLIC 2017—Wrocław, Poland. Retrieved from
  36. Nath, S. B., Marcus, S. C., & Druss, B. G. (2006). Retractions in the research literature: Misconduct or mistakes? Medical Journal of Australia, 185(3), 152–154.Google Scholar
  37. Rai, R., & Sabharwal, S. (2017). Retracted publications in orthopaedics: Prevalence, characteristics, and trends. Journal of Bone and Joint Surgery, 99(9), e44.CrossRefGoogle Scholar
  38. RetractionWatch. (2017). The 2017 retraction watch year in review (hint: Our database is nearly done). Retrieved from
  39. Singh, H. P., Mahendra, A., Yadav, B., Singh, H., Arora, N., & Arora, M. (2014). A comprehensive analysis of articles retracted between 2004 and 2013 from biomedical literature: A call for reforms. Journal of Traditional and Complementary Medicine, 4(3), 136.CrossRefGoogle Scholar
  40. Steen, R. G. (2011). Retractions in the scientific literature: Is the incidence of research fraud increasing? Journal of Medical Ethics, 37, 249–253.CrossRefGoogle Scholar
  41. Steen, R. G., Casadevall, A., & Fang, F. C. (2013). Why has the number of scientific retractions increased? PLoS ONE, 8(7), e68397.CrossRefGoogle Scholar
  42. Stretton, S., Bramich, N. J., Keys, J. R., Monk, J. A., Ely, J. A., Haley, C., et al. (2012). Publication misconduct and plagiarism retractions: A systematic, retrospective study. Current Medical Research and Opinion, 28(10), 1575–1583.CrossRefGoogle Scholar
  43. Trikalinos, N. A., Evangelou, E., & Ioannidis, J. P. (2008). Falsified papers in high-impact journals were slow to retract and indistinguishable from nonfraudulent papers. Journal of Clinical Epidemiology, 61(5), 464–470.CrossRefGoogle Scholar
  44. van Leeuwen, T., & Luwel, M. (2014). An in-depth analysis of papers retracted in the Web of Science. STI 2014 Leiden, 337.Google Scholar
  45. Wager, E., Barbour, V., Yentis, S., & Kleinert, S. [on behalf of the COPE Council]. (2009). Retraction guidelines. Retrieved from
  46. Wikipedia. (2017). Séralini affair. Retrieved from

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2018

Authors and Affiliations

  1. 1.Bar-Ilan UniversityRamat GanIsrael
  2. 2.Icahn School of Medicine at Mount SinaiNew YorkUSA

Personalised recommendations