Scientometrics

, Volume 115, Issue 2, pp 1087–1095 | Cite as

Proposal of a stochastic model to determine the bibliometric variables influencing the quality of a journal: application to the field of Dentistry

  • Pilar Valderrama
  • Manuel Escabias
  • Evaristo Jiménez-Contreras
  • Alberto Rodríguez-Archilla
  • Mariano J. Valderrama
Article
  • 86 Downloads

Abstract

On the basis of the Impact Factor of Journal Citation Reports developed by ISI as a journal quality indicator, this paper puts forth an ordinal regression model to estimate the journal’s position by terciles. The set of explanatory variables includes the H-index of its Editor-in-chief, percentage of papers published in the journal that received external funding, average number of papers published yearly, and two factors concerning the scope and structure of the journal. The proposed model was applied to the field of Dentistry, Oral Surgery and Medicine, and led us to the conclusion that the above mentioned covariables alone had a significant input in the model, but not the factors. The essay performed on a sample of 30 Dentistry journals included in JCR provided a confirmatory correct classification rate (CCR) of 80%, with a predictive CCR of 75% on a sample of eight new journals not previously considered in the phase of model estimation.

Keywords

Dentistry H-index Ordinal regression Journal Citation Reports Correct classification rate 

Notes

Acknowledgements

This work was supported by MTM2017-88708-P of Secretaría de Estado de Investigación, Desarrollo e Innovación, Ministerio de Economía y Competitividad de España.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflicts of interest.

References

  1. Alberts, B. (2013). Impact factor distortions. Science, 340(6134), 787.CrossRefGoogle Scholar
  2. Althouse, B. M., West, J. D., Bergstrom, C. T., & Bergstrom, T. (2009). Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology, 60(1), 27–34.CrossRefGoogle Scholar
  3. Anscombe, F. J. (1948). The validity of comparative experiments. Journal of the Royal Statistical Society, Series A, 111(3), 181–211.MathSciNetCrossRefGoogle Scholar
  4. Bollen, J., Van de Sompel, H., Hagberg, A., & Chute, R. (2009). A principal component analysis of 39 scientific impact measures. PLOS ONE, 4(6), 0006022.  https://doi.org/10.1371/journal.pone.CrossRefGoogle Scholar
  5. Bordons, M., Fernandez, M. T., & Gomez, I. (2002). Advantages and limitations in the use of impact factor measures for the assessment of research performance in a peripheral country. Scientometrics, 53(2), 195–206.CrossRefGoogle Scholar
  6. Bornmann, L., Marx, W., Gasparyan, A. Y., & Kitas, G. D. (2012). Diversity, value and limitations of the journal impact factor and alternative metrics. Rheumatology International, 32(7), 1861–1867.CrossRefGoogle Scholar
  7. Bornmann, L., Mutz, R., Marx, W., Schier, H., & Daniel, H. D. (2011). A multilevel modeling approach to investigating the predictive validity of editorial decisions: Do the editors of a high profile journal select manuscripts that are highly cited after publication? Journal of the Royal Statistical Society, Series A, 174(4), 857–879.MathSciNetCrossRefGoogle Scholar
  8. Bornmann, L., & Williams, R. (2017). Can the journal impact factor be used as a criterion for the selection of junior researchers? A large-scale empirical study based on Researcher ID data. Journal of Informetrics, 11(3), 788–799.CrossRefGoogle Scholar
  9. Buela, G. (2003). Evaluating quality of articles and scientific journals. Proposal of weighted impact factor and a quality index? Psicothema, 15(1), 23–35.Google Scholar
  10. Falagas, M. E., & Alexiou, V. G. (2008). The top-ten in journal impact factor manipulation. Archivum Immunologiae et Therapiae Experimentalis, 56(4), 223–226.CrossRefGoogle Scholar
  11. Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008a). Comparison of SCImago journal rank indicator with journal impact factor. FASEB Journal, 22(8), 2623–2628.CrossRefGoogle Scholar
  12. Fassoulaki, A., Papilas, K., Paraskeva, A., & Patris, N. (2002). Impact factor bias and proposed adjustments for its determination. Acta Anaetesiologica Scandinavica, 46(7), 902–905.CrossRefGoogle Scholar
  13. Fassoulaki, A., Paraskeva, A., Papilas, K., & Karabinis, G. (2000). Self-citations in six anaesthesia journals and their significance in determining the impact factor. British Journal of Anaesthesia, 84(2), 266–269.CrossRefGoogle Scholar
  14. Garfield, E. (1955). Citation indexes for science: A new dimension in documentation through association of ideas. Science, 122(3159), 108–111.CrossRefGoogle Scholar
  15. Garfield, E. (1999). Journal impact factor: A brief review. Canadian Medical Association Journal, 161, 979–980.Google Scholar
  16. Garfield, E. (2006). The history and meaning of the journal impact factor. Journal of the American Medical Association, 295(1), 90–93.CrossRefGoogle Scholar
  17. Hansson, S. (1995). Impact factor as a misleading tool in evaluation of medical journals. The Lancet, 346(8979), 906–906.CrossRefGoogle Scholar
  18. Kay, J., Memon, M., de Sa, D., Simunovic, N., Duong, A., Karlsson, J., et al. (2017). The H-index of editorial board members correlates positively with the impact factor of Sports Medicine journals. Orthopaedic Journal of Sports Medicine, 5(3), 2325967117694024.  https://doi.org/10.1177/2325967117694024.CrossRefGoogle Scholar
  19. Kurmis, A. P. (2003). Understanding the limitations of the journal impact factor. The Journal of Bone and Joint Surgery, 85(12), 2449–2454.CrossRefGoogle Scholar
  20. Leydesdorff, L. (2012). Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers. Scientometrics, 92(2), 355–365.CrossRefGoogle Scholar
  21. Leydesdorff, L., & Bornmann, L. (2011). How fractional counting of citations affects the impact factor: Normalization in terms of differences in citation potentials among fields of Science. Journal of the American Society for Information Science and Technology, 62(2), 217–229.CrossRefGoogle Scholar
  22. Leydesdorff, L., & Opthof, T. (2010). Scopus’s source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. Journal of the American Society for Information Science and Technology, 61(11), 2365–2369.CrossRefGoogle Scholar
  23. Lucena, C., Souza, E. M., Voinea, G. C., Pulgar, R., Valderrama, M. J., & De-Deus, G. (2017). A quality assessment of randomized controlled trial reports in Endodontics. International Endodontic Journal, 50(3), 237–250.CrossRefGoogle Scholar
  24. Malay, D. S. (2013). Impact factors and other measures of a journal’s influence. The Journal of Food and Ankle Surgery, 52(3), 285–287.CrossRefGoogle Scholar
  25. McVeigh, M. F., & Mann, S. J. (2009). The journal impact factor denominator defining citable (counted) items. Journal of the American Medical Association, 302(10), 1107–1109.CrossRefGoogle Scholar
  26. Moed, H. F. (2002). The impact-factors debate: The ISI’s uses and limits. Nature, 415(6873), 731–732.CrossRefGoogle Scholar
  27. Moed, H. F., & Vanleeuwen, T. N. (1995). Improving the accuracy of Institute for Scientific Information journal impact factors. Journal of the American Society for Information Science and Technology, 46(6), 461–467.CrossRefGoogle Scholar
  28. Mutz, R., & Daniel, H.-D. (2012a). The generalized propensity score methodology for estimating unbiased journal impact factors. Scientometrics, 92, 377–390.CrossRefGoogle Scholar
  29. Mutz, R., & Daniel, H.-D. (2012b). Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor. Journal of Informetrics, 6(2), 169–176.CrossRefGoogle Scholar
  30. Roberts, R. J. (2017). An obituary for the impact factor. Nature, 546(7660), 600.CrossRefGoogle Scholar
  31. Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91(1), 42–46.Google Scholar
  32. Seglen, P. O. (1997a). Citations and journal impact factors: Questionable indicators of research quality. Allergy, 52(11), 1050–1056.CrossRefGoogle Scholar
  33. Seglen, P. O. (1997b). Why the impact factor of journals should not be used for evaluating research. British Medical Journal, 314(7079), 498–502.CrossRefGoogle Scholar
  34. Seglen, P. O. (1998). Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthopaedica Scandinavica, 69(3), 224–229.CrossRefGoogle Scholar
  35. Simons, K. (2008). The misused impact factor. Science, 322(5899), 165.CrossRefGoogle Scholar
  36. Valderrama, P., Escabias, M., Jiménez-Contreras, E. Valderrama, M. J. & Baca, P. (2017). Bibliometric variables determining the quality of a dentistry journal. In Skiadas, C. H. (Ed.), Proceedings of the 17th conference of the applied stochastic models and data analysis international society, pp. 825–831.Google Scholar
  37. Van Noorden, R. (2016). Impact factor gets heavyweight rival. Nature, 540(7633), 325–326.CrossRefGoogle Scholar
  38. Vanclay, J. K. (2012). Impact factor: Outdated artefact or stepping-stone to journal certification? Scientometrics, 92(2), 211–238.CrossRefGoogle Scholar
  39. Varki, A. (2017). Rename the impact factor. Nature, 548(7668), 393.CrossRefGoogle Scholar
  40. Wagner, G., Prester, J., Roche, M., Benlian, A., & Schryen, G. (2006). Factors affecting the scientific impact of literature reviews: A scientometric study. In proceedings of the 37th international conference on information systems, Dublin 2016, Vol. 23, pp. 1659–1682.Google Scholar
  41. Wilsdon, J., et al. (2015). The metric tide: Report of the independent review of the role of metrics in research assessment and management. HEFCE.  https://doi.org/10.13140/RG.2.1.4929.1363.
  42. Zitt, M., & Small, H. (2008). Modifying the journal impact factor by fractional citation weighting: The audience factor. Journal of the American Society for Information Science and Technology, 59(11), 1856–1860.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2018

Authors and Affiliations

  1. 1.Department of Statistics and Operations ResearchUniversity of GranadaGranadaSpain
  2. 2.Department of Information and CommunicationUniversity of GranadaGranadaSpain
  3. 3.Department of DentistryUniversity of GranadaGranadaSpain

Personalised recommendations