Skip to main content
SpringerLink
Log in

Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The h-index, introduced by Hirsch in 2005, was used by Schubert in 2009 to assess single publications. In 2011, Bornmann, Schier, Marx, and Daniel confirmed that the h-index is effective when assessing papers in chemistry. Quite a few Hirsch-type indices originate from the h-index. Can these Hirsch-type indices also be effectively used for assessing single publications? Will they behave the same or differently? In this study, the research objects were 26 kinds of Hirsch-type indices (including the original h-index) and three traditional methods, a total of 29 indicators. Based on the original definitions of these indicators and our new explanations of generations (i.e. mixed, pure, and non-pure generations of citations), we defined/redefined 29 paper-level metrics, calculated their values to assess publications, considered the correlations between those indices and the h-index or Wu’s w-index, and did factor analysis to contrast effectiveness. It was found that a few Hirsch-type indices (i.e. the f-index, rational h-index, real h-index, j-index, hg-index, Woeginger’s w-index, and tapered h-index) are highly correlated with the h-index but not close to Wu’s w-index, while some other indices (i.e. the a-index, h(5,2)-index, q2-index, r-index, maxprod, e-index, p-index, and weighted h-index) have relatively low correlations with the h-index but are close to Wu’s w-index. The normalized h-index and ph-ratio are obviously different from the other indices, and in most cases, their correlation coefficients with the h-index or Wu’s w-index are statistically non-significant (p > .05) or negative significant (p < .01). We argue that indices which are neither too near to nor too far from the h-index could be much more promising than others.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2010). hg-index: A new index to characterize the scientific output of researchers based on the h- and g-indices. Scientometrics, 82(2), 391–400.

    Article  Google Scholar 

  • Anderson, T., Hankin, R., & Killworth, P. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 76(3), 577–588.

    Article  Google Scholar 

  • Banks, M. G. (2006). An extension of the Hirsch index: Indexing scientific topics and compounds. Scientometrics, 69(1), 161–168.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. D. (2011a). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), 346–359.

    Article  Google Scholar 

  • Bornmann, L., Schier, H., Marx, W., & Daniel, H. D. (2011b). Does the h index for assessing single publications really work? A case study on papers published in chemistry. Scientometrics, 89(3), 835–843.

    Article  Google Scholar 

  • Braun, T., Glänzel, W., & Schubert, A. (2005). A Hirsch-type index for journals. The Scientist, 19(22), 8–10.

    Google Scholar 

  • Cabrerizo, F. J., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core. Journal of Informetrics, 4(1), 23–28.

    Article  Google Scholar 

  • Chan, H. F., Frey, B. S., Gallus, J., Schaffner, M., Torgler, B., & Whyte, S. (2016). External influence as an indicator of scholarly importance. CESifo Economic Studies, 62(1), 170–195.

    Article  Google Scholar 

  • Egghe, L. (2006a). An improvement of the h-index: The g-index. ISSI Newsletter, 2(1), 8–9.

    MathSciNet  Google Scholar 

  • Egghe, L. (2006b). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.

    Article  MathSciNet  Google Scholar 

  • Egghe, L. (2010). On the relation between Schubert’s h-index of a single paper and its total number of received citations. Scientometrics, 84(1), 115–117.

    Article  Google Scholar 

  • Egghe, L., & Rousseau, R. (2008). An h-index weighted by citation impact. Information Processing and Management, 44(2), 770–780.

    Article  Google Scholar 

  • Ellison, G. (2010). How does the market use citation data? The Hirsch index in economics. National Bureau of Economic Research. http://www.nber.org/papers/w16419. Accessed October 8, 2010.

  • Fragkiadaki, E., & Evangelidis, G. (2016). Three novel indirect indicators for the assessment of papers and authors based on generations of citations. Scientometrics, 106(2), 657–694.

    Article  Google Scholar 

  • Glänzel, W., & Schubert, A. (2010). Hirsch-type characteristics of the tail of distributions. The generalised h-index. Journal of Informetrics, 4(1), 118–123.

    Article  Google Scholar 

  • Guns, R., & Rousseau, R. (2009). Real and rational variants of the h-index and the g-index. Journal of Informetrics, 3(1), 64–71.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  MATH  Google Scholar 

  • Hu, X., Rousseau, R., & Chen, J. (2011). On the definition of forward and backward citation generations. Journal of Informetrics, 5(1), 27–36.

    Article  Google Scholar 

  • Jacsó, P. (2009). The h-index for countries in Web of Science and Scopus. Online Information Review, 33(4), 831–837.

    Article  Google Scholar 

  • Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.

    Article  Google Scholar 

  • Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.

    Google Scholar 

  • Kosmulski, M. (2007). MAXPROD-A new index for assessment of the scientific output of an individual, and a comparison. Cybermetrics , 11(1), 1–5.

    Google Scholar 

  • Kosmulski, M. (2013). Family-tree of bibliometric indices. Journal of Informetrics, 7(2), 313–317.

    Article  Google Scholar 

  • Lacasse, J. R., Hodge, D. R., & Bean, K. F. (2011). Evaluating the productivity of social work scholars using the h-Index. Research on Social Work Practice, 21(5), 599–607.

    Article  Google Scholar 

  • Mingers, J. (2009). Measuring the research contribution of management academics using the Hirsch-index. Journal of the Operational Research Society, 60(9), 1143–1153.

    Article  Google Scholar 

  • Molinari, A., & Molinari, J. F. (2008). Mathematical aspects of a new criterion for ranking scientific institutions based on the h-index. Scientometrics, 75(2), 339–356.

    Article  Google Scholar 

  • Mugnaini, R., Packer, A. L., & Meneghini, R. (2008). Comparison of scientists of the Brazilian Academy of Sciences and of the National Academy of Sciences of the USA on the basis of the h-index. Brazilian Journal of Medical and Biological Research, 41(4), 258–262.

    Article  Google Scholar 

  • Navon, D. (2009). The h-i index: A proposed new metric of individual scientific output. Cybermetrics, 13(1), paper 3.

  • Olden, J. D. (2007). How do ecological journals stack-up? Ranking of scientific quality according to the h index. Ecoscience, 14(3), 370–376.

    Article  Google Scholar 

  • Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.

    Article  Google Scholar 

  • Prathap, G. (2010a). The 100 most prolific economists using the p-index. Scientometrics, 84(1), 167–172.

    Article  Google Scholar 

  • Prathap, G. (2010b). Is there a place for a mock h-index? Scientometrics, 84(1), 153–165.

    Article  Google Scholar 

  • Rosenberg, M. S. (2014). A biologist’s guide to impact factors. PeerJ PrePrints 2: e477v1. doi:10.7287/peerj.preprints.477v1.

  • Rousseau, R. (1987). The Gozinto theorem: Using citations to determine influences on a scientific publication. Scientometrics, 11(3–4), 217–229.

    Article  Google Scholar 

  • Rousseau, R. (2006). New developments related to the Hirsch index. Science Focus, 1(4), 23–25. (in Chinese).

    Google Scholar 

  • Ruane, F., & Tol, R. S. J. (2008). Rational (successive) h-indices: An application to economics in the Republic of Ireland. Scientometrics, 75(2), 395–405.

    Article  Google Scholar 

  • Schreiber, M. (2010). Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers. Annalen der Physik, 522(8), 536–554.

    Article  Google Scholar 

  • Schubert, A. (2007). Successive h-indices. Scientometrics, 70(1), 201–205.

    Article  Google Scholar 

  • Schubert, A. (2009). Using the h-index for assessing single publications. Scientometrics, 78(3), 559–565.

    Article  Google Scholar 

  • Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics, 72(2), 253–280.

    Article  Google Scholar 

  • Thor, A., & Bornmann, L. (2011). The calculation of the single publicationhindex and related performance measures. Online Information Review, 35(2), 291–300.

    Article  Google Scholar 

  • Todeschini, R. (2011). The j-index: A new bibliometric index and multivariate comparisons between other common indices. Scientometrics, 87(3), 621–639.

    Article  Google Scholar 

  • Tol, R. S. J. (2009). The h-index and its alternatives: An application to the 100 most prolific economists. Scientometrics, 80(2), 317–324.

    Article  Google Scholar 

  • Turaga, K. K., & Gamblin, T. C. (2012). Measuring the surgical academic output of an institution: The “institutional” h-Index. Journal of Surgical Education, 69(4), 499–503.

    Article  Google Scholar 

  • Vinkler, P. (2009). The π-index: a new indicator for assessing scientific impact. Journal of Information Science, 35(5), 602–612.

    Article  Google Scholar 

  • Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365–391.

    Article  Google Scholar 

  • Watkins, M. W., & Chan-Park, C. Y. (2015). The research impact of school psychology faculty. Journal of School Psychology, 53(3), 231–241.

    Article  Google Scholar 

  • Wildgaard, L., Schneider, J. W., & Larsen, B. (2014). A review of the characteristics of 108 author-level bibliometric indicators. Scientometrics, 101(1), 125–158.

    Article  Google Scholar 

  • Woeginger, G. J. (2008). An axiomatic characterization of the Hirsch-index. Mathematical Social Sciences, 56(2), 224–232.

    Article  MathSciNet  MATH  Google Scholar 

  • Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers. Journal of the American Society for Information Science and Technology, 61(3), 609–614.

    Google Scholar 

  • Zhang, C.-T. (2009). The e-index, complementing the h-index for excess citations. PLoS One, 4(5), e5429.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to acknowledge the helpful comments of the anonymous reviewer. This research was supported in part by the National Natural Science Foundation of China (Grants 71273250 and 70973117).

Author information

Authors and Affiliations

  1. School of Management, University of Science and Technology of China, 96 Jinzhai Road, Hefei, 230026, China

    Zhenbin Yan, Qiang Wu & Xingchen Li

Authors
  1. Zhenbin Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingchen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiang Wu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, Z., Wu, Q. & Li, X. Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators. Scientometrics 109, 1815–1833 (2016). https://doi.org/10.1007/s11192-016-2147-3

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-016-2147-3

Keywords

Search

Navigation