Skip to main content
Log in

Evaluation of h-index, its variants and extensions based on publication age & citation intensity in civil engineering

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The scientific community has proposed diversified set of parameters to rank researchers, including publications, citations, h-index, different variants and extensions of h-index. However, there is a debate in the scientific Community which index ranks authors in a better way. Current state-of-the-art depicts that these indices are evaluated on imaginary case scenarios and small datasets. Furthermore, these indices are evaluated on different datasets, making it difficult to grasp the contribution and importance of each index over the others. To analyze the individual behavior of each index, these indices should comprehensively be evaluated on some extensive data set. This study emphasizes on the scrutiny of h-index, some of its variants and extensions to rank authors. These indices are evaluated using a comprehensive data set of Civil Engineering field. For the evaluation of results obtained from these indices, first correlation was calculated among indices. There exists weak correlation between various indices, which demonstrates that the author’s rankings acquired from these indices are not identical. Secondly, occurrences of awardees are checked in all ranked lists. The prestigious award winners of four Civil Engineering societies are considered as a benchmark. In top 10% of ranked list, maximum 47% of the awardees were brought by Wu-index. Overall, none of the index succeeded in bringing 100% awardees to the top rankings. Highest number of awardees on top of all ranked lists are found to be from ACI (American Concrete Institute), which shows ACI might be dependent on these indices for its criterion to honor awards.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://cedb.asce.org/CEDBsearch/.

References

  • Afzal, M. T., & Maurer, H. A. (2011). Expertise recommender system for scientific community. Journal of Universal Computer Science, 17(11), 1529–1549.

    Google Scholar 

  • Aguillo, I. F. (2011). Is Google Scholar useful for bibliometrics? A webometric analysis. Scientometrics, 91(2), 343–351.

    Article  Google Scholar 

  • Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). h-Index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics, 3(4), 273–289.

  • Anderson, T., Hankin, R., & Killworth, P. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 76(3), 577–588.

    Article  Google Scholar 

  • Ayaz, S., & Afzal, M. T. (2016). Identification of conversion factor for completing-h index for the field of mathematics. Scientometrics, 109(3), 1511–1524.

    Article  Google Scholar 

  • Balog, K., Azzopardi, L., & De Rijke, M. (2006). Formal models for expert finding in enterprise corpora. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 43–50). ACM.

  • Beel, J., Gipp, B., & Wilde, E. (2009). Academic search engine optimization (ASEO): Optimizing scholarly literature for Google Scholar & Co. Journal of Scholarly Publishing, 41(2), 176–190.

    Google Scholar 

  • Belew, R. K. (2005). Scientific impact quantity and quality: Analysis of two sources of bibliographic data. arXiv preprint cs/0504036.

  • Bogers, T., & Van den Bosch, A. (2008). Recommending scientific articles using citeulike. In Proceedings of the 2008 ACM conference on Recommender systems (pp. 287–290). ACM.

  • Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), 346–359.

    Article  Google Scholar 

  • Burrell, Q. (2007). Hirsch index or Hirsch rate? Some thoughts arising from Liang’s data. Scientometrics, 73(1), 19–28.

    Article  Google Scholar 

  • Cabrerizo, F. J., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core. Journal of Informetrics, 4(1), 23–28.

    Article  Google Scholar 

  • Cameron, D. H. L., Aleman-Meza, B., Decker, S., & Arpinar, I. B. (2007). SEMEF: A taxonomy-based discovery of experts, expertise and collaboration networks (Doctoral dissertation, University of Georgia).

  • Corder, G. W., & Foreman, D. I. (2009). Comparing variables of ordinal or dichotomous scales: Spearman rank-order, Point-biserial, and biserial correlations (pp. 122–154). Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach.

    Google Scholar 

  • Czarnecki, L., Kaźmierkowski, M. P., & Rogalski, A. (2013). Doing Hirsch proud; shaping H-index in engineering sciences. Bulletin of the Polish Academy of Sciences: Technical Sciences, 61(1), 5–21.

    Google Scholar 

  • De Winter, J. C., Zadpoor, A. A., & Dodou, D. (2014). The expansion of Google Scholar versus web of science: A longitudinal study. Scientometrics, 98(2), 1547–1565.

    Article  Google Scholar 

  • Deng, H., Han, J., Lyu, M. R., & King, I. (2012). Modeling and exploiting heterogeneous bibliographic networks for expertise ranking. In Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries (pp. 71–80). ACM.

  • Dienes, K. R. (2015). Completing h. Journal of Informetrics, 9(2), 385–397.

    Article  MathSciNet  Google Scholar 

  • Egghe, L. (2006). An improvement of the h-index: The g-index. ISSI Newsletter, 2(1), 8–9.

    MathSciNet  Google Scholar 

  • Egghe, L. (2011). Characterizations of the generalized Wu- and Kosmulski-indices in Lotkaian systems. Journal of Informetrics, 5(3), 439–445.

    Article  Google Scholar 

  • Ellison, G. (2013). How does the market use citation data? The Hirsch index in economics. American Economic Journal: Applied Economics, 5(3), 63–90.

    Google Scholar 

  • Evans, J. D. (1996). Straightforward statistics for the behavioral sciences. Pacific Grove: Brooks/Cole publishing.

    Google Scholar 

  • Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008). Comparison of PubMed, Scopus, web of science, and Google Scholar: strengths and weaknesses. The FASEB Journal, 22(2), 338–342.

    Article  Google Scholar 

  • Fang, H., & Zhai, C. (2007). Probabilistic models for expert finding. In European Conference on Information Retrieval (pp. 418–430). Berlin: Springer.

  • Harzing, A. W. (2014). A longitudinal study of Google Scholar coverage between 2012 and 2013. Scientometrics, 98(1), 565–575.

    Article  Google Scholar 

  • Henderson, J. (2005). Google Scholar: A source for clinicians? Canadian Medical Association Journal, 172(12), 1549–1550.

    Article  Google Scholar 

  • Hirsch, Jorge E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  MATH  Google Scholar 

  • Jacsó, P. (2008). The pros and cons of computing the h-index using Google Scholar. Online Information Review, 32(3), 437–452.

  • Jin, B. (2006). H-index: an evaluation indicator proposed by scientist. Science Focus, 1(1), 8–9. (in Chinese).

    MathSciNet  Google Scholar 

  • Jin, B. (2007). The AR-index: complementing the h-index. ISSI Newsletter, 3(1), 6.

    Google Scholar 

  • Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.

    Article  Google Scholar 

  • Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.

    Google Scholar 

  • Kosmulski, M. (2007). MAXPROD—A new index for assessment of the scientific output of an individual, and a comparison. Cybermetrics, 11(1), 1–5.

    Google Scholar 

  • Kosmulski, M. (2013). Family-tree of bibliometric indices. Journal of Informetrics, 7(2), 313–317.

    Article  Google Scholar 

  • Liu, Y. & Rousseau, R. (2007). Hirsch-type indices and library management: The case of Tongji University Library. In 11th International Conference of the International Society for Scientrometrics and Informetrics, June 25–27, (pp. 514–522) Madrid, Spain.

  • Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105–2125.

    Article  Google Scholar 

  • Moed, H. F., Bar-Ilan, J., & Halevi, G. (2016). A new methodology for comparing Google Scholar and scopus. Journal of Informetrics, 10(2), 533–551.

    Article  Google Scholar 

  • Noruzi, A. (2005). Google Scholar: The new generation of citation indexes. Libri, 55(4), 170–180.

  • Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.

    Article  Google Scholar 

  • Rosenstreich, D., & Wooliscroft, B. (2009). Measuring the impact of accounting journals using Google Scholar and the g-index. The British Accounting Review, 41(4), 227–239.

    Article  Google Scholar 

  • Schreiber, M. (2008). An empirical investigation of the g-index for 26 physicists in comparison with the h-index, the A-index, and the R-index. Journal of the American Society for Information Science and Technology, 59(9), 1513–1522.

    Article  Google Scholar 

  • Schreiber, M. (2010). Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers. Annalen der Physik, 522(8), 536–554.

    Article  Google Scholar 

  • Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics, 72(2), 253–280.

    Article  Google Scholar 

  • Sun, J., Ma, J., Cheng, X., Liu, Z. & Cao, X. (2013). Finding an expert: A model recommendation system. In Thirty Fourth International Conference on Information Systems, Milan.

  • Tol, R. (2009). The h-index and its alternatives: An application to the 100 most prolific economists. Scientometrics, 80(2), 317–324.

    Article  Google Scholar 

  • Van Raan, A. F. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502.

    Article  Google Scholar 

  • Wildgaard, L., Schneider, J. W., & Larsen, B. (2014). A review of the characteristics of 108 author-level bibliometric indicators. Scientometrics, 10191, 125–158.

    Article  Google Scholar 

  • Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers. Journal of the American Society for Information Science and Technology, 61(3), 609–614.

    Google Scholar 

  • Wu, Q., & Zhang, P. (2017). Some indices violating the basic domination relation. Scientometrics, 113(1), 495–500.

    Article  Google Scholar 

  • Yan, Z., Wu, Q., & Li, X. (2016). Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators. Scientometrics, 109(3), 1815–1833.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Muhammad Tanvir Afzal.

Appendix 1

Appendix 1

See Table 7.

Table 7 Indices and their definition/formula

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Raheel, M., Ayaz, S. & Afzal, M.T. Evaluation of h-index, its variants and extensions based on publication age & citation intensity in civil engineering. Scientometrics 114, 1107–1127 (2018). https://doi.org/10.1007/s11192-017-2633-2

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-017-2633-2

Keywords

Navigation