, Volume 114, Issue 3, pp 1107–1127 | Cite as

Evaluation of h-index, its variants and extensions based on publication age & citation intensity in civil engineering

  • Muhammad Raheel
  • Samreen Ayaz
  • Muhammad Tanvir Afzal


The scientific community has proposed diversified set of parameters to rank researchers, including publications, citations, h-index, different variants and extensions of h-index. However, there is a debate in the scientific Community which index ranks authors in a better way. Current state-of-the-art depicts that these indices are evaluated on imaginary case scenarios and small datasets. Furthermore, these indices are evaluated on different datasets, making it difficult to grasp the contribution and importance of each index over the others. To analyze the individual behavior of each index, these indices should comprehensively be evaluated on some extensive data set. This study emphasizes on the scrutiny of h-index, some of its variants and extensions to rank authors. These indices are evaluated using a comprehensive data set of Civil Engineering field. For the evaluation of results obtained from these indices, first correlation was calculated among indices. There exists weak correlation between various indices, which demonstrates that the author’s rankings acquired from these indices are not identical. Secondly, occurrences of awardees are checked in all ranked lists. The prestigious award winners of four Civil Engineering societies are considered as a benchmark. In top 10% of ranked list, maximum 47% of the awardees were brought by Wu-index. Overall, none of the index succeeded in bringing 100% awardees to the top rankings. Highest number of awardees on top of all ranked lists are found to be from ACI (American Concrete Institute), which shows ACI might be dependent on these indices for its criterion to honor awards.


ASCE CEDB h-Index Variants of h-index Civil engineering field Author ranking 


  1. Afzal, M. T., & Maurer, H. A. (2011). Expertise recommender system for scientific community. Journal of Universal Computer Science, 17(11), 1529–1549.Google Scholar
  2. Aguillo, I. F. (2011). Is Google Scholar useful for bibliometrics? A webometric analysis. Scientometrics, 91(2), 343–351.CrossRefGoogle Scholar
  3. Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). h-Index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics, 3(4), 273–289.Google Scholar
  4. Anderson, T., Hankin, R., & Killworth, P. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 76(3), 577–588.CrossRefGoogle Scholar
  5. Ayaz, S., & Afzal, M. T. (2016). Identification of conversion factor for completing-h index for the field of mathematics. Scientometrics, 109(3), 1511–1524.CrossRefGoogle Scholar
  6. Balog, K., Azzopardi, L., & De Rijke, M. (2006). Formal models for expert finding in enterprise corpora. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 43–50). ACM.Google Scholar
  7. Beel, J., Gipp, B., & Wilde, E. (2009). Academic search engine optimization (ASEO): Optimizing scholarly literature for Google Scholar & Co. Journal of Scholarly Publishing, 41(2), 176–190.Google Scholar
  8. Belew, R. K. (2005). Scientific impact quantity and quality: Analysis of two sources of bibliographic data. arXiv preprint cs/0504036.Google Scholar
  9. Bogers, T., & Van den Bosch, A. (2008). Recommending scientific articles using citeulike. In Proceedings of the 2008 ACM conference on Recommender systems (pp. 287–290). ACM.Google Scholar
  10. Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), 346–359.CrossRefGoogle Scholar
  11. Burrell, Q. (2007). Hirsch index or Hirsch rate? Some thoughts arising from Liang’s data. Scientometrics, 73(1), 19–28.CrossRefGoogle Scholar
  12. Cabrerizo, F. J., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core. Journal of Informetrics, 4(1), 23–28.CrossRefGoogle Scholar
  13. Cameron, D. H. L., Aleman-Meza, B., Decker, S., & Arpinar, I. B. (2007). SEMEF: A taxonomy-based discovery of experts, expertise and collaboration networks (Doctoral dissertation, University of Georgia).Google Scholar
  14. Corder, G. W., & Foreman, D. I. (2009). Comparing variables of ordinal or dichotomous scales: Spearman rank-order, Point-biserial, and biserial correlations (pp. 122–154). Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach.Google Scholar
  15. Czarnecki, L., Kaźmierkowski, M. P., & Rogalski, A. (2013). Doing Hirsch proud; shaping H-index in engineering sciences. Bulletin of the Polish Academy of Sciences: Technical Sciences, 61(1), 5–21.Google Scholar
  16. De Winter, J. C., Zadpoor, A. A., & Dodou, D. (2014). The expansion of Google Scholar versus web of science: A longitudinal study. Scientometrics, 98(2), 1547–1565.CrossRefGoogle Scholar
  17. Deng, H., Han, J., Lyu, M. R., & King, I. (2012). Modeling and exploiting heterogeneous bibliographic networks for expertise ranking. In Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries (pp. 71–80). ACM.Google Scholar
  18. Dienes, K. R. (2015). Completing h. Journal of Informetrics, 9(2), 385–397.MathSciNetCrossRefGoogle Scholar
  19. Egghe, L. (2006). An improvement of the h-index: The g-index. ISSI Newsletter, 2(1), 8–9.MathSciNetGoogle Scholar
  20. Egghe, L. (2011). Characterizations of the generalized Wu- and Kosmulski-indices in Lotkaian systems. Journal of Informetrics, 5(3), 439–445.CrossRefGoogle Scholar
  21. Ellison, G. (2013). How does the market use citation data? The Hirsch index in economics. American Economic Journal: Applied Economics, 5(3), 63–90.Google Scholar
  22. Evans, J. D. (1996). Straightforward statistics for the behavioral sciences. Pacific Grove: Brooks/Cole publishing.Google Scholar
  23. Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008). Comparison of PubMed, Scopus, web of science, and Google Scholar: strengths and weaknesses. The FASEB Journal, 22(2), 338–342.CrossRefGoogle Scholar
  24. Fang, H., & Zhai, C. (2007). Probabilistic models for expert finding. In European Conference on Information Retrieval (pp. 418–430). Berlin: Springer.Google Scholar
  25. Harzing, A. W. (2014). A longitudinal study of Google Scholar coverage between 2012 and 2013. Scientometrics, 98(1), 565–575.CrossRefGoogle Scholar
  26. Henderson, J. (2005). Google Scholar: A source for clinicians? Canadian Medical Association Journal, 172(12), 1549–1550.CrossRefGoogle Scholar
  27. Hirsch, Jorge E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.CrossRefMATHGoogle Scholar
  28. Jacsó, P. (2008). The pros and cons of computing the h-index using Google Scholar. Online Information Review, 32(3), 437–452.Google Scholar
  29. Jin, B. (2006). H-index: an evaluation indicator proposed by scientist. Science Focus, 1(1), 8–9. (in Chinese).MathSciNetGoogle Scholar
  30. Jin, B. (2007). The AR-index: complementing the h-index. ISSI Newsletter, 3(1), 6.Google Scholar
  31. Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.CrossRefGoogle Scholar
  32. Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.Google Scholar
  33. Kosmulski, M. (2007). MAXPROD—A new index for assessment of the scientific output of an individual, and a comparison. Cybermetrics, 11(1), 1–5.Google Scholar
  34. Kosmulski, M. (2013). Family-tree of bibliometric indices. Journal of Informetrics, 7(2), 313–317.CrossRefGoogle Scholar
  35. Liu, Y. & Rousseau, R. (2007). Hirsch-type indices and library management: The case of Tongji University Library. In 11th International Conference of the International Society for Scientrometrics and Informetrics, June 25–27, (pp. 514–522) Madrid, Spain.Google Scholar
  36. Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105–2125.CrossRefGoogle Scholar
  37. Moed, H. F., Bar-Ilan, J., & Halevi, G. (2016). A new methodology for comparing Google Scholar and scopus. Journal of Informetrics, 10(2), 533–551.CrossRefGoogle Scholar
  38. Noruzi, A. (2005). Google Scholar: The new generation of citation indexes. Libri, 55(4), 170–180.Google Scholar
  39. Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.CrossRefGoogle Scholar
  40. Rosenstreich, D., & Wooliscroft, B. (2009). Measuring the impact of accounting journals using Google Scholar and the g-index. The British Accounting Review, 41(4), 227–239.CrossRefGoogle Scholar
  41. Schreiber, M. (2008). An empirical investigation of the g-index for 26 physicists in comparison with the h-index, the A-index, and the R-index. Journal of the American Society for Information Science and Technology, 59(9), 1513–1522.CrossRefGoogle Scholar
  42. Schreiber, M. (2010). Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers. Annalen der Physik, 522(8), 536–554.CrossRefGoogle Scholar
  43. Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics, 72(2), 253–280.CrossRefGoogle Scholar
  44. Sun, J., Ma, J., Cheng, X., Liu, Z. & Cao, X. (2013). Finding an expert: A model recommendation system. In Thirty Fourth International Conference on Information Systems, Milan.Google Scholar
  45. Tol, R. (2009). The h-index and its alternatives: An application to the 100 most prolific economists. Scientometrics, 80(2), 317–324.CrossRefGoogle Scholar
  46. Van Raan, A. F. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502.CrossRefGoogle Scholar
  47. Wildgaard, L., Schneider, J. W., & Larsen, B. (2014). A review of the characteristics of 108 author-level bibliometric indicators. Scientometrics, 10191, 125–158.CrossRefGoogle Scholar
  48. Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers. Journal of the American Society for Information Science and Technology, 61(3), 609–614.Google Scholar
  49. Wu, Q., & Zhang, P. (2017). Some indices violating the basic domination relation. Scientometrics, 113(1), 495–500.CrossRefGoogle Scholar
  50. Yan, Z., Wu, Q., & Li, X. (2016). Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators. Scientometrics, 109(3), 1815–1833.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceCapital University of Science and TechnologyIslamabadPakistan

Personalised recommendations