, Volume 121, Issue 2, pp 653–673 | Cite as

Evaluation of h-index and its qualitative and quantitative variants in Neuroscience

  • Madiha Ameer
  • Muhammad Tanvir AfzalEmail author


In the scientific community, different parameters such as, publications, citations, h-index and different variants of h-index are being used to rank the authors. Authors ranking assists in various objectives, including scientific domain expert’s selection and subsequently utilizing their services for different purposes. It can also be used in ranking institutions, selecting journal’s editors, tenure appointments and awarding promotions. There is a need to nominate the best performing index that can rank the authors in a best possible way. All of the existing parameters are either examined on fictional case scenarios or on insufficient datasets belonging to different fields, due to which it is hard to figure out the comparative performance of each index. In order to strongly validate the performance of each individual index these indices should be examined on the large size dataset. This research focuses attention on the evaluation of h-index and some of its variants, both quantitative and qualitative for ranking authors. All these indices are evaluated on a large size dataset belonging to the field of Neuroscience. To evaluate the results of these indices award winners of five different Neuroscience societies are considered as a benchmark. This study will be helpful in distinguishing more dominant authors based on the evaluation results of these quantitative and qualitative indices. In order to evaluate the results obtained from these indices, firstly the correlation among the indices is calculated. Most of the indices such as h-index, hg-index, R-index and m-quotient have strong correlation among them. However, the correlation of f-index with all other indices is weak, which indicates that ranking achieved by the f-index is not similar to others. Secondly the occurrence of award winners in ranked list of each index is calculated and contribution of each index in bringing awardees in top position is checked. None of the index brought 100% of awardees in top positions of the ranking. In quantitative indices hg-index brought maximum 52% awardees in top 10% of its ranked list. While in qualitative indices R-index validated itself as a best performing index by ranking 52% awardees in top 10% of its ranked list. Moreover, it is analyzed that most of the award winners belonging to the SfN (Society for Neuroscience) and CNS (Cognitive Neuroscience Society) lies on the top positions of ranked lists obtained from h-index and hg-index which specify that there might be a relationship between SfN, NAS and the h and hg-indices.


Authors ranking Variants of h-index Neuroscience field Quantitative indices Qualitative indices SfANS 


  1. Ain, Q. U., Riaz, H., & Afzal, M. T. (2019). Evaluation of h-index and its citation intensity based variants in the field of mathematics. Scientometrics, 119(1), 187–211.CrossRefGoogle Scholar
  2. Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). h-Index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics, 3(4), 273–289.CrossRefGoogle Scholar
  3. Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2010). hg-index: A new index to characterize the scientific output of researchers based on the h-and g-indices. Scientometrics, 82(2), 391–400.CrossRefGoogle Scholar
  4. Aoun, S. G., Bendok, B. R., Rahme, R. J., Dacey, R. G., & Batjer, H. H. (2013). Standardizing the evaluation of scientific and academic performance in neurosurgery—Critical review of the “h” index and its variants. World Neurosurgery, 80(5), e85–e90.CrossRefGoogle Scholar
  5. Ayaz, S., & Afzal, M. T. (2016). Identification of conversion factor for completing-h index for the field of mathematics. Scientometrics, 109(3), 1511–1524.CrossRefGoogle Scholar
  6. Balog, K., Azzopardi, L., & De Rijke, M. (2006, August). Formal models for expert finding in enterprise corpora. In Proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval (pp. 43–50). ACM.Google Scholar
  7. Bornmann, L. (2011). Do we need the e-index in addition to the h-index and its variants? Journal of the American Society for Information Science and Technology, 62(7), 1433–1434.CrossRefGoogle Scholar
  8. Bornmann, L., & Marx, W. (2011). The h index as a research performance indicator. European Science Editing, 37(3), 77–80.Google Scholar
  9. Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the Association for Information Science and Technology, 59(5), 830–837.Google Scholar
  10. Bornmann, L., Mutz, R., & Daniel, H. D. (2009a). Do we need the h index and its variants in addition to standard bibliometric measures? Journal of the Association for Information Science and Technology, 60(6), 1286–1289.Google Scholar
  11. Bornmann, L., Mutz, R., & Daniel, H. D. (2010). The h index research output measurement: Two approaches to enhance its accuracy. Journal of Informetrics, 4(3), 407–414.CrossRefGoogle Scholar
  12. Bornmann, L., Mutz, R., Daniel, H. D., Wallon, G., & Ledin, A. (2009b). Are there really two types of h index variants? A validation study by using molecular life sciences data. Research Evaluation, 18(3), 185–190.CrossRefGoogle Scholar
  13. Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), 346–359.CrossRefGoogle Scholar
  14. Cabrerizo, F. J., Alonso, S., Herrera-Viedma, E., & Herrera, F. (2010). q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core. Journal of Informetrics, 4(1), 23–28.CrossRefGoogle Scholar
  15. Cameron, D. H. L., Aleman-Meza, B., Decker, S., & Arpinar, I. B. (2007). SEMEF: A taxonomy-based discovery of experts, expertise and collaboration networks. Doctoral dissertation, University of Georgia.Google Scholar
  16. Chen, Y., & Hoshower, L. B. (2003). Student evaluation of teaching effectiveness: An assessment of student perception and motivation. Assessment and Evaluation in Higher Education, 28(1), 71–88.CrossRefGoogle Scholar
  17. Clayson, D. E. (2009). Student evaluations of teaching: Are they related to what students learn? A meta-analysis and review of the literature. Journal of Marketing Education, 31(1), 16–30.CrossRefGoogle Scholar
  18. Corder, G. W., & Foreman, D. I. (2009). Comparing variables of ordinal or dichotomous scales: Spearman rank-order, point-biserial, and biserial correlations. In Nonparametric statistics for non-statisticians: A step-by-step approach (pp. 122–154).Google Scholar
  19. Davison, E., & Price, J. (2009). How do we rate? An evaluation of online student evaluations. Assessment and Evaluation in Higher Education, 34(1), 51–65.CrossRefGoogle Scholar
  20. Dorta-González, P., & Dorta-González, M. I. (2013). The student evaluation of teaching and the competence of students as evaluators. arXiv preprint arXiv:1301.7628.
  21. Egghe, L. (2006). An improvement of the h-index: The g-index. ISSI Newsletter, 2, 8–9.Google Scholar
  22. Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008). Comparison of PubMed, Scopus, Web of science, and Google Scholar: Strengths and weaknesses. The FASEB Journal, 22(2), 338–342.CrossRefGoogle Scholar
  23. Gardner, D., Akil, H., Ascoli, G. A., Bowden, D. M., Bug, W., Donohue, D. E., et al. (2008). The neuroscience information framework: A data and knowledge environment for neuroscience. Neuroinformatics, 6(3), 149–160.CrossRefGoogle Scholar
  24. Ghani, R., Qayyum, F., Afzal, M. T., & Maurer, H. (2019). Comprehensive evaluation of h-index and its extensions in the domain of mathematics. Scientometrics, 118(3), 809–822.CrossRefGoogle Scholar
  25. Harzing, A. W. (2014). A longitudinal study of Google Scholar coverage between 2012 and 2013. Scientometrics, 98(1), 565–575.CrossRefGoogle Scholar
  26. Harzing, A. W., & Alakangas, S. (2016). Google Scholar, Scopus and the Web of science: A longitudinal and cross-disciplinary comparison. Scientometrics, 106(2), 787–804.CrossRefGoogle Scholar
  27. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.CrossRefGoogle Scholar
  28. Hirsch, J. E. (2010). An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship. Scientometrics, 85(3), 741–754.CrossRefGoogle Scholar
  29. Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R-and ar-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.CrossRefGoogle Scholar
  30. Lowry, P., Moody, G., Gaskin, J., Galletta, D., Humphreys, S., Barlow, J., et al. (2013). Evaluating journal quality and the association for information systems (AIS) senior scholars’ journal basket via bibliometric measures: Do expert journal assessments add value?. Rochester: Social Science Research Network.Google Scholar
  31. Raheel, M., Ayaz, S., & Afzal, M. T. (2018). Evaluation of h-index, its variants and extensions based on publication age and citation intensity in civil engineering. Scientometrics, 114(3), 1107–1127.CrossRefGoogle Scholar
  32. Rousseau, R. (2006). New developments related to the Hirsch index.
  33. Schreiber, M., Malesios, C. C., & Psarakis, S. (2011). Categorizing h-index variants. Research Evaluation, 20(5), 397–409.CrossRefGoogle Scholar
  34. Sedgwick, P. (2012). Pearson’s correlation coefficient. Bmj, 345, e4483.CrossRefGoogle Scholar
  35. Slate, J. R., LaPrairie, K. N., Schulte, D. P., & Onwuegbuzie, A. J. (2011). Views of effective college faculty: A mixed analysis. Assessment and Evaluation in Higher Education, 36(3), 331–346.CrossRefGoogle Scholar
  36. Tol, R. (2009). The h-index and its alternatives: An application to the 100 most prolific economists. Scientometrics, 80(2), 317–324.CrossRefGoogle Scholar
  37. Van Raan, A. F. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502.CrossRefGoogle Scholar
  38. Waltman, L., & Van Eck, N. J. (2012). The inconsistency of the h-index. Journal of the Association for Information Science and Technology, 63(2), 406–415.Google Scholar
  39. Wu, Q. (2010). The w-index: A measure to assess scientific impact by focusing on widely cited papers. Journal of the Association for Information Science and Technology, 61(3), 609–614.Google Scholar
  40. Zhang, C. T. (2009). The e-index, complementing the h-index for excess citations. PLoS ONE, 4(5), e5429.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2019

Authors and Affiliations

  1. 1.Capital University of Science and TechnologyIslamabadPakistan

Personalised recommendations