Advertisement

Relative Value of Adapted Novel Bibliometrics in Evaluating Surgical Academic Impact and Reach

  • David B. T. RobinsonEmail author
  • Luke Hopkins
  • Chris Brown
  • Tarig Abdelrahman
  • Arfon G. Powell
  • Richard J. Egan
  • Wyn G. Lewis
Original Scientific Report

Abstract

Background

The Hirsch index, often used to assess research impact, suffers from questionable validity within the context of General Surgery, and consequently adapted bibliometrics and altmetrics have emerged, including the r-index, m-index, g-index and i10-index. This study aimed to assess the relative value of these novel bibliometrics in a single UK Deanery General Surgical Consultant cohort.

Method

Five indices (h, r, m, g and i10) and altmetric scores (AS) were calculated for 151 general surgical consultants in a UK Deanery. Indices and AS were calculated from publication data via the Scopus search engine with assessment of construct validity and reliability.

Results

The median number of publications, h-index, r-index, m-index, g-index and i10-index were 13 (range 0–389), 5 (range 0–63), 5.2 (range 0–64.8), 0.33 (range 0–1.5), 10 (range 0–125) and 4 (range 0–245), respectively. Correlation coefficients of r-index, m-index, g-index and i10-index with h-index were 0.913 (p < 0.001), 0.716 (p < 0.001), 0.961 (p < 0.001) and 0.939 (p < 0.001), respectively. Significant variance was observed when the cohort was ranked by individual bibliometric measures; the median ranking shifts were: r-index − 2 (− 46 to + 23); m-index − 6.5 (− 53 to + 22); g-index − 0.5 (− 24 to + 13); and i10-index 0 (− 8 to + 11), respectively (p < 0.001). The median altmetric score and AS index were 0 (range 0–225.5) and 1 (range 0–10), respectively; AS index correlated strongly with h-index (correlation coefficient 0.390, p < 0.001).

Conclusions

Adapted bibliometric indices appear to be equally valid measures of evaluating academic productivity, impact and reach.

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. 1.
    OECD and SCImago Research Group (CSIC) (2016) Compendium of bibliometric science indicators. http://oe.cd/scientometrics. Accessed 28/06/18
  2. 2.
    Otlet P (1934) Traité De Documentation: Le Livre Sur Le Livre, Théorie Et Pratique, Editiones Mundaneum: Mons, BelgiumGoogle Scholar
  3. 3.
    Pritchard A (1969) Statistical bibliography or bibliometrics? J Doc 25(4):348–349Google Scholar
  4. 4.
    Hirsch JE (2005) An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA 102(46):16569–16572CrossRefGoogle Scholar
  5. 5.
    Abdelrahman T, Brown J, Wheat J et al (2016) Hirsch index value and variability related to general surgery in a UK deanery. J Surg Educ. 73(1):111–115CrossRefGoogle Scholar
  6. 6.
    Garner RM, Hirsch JA, Albuquerque FC et al (2018) Bibliometric indices: defining academic productivity and citation rates of researchers, departments and journals. J NeuroInterv Surg 10:102–106CrossRefGoogle Scholar
  7. 7.
    Sharma B, Boet S, Grantcharov T et al (2013) The h-index outperforms other bibliometrics in the assessment of research performance in general surgery: a province-wide study. Surgery 153:493–501CrossRefGoogle Scholar
  8. 8.
    Svider PF, Pashkova AA, Choudhry Z et al (2013) Comparison of scholarly impact among surgical specialties: an examination of 2429 academic surgeons. Laryngoscope 123:884–889CrossRefGoogle Scholar
  9. 9.
    Bould MD, Boet S, Sharma B et al (2011) h-indices in a university department of anaesthesia: an evaluation of their feasibility, reliability, and validity as an assessment of academic performance. Br J Anaesth 106(3):325–330CrossRefGoogle Scholar
  10. 10.
    Hirsch J (2007) Does the h-index have predictive power? Proc Natl Acad Sci USA 104(49):19193–19198CrossRefGoogle Scholar
  11. 11.
    Sidropoulos A, Katsaros D, Manolopoulos Y (2007) Generalised Hirsh h-index for disclosing latent facts in citation networks. Scientometrics 72:253–280CrossRefGoogle Scholar
  12. 12.
    Romanovsky AA (2012) Revised h index for biomedical research. Cell Cycle 11(22):4118–4121CrossRefGoogle Scholar
  13. 13.
    Baerlocher MO, Newton M, Gautam T et al (2007) The meaning of author order in medical research. J Investig Med 55:174–180CrossRefGoogle Scholar
  14. 14.
    McInerney J (2011) h-Index, M-Index and Google Citations. http://mcinerneylab.com/research/h-index-m-index-and-google-citations/. Accessed 02/07/2018
  15. 15.
    Joint Committee on Surgical Training (2017/2018) Guidelines for the award of a CCT in General Surgery. https://www.jcst.org/quality-assurance/certification-guidelines-and-checklists/. Accessed 05/08/2018
  16. 16.
    Egghe L (2006) Theory and practice of the g-index. Scientometrics 69:131CrossRefGoogle Scholar
  17. 17.
    Sarkar S, Seshadri D (2015) Citation indices: measuring the “Impact” of published work. Indian J Psychol Med 37(3):376CrossRefGoogle Scholar
  18. 18.
    Kozak M, Bornmann L (2012) A new family of cumulative indexes for measuring scientific performance. PLoS ONE 7(10):e47679CrossRefGoogle Scholar
  19. 19.
    Altmetric Support. How is the Altmetric Attention Score calculated? https://help.altmetric.com/support/solutions/articles/6000060969-how-is-the-altmetric-attention-score-calculated. Accessed 30/08/18
  20. 20.
    Altmetric Support. When did Altmetric start tracking attention to each attention source? https://help.altmetric.com/support/solutions/articles/6000136884-when-did-altmetric-start-tracking-attention-to-each-attention-source. Accessed 28/08/18
  21. 21.
    Powell A, Bevan V, Brown C et al (2018) Altmetric versus bibliometric perspective regarding publication impact and force. World J Surg 42(9):2745–2756.  https://doi.org/10.1007/s00268-018-4579-9 CrossRefGoogle Scholar
  22. 22.
    The Economist Explains (2017) The problem with scientific publication, and how to fix it. https://www.economist.com/blogs/economist-explains/2017/03/economist-explains-23. Accessed 02/07/2018
  23. 23.
    Gingras Y (2016) Bibliometrics and research evaluation uses and abuses. MIT Press, MassachusettsGoogle Scholar
  24. 24.
    Matthew 13:12, Holy Bible: King James VersionGoogle Scholar
  25. 25.
    Fuller CD, Choi M, Thomas CR (2009) Bibliometric analysis of radiation oncology departmental scholarly publication productivity at domestic residency training institutions. J Am Coll Radiol 6(2):112–118CrossRefGoogle Scholar
  26. 26.
    Falagas ME, Pitsouni EI, Malietzis GA et al (2008) Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB J 22(2):338–342CrossRefGoogle Scholar
  27. 27.
    Lee J, Kraus KL, Couldwell WT (2009) Use of the h index in neurosurgery. Clinical article. J Neurosurg 111(2):387–392CrossRefGoogle Scholar

Copyright information

© Société Internationale de Chirurgie 2018

Authors and Affiliations

  1. 1.Wales Deanery PGMDE School of SurgeryHealth Education and Improvement WalesNantgarwUK
  2. 2.Division of Cancer and Genetics, Heath ParkCardiff University School of MedicineCardiffUK
  3. 3.Department of SurgeryMorriston HospitalSwanseaUK

Personalised recommendations