Skip to main content
SpringerLink
Log in

Designing a Composite Index for research performance evaluation at the national or regional level: ranking Central Universities in India

  • Published:
Scientometrics Aims and scope Submit manuscript

An Erratum to this article was published on 17 June 2016

Abstract

It is now generally accepted that institutions of higher education and research, largely publicly funded, need to be subjected to some benchmarking process or performance evaluation. Currently there are several international ranking exercises that rank institutions at the global level, using a variety of performance criteria such as research publication data, citations, awards and reputation surveys etc. In these ranking exercises, the data are combined in specified ways to create an index which is then used to rank the institutions. These lists are generally limited to the top 500–1000 institutions in the world. Further, some criteria (e.g., the Nobel Prize), used in some of the ranking exercises, are not relevant for the large number of institutions that are in the medium range. In this paper we propose a multidimensional ‘Quality–Quantity’ Composite Index for a group of institutions using bibliometric data, that can be used for ranking and for decision making or policy purposes at the national or regional level. The index is applied here to rank Central Universities in India. The ranks obtained compare well with those obtained with the h-index and partially with the size-dependent Leiden ranking and University Ranking by Academic Performance. A generalized model for the index using other variables and variable weights is proposed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://www.shanghairanking.com/.

  2. http://www.leidenranking.com/methodology/indicator.

  3. http://www.webometrics.info/en/Methodology.

  4. http://www.scimagoir.com/methodology.php.

  5. https://www.timeshighereducation.co.uk/.

  6. http://www.topuniversities.com/.

  7. http://www.urapcenter.org/2014/index.php.

  8. www.ugc.ac.in.

  9. https://en.wikipedia.org/wiki/University_Grants_Commission_(India).

  10. http://www.ugc.ac.in/pdfnews/0342004_vacant-position-CU-as-on-01-01-2014.pdf.

  11. No. of papers of an institution in top 1 % of cited papers, or HiCP, where cited papers includes only papers from the current set of institutions.

  12. Journal Citation Reports-Thompson Reuters, https://jcr.incites.thomsonreuters.com/.

References

  • Academic Ranking of World Universities. (2011). Ranking methodology. Available at: www.arwu.org/ARWUMethodology2007.jsp. Accessed May 11, 2015.

  • Aguillo, I. F., Ortega, J. L., & Fernández, M. (2008). Webometric ranking of world universities: Introduction, methodology, and future developments. Higher Education in Europe, 33(2–3), 233–244.

    Article  Google Scholar 

  • Basu, A., & Aggarwal, R. (2001). International collaboration in science in India and its impact on institutional performance. Scientometrics, 52(3), 379–394.

    Article  Google Scholar 

  • Basu, A., Aggarwal, R., Kumar, D., Kathuria, G., Jha, S., & Rani, T. (2000). Scientific productivity…where do we stand? Second brief report on bibliometric indicators of Indianscience, NISTADS Report, January 2000.

  • Basu, A., & Nagpaul, P. S. (1998). National mapping of science. NISTADS Report # REP-248/98.

  • Billaut, J. C., Bouyssou, D., & Vincke, P. (2010). Should you believe in the Shanghai ranking? An MCDM view. Scientometrics, 84(1), 237–263.

    Article  Google Scholar 

  • Bornmann, L. (2013). How to analyze percentile impact data meaningfully in bibliometrics? The statistical analysis of distributions, percentile rank classes, and top-cited papers. Journal of the Association for Information Science and Technology, 64(3), 587–595.

    Article  Google Scholar 

  • Bornmann, L., Bowman, B. F., Bauer, J., Marx, W., Schier, H., & Palzenberger, M. (2014), Bibliometric standards for evaluating research institutes in the natural sciences. In B. Cronin, C. R. Sugimoto (Eds.), Beyond bibliometrics (pp. 201–224). Massachusetts Institute of Technology. ISBN: 978-0-262-525551-0.

  • Bornmann, L., & Marx, W. (2014). How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations. Scientometrics, 98(1), 487–509.

    Article  Google Scholar 

  • Bornmann, L., Moritz, S., Felix de Moya, A., & Rüdiger, M. (2014b). What is the effect of country-specific characteristics on the research performance of scientific institutions? Using multi-level statistical models to rank and map universities and research-focused institutions worldwide. Journal of Informetrics, 8(3), 581–593.

    Article  Google Scholar 

  • Bornmann, L., Moritz, S., Felix de Moya, A., & Rüdiger, M. (2015). Ranking and mapping of universities and research-focused institutions worldwide: The third release of excellencemapping.net. Collnet Journal of Scientometrics and Information Management, 9(1), 65–72.

    Article  Google Scholar 

  • Bradford, S. C. (1934). Sources of information on specific subjects. Engineering, 137, 85–86. Reprinted in 1985 in Journal of Information Science, 10(4), 173–180.

  • Costas, R., & Bordons, M. (2007). The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level, Special Issue on h-index. Journal of Informetrics, 1(3), 193–203.

    Article  Google Scholar 

  • García, J. A., Rodriguez-Sánchez, R., Fdez-Valdivia, J., Torres-Salinas, D., & Herrera, F. (2012). Ranking of research output of universities on the basis of the multidimensional prestige of influential fields: Spanish universities as a case of study. Scientometrics, 93(3), 1081–1099.

    Article  Google Scholar 

  • Geraci, M., & Esposti, D. M. (2011). Where do Italian universities stand? An in depth statistical analysis of national and international rankings. Scientometrics, 87(3), 667–681.

    Article  Google Scholar 

  • Glänzel, W. (2001). National characteristics in international scientific co-authorship relations. Scientometrics, 51(1), 69–115.

    Article  Google Scholar 

  • Gupta, B. M. (2011). Indian S&T during 15 years (1996–2010): A quantitative assessment using publication data. DESIDOC Journal of Library and Information Technology, 31(5), 359–370.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. PNAS, 102(46), 16569–16572.

    Article  Google Scholar 

  • Huang, M.-H. (2012). Exploring the h-index at the institutional level: A practical application in world university rankings. Online Information Review, 36(4), 534–547.

    Article  Google Scholar 

  • Huang, M. H., & Chi, P. S. (2010). A comparative analysis of the application of h-index, g-index, and a-index in institutional-level research. Journal of Library and Information Studies, 8(2), 1–10.

    Google Scholar 

  • Huang, M.-H., & Lin, C. S. (2011). Counting methods and university ranking by h-index, ASIST 2011, October 9–13, 2011, New Orleans, LA, USA.

  • Jeremic, V., Bulajic, M., Martic, M., & Radojicic, Z. (2011). A fresh approach to evaluating the academic ranking of world universities. Scientometrics, 87(3), 587–596.

    Article  Google Scholar 

  • Lazaridis, T. (2010). Ranking university departments using the mean h-index. Scientometrics, 82(2), 211–216.

    Article  Google Scholar 

  • Leydesdorff, L., & Opthof, T. (2010). Normalization, CWTS indicators, and the Leiden rankings: Differences in citation behavior at the level of fields. arXiv preprint arXiv:1003.3977.

  • Liu, N. C., & Cheng, Y. (2005). The Academic Ranking of World Universities. Higher Education in Europe, 30(2), 127–136.

    Article  Google Scholar 

  • Liu, N. C., Cheng, Y., & Liu, L. (2005). Academic ranking of world universities using scientometrics: A comment to the Fatal Attraction. Scientometrics, 64(1), 101–109.

    Article  Google Scholar 

  • Liu, N. C., & Liu, L. (2005). University rankings in China. Higher Education in Europe, 30(2), 217–227.

    Article  Google Scholar 

  • Matthews, A. P. (2012). South African universities in world rankings. Scientometrics, 92(3), 675–695.

    Article  MathSciNet  Google Scholar 

  • Molinari, A., & Molinari, J. (2008). Mathematical aspects of a new criterion for ranking scientific institutions based on the h-index. Scientometrics, 75(2), 339–356.

    Article  MathSciNet  Google Scholar 

  • National Institutional Ranking Framework, NIRF. (2015). A methodology for ranking of universities and colleges in India. Ministry of Human Resource Development, MHRD https://www.nirfindia.org/Docs/Ranking%20Framework%20for%20Universities%20and%20Colleges.pdf. Accessed January 2016.

  • Nishy, P., Panwar, Y., Prasad, S., Mandal, G., & Prathap, G. (2012). An impact-citations-exergy (iCX) trajectory analysis of leading research institutions in India. Scientometrics, 91(1), 245–251.

    Article  Google Scholar 

  • Singh, V. K., Uddin, A., & Pinto, D. (2015). Computer science research: The Top 100 institutions in India and in the world. Scientometrics, 104(2), 529–553.

    Article  Google Scholar 

  • Torres Salinas, D., Moreno-Torres, J. G., Delgado-Lopez-Cozar, E., & Herrera, F. (2011). A methodology for institution-field ranking based on a bi-dimensional analysis. Scientometrics, 88(3), 771–786.

    Article  Google Scholar 

  • Uddin, A., Bhoosreddy, J., Tiwari, M., & Singh, V. K. (2016). A sciento-text framework to characterize research strength of institutions at fine-grained thematic area level. Scientometrics, 106(3), 1135–1150.

    Article  Google Scholar 

  • Uddin, A., & Singh, V. K. (2015). A Quantity–Quality composite ranking of Indian Institutions in CS Research. IETE Technical Review, 32(4), 273–283.

    Article  Google Scholar 

  • UGC Note on Vacant Positions. (2014). http://www.ugc.ac.in/pdfnews/0342004_vacant-position-CU-as-on-01-01-2014.pdf. Accessed on June 18, 2014.

  • Van Raan, A. F. J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.

    Article  Google Scholar 

  • Vinkler, P. (2006). Composite scientometric indicators for evaluating publications of research institutes. Scientometrics, 68(3), 629–642.

    Article  Google Scholar 

  • Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E. C. M., Tijssen, R. J. W., Van Eck, N. J., et al. (2012). The Leiden ranking 2011/2012: Data collection, indicators and interpretation. Journal of the American Society of Information Science and Technology, 63(12), 2419–2432.

    Article  Google Scholar 

  • Waltman, L., & Van Eck, N. J. (2012). A new methodology for constructing a publication level classification system of Science. Journal of the American Society of Information Science and Technology, 63(12), 2378–2392.

    Article  Google Scholar 

  • Waltman, L., Van Eck, N. J., Leeuwen, T. N., Visser, M. S., & Van Raan, A. F. J. (2011). Towards a new crown indicator: An empirical analysis. Scientometrics, 87(3), 467–481.

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by research grants from Department of Science and Technology, Government of India (Grant: INT/MEXICO/P-13/2012) and University Grants Commission of India [Grant: F. No. 41-624/2012(SR)].

Author information

Authors and Affiliations

  1. CSIR-NISTADS, K.S. Krishnan Marg, New Delhi, India

    Aparna Basu

  2. Department of Computer Science, South Asian University, New Delhi, India

    Sumit Kumar Banshal & Khushboo Singhal

  3. Department of Computer Science, Banaras Hindu University, Varanasi, 221005, India

    Vivek Kumar Singh

Authors
  1. Aparna Basu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sumit Kumar Banshal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Khushboo Singhal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vivek Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vivek Kumar Singh.

Ethics declarations

Conflict of interest

None.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Basu, A., Banshal, S.K., Singhal, K. et al. Designing a Composite Index for research performance evaluation at the national or regional level: ranking Central Universities in India. Scientometrics 107, 1171–1193 (2016). https://doi.org/10.1007/s11192-016-1935-0

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-016-1935-0

Keywords

Search

Navigation