Skip to main content

Advertisement

SpringerLink
Log in

Ranking of rankings: benchmarking twenty-five higher education ranking systems in Europe

  • Published:
Higher Education Aims and scope Submit manuscript

Abstract

The purpose of this study is to evaluate the ranking practices of 25 European higher education ranking systems (HERSs). Ranking practices were assessed with 14 quantitative measures derived from the Berlin Principles on Ranking of Higher Education Institutions (BPs). HERSs were then ranked according to their degree of congruence with the BPs. Additionally, the three domains of methodology, transparency, and consumer-friendliness were proposed to underlie the BPs, and the measures were also applied for assessing HERSs regarding these domains. Results indicate that the cooperating HERSs by CHE (Germany), AQA (Austria) and swissUp (Switzerland) exhibit the highest congruence with the BPs. However, no European HERS demonstrates good overall congruence with the BPs, mainly due to methodological shortcomings. Results further show that HERSs compiled and published by non-profit research entities seem to perform better than the majority of HERSs published by for-profit news outlets. International comparisons indicate that HERSs published in German-speaking countries and the Netherlands tend to exhibit a higher congruence with the BPs. Overall, this study hopes to stimulate the improvement of ranking practices through benchmarking with existing exemplary models. The quantitative assessment tool further promises to be useful in explaining relative stability or change of higher education institutions in HERSs, as well as in helping to explain resource allocation pressures within higher education institutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alexander von Humboldt-Stiftung. (2005). Das Humboldt-Ranking. http://www.humboldt-foundation.de/de/aktuelles/presse/doc/2006ranking72.pdf. Accessed December 29 2006.

  • Altbach, P. G. (2003). The costs and benefits of world class universities. International Higher Education, 33. http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/News33/text003.htm. Accessed September 27 2008.

  • Altbach, P. G. (2006). The dilemmas of ranking. International Higher Education, 42, 2–4. http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/Number42/p2_Altbach.htm. Accessed September 27 2008.

  • ARRA. (2005). Assessment of public universities and their faculties. http://www.arra.sk/oldweb/ARRA-Report2005-EN.pdf. Accessed December 13 2006.

  • Bellon, B. (2007). Evaluation practices and methodologies: Lessons for university ranking. In J. Sadlak & N. C. Liu (Eds.), The world-class university and ranking: Aiming beyond status (pp. 123–145). Bucharest, Rumania: UNESCO-CEPES.

    Google Scholar 

  • Berghoff, S., Federkeil, G., Giebisch, P., Hachmeister, C.-D., Hennings, M., & Müller-Böling, D. (2007). CHE—HochschulRanking: Vorgehensweise und Indikatoren—2007. Guetersloh, Germany: Centrum für Hochschulentwicklung.

    Google Scholar 

  • Birnbaum, R. (2006). No world-class university left behind. International Higher Education, 47. http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/Number47/p7_Birnbaum.htm. Accessed September 27 2008.

  • Bowden, R. (2000). Fantasy higher education: University and college league tables. Quality in Higher Education, 6(1), 47–60.

    Article  Google Scholar 

  • Breimer, D. (2007). Rankings: How should universities respond? Paper presented at the 2nd Leiden University International Symposium on Ranking, Leiden, the Netherlands.

  • Brooks, R. L. (2005). Measuring university quality. The Review of Higher Education, 29(1), 1–21.

    Article  Google Scholar 

  • CEST. (2004). CEST scientometrics research portfolios: Universities and colleges participating in the champions league—Diagrams and profiles, 1998–2002. Bern, Switzerland: Center for Science and Technology Studies of the Swiss Federal Government.

    Google Scholar 

  • Cheng, Y., & Liu, N. C. (2008). Examining major rankings according to the Berlin Principles. Higher Education in Europe, 33(2/3), 201–208.

    Article  Google Scholar 

  • Clarke, M. (2002). Some guidelines for academic quality rankings. Higher Education in Europe, 27(4), 443–459.

    Article  Google Scholar 

  • de Miguel, J., Cais, J., & Vaquera, E. (2001). Excelencia: Calidad de las universidades espanolas. Madrid, Spain: Centro de Investigaciones Sociologicas.

    Google Scholar 

  • de Miguel, J., Vaquera, E., & Sanchez, J. (2005). Spanish universities and the ranking 2005 initiative. Higher Education in Europe, 30(2), 199–215.

    Article  Google Scholar 

  • de Wit, K. (2003). The consequences of European integration for higher education. Higher Education Policy, 16(2), 161–178.

    Article  Google Scholar 

  • Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: A cross-national analysis of university ranking systems. Higher Education, 49(4), 495–533.

    Article  Google Scholar 

  • Elsevier. (2006, October). Studeren. Elsevier Thema.

  • Enserink, M. (2007). Who ranks the university rankers? Science, 317(5841), 1026–1028.

    Article  Google Scholar 

  • Federkeil, G. (2008). Rankings and quality assurance in higher education. Higher Education in Europe, 33(2/3), 219–231.

    Article  Google Scholar 

  • Gaertner, R., Birke, B., & Kohler, A. (2007). Hochschulranking 2007—Ingenieurwissenschaften, Geisteswissenschaften, Erziehungswissenschaften und Psychologie: Bericht zu Ergebnissen der österreichischen Universitäten, Verfahren und Methodik. Vienna, Austria: Austrian Agency for Quality Assurance.

    Google Scholar 

  • Hazelkorn, E. (2007). The impact of league tables and ranking systems on higher education decision making. Higher Education Management and Policy, 29(2), 87–110.

    Google Scholar 

  • Hendel, D. D., & Stolz, I. (2008). A comparative analysis of higher education ranking systems in Europe. Tertiary Education and Management, 14(3), 173–189.

    Article  Google Scholar 

  • IHEP. (2007). College and university ranking systems: Global perspectives and American challenges. Washington, DC: Institute for Higher Education Policy.

    Google Scholar 

  • IREG. (2006). Berlin Principles on Ranking of Higher Education Institutions. Berlin, Germany: International Ranking Expert Group.

    Google Scholar 

  • Karlsson, S., & Wadskog, D. (2006). Hur mycket citeras Svenska publikationer? Bibliometrisk översikt över Sveriges vetenskapliga publicering mellan 1982 och 2004. Stockholm, Sweden: Vetenskapsrådet.

    Google Scholar 

  • Lebherz, C., Mohr, C., Henning, M., & Sedlmeier, P. (2005). Wie brauchbar sind Hochschul-Rankings? Eine empirische Analyse. In U. Teichler & R. Tippelt (Eds.), Hochschullandschaft im Wandel (pp. 188–208). Germany; Switzerland, Beltz: Weinheim; Basel.

    Google Scholar 

  • Longden, B., & Yorke, M. (2006). Institutional rankings, marketing, and the needs of intending students. Paper presented at the Annual EAIR Forum, Rome, August 30 to September 1.

  • Macleod, D. (2007). The guardian university guide 2008. London, UK: Guardian Newspapers.

  • Marginson, S. (2007). Global university comparison: the second stage. Griffith University/IRU symposium on international trends in university rankings and classifications. http://www.griffith.edu.au/conference/university-rankings/pdf/simon-marginson-paper.pdf. Accessed May 15 2008.

  • Marginson, S., & van der Wende, M. (2007). To rank or to be ranked: The impact of global rankings in higher education. Journal of Studies in International Education, 11(3/4), 306–329.

    Article  Google Scholar 

  • McCormick, A. C. (2008). The complex interplay between classification and ranking of colleges and universities: Should the Berlin Principles apply equally to classification? Higher Education in Europe, 33(2/3), 209–218.

    Article  Google Scholar 

  • McDonough, P. M., Lising, A., Walpole, A. M., & Perez, L. X. (1998). College rankings: Democratized college knowledge for whom? Research in Higher Education, 39(5), 513–537.

    Article  Google Scholar 

  • Merisotis, J. P. (2002). Summary report of the invitational roundtable on statistical indicators for the quality assessment of higher/tertiary education institutions: Ranking and league table methodologies. Higher Education in Europe, 27(4), 476–480.

    Google Scholar 

  • Merisotis, J., & Sadlak, J. (2005). Higher education rankings: Evolution, acceptance, and dialogue. Higher Education in Europe, 30(2), 97–101.

    Article  Google Scholar 

  • Müller-Böling, D. (2001a). Hochschulranking—Aussagefähigkeit, Methoden, Probleme: Eine Einführung. In D. Müller-Böling, S. Hornbostel, & S. Berghoff (Eds.), Hochschulranking: Aussagefähigkeit, Methoden, Probleme; wissenschaftliche Tagung Berlin 25/26. September 2000 (pp. 5–6). Guetersloh, Germany: Bertelsmann-Stiftung.

    Google Scholar 

  • Müller-Böling, D. (2001b). Qualität an deutschen Hochschulen: Der Beitrag des Hochschulrankings. In Schweizerische Akademie der Geistes- und Sozialwissenschaften (Ed.), Welche Qualität in den Sozialwissenschaften? Ansätze, Erfahrungen, Perspektiven: Eine Tagung des Wissenschaftspolitischen Rates für Sozialwissenschaften (CPS/WRS), des Schwerpunktprogramms ‘Zukunft Schweiz’, der Schweizerischen Evaluationsgesellschaft SEVAL und der Schweizerischen Akademie der Geistes- und Sozialwissenschaften, Gurten/Bern, 16./17. Mai 2001 (pp. 125–142). Bern, Switzerland: Schweizerische Akademie der Geistes- und Sozialwissenschaften.

  • O’Leary, J., Hindmarsh, A., & Kingston, B. (2006a). The Times Good University Guide 2007. London, UK: Times Books.

    Google Scholar 

  • O’Leary, J., Quacquarelli, N., & Ince, M. (2006b). Guide to world’s top universities: Exclusively featuring the complete THES/QS world university rankings. London, UK: QS Quacquarelli Symonds, Blackwell Publishing.

    Google Scholar 

  • PRIME. (2006). Observatory of the European university: Methodological guide. Lugano, Switzerland: PRIME Network of Excellence.

    Google Scholar 

  • Proulx, R. (2006). World university rankings: The need for a new paradigm. Paper presented at the 28th annual EAIR Forum, Rome, August 30–September 1.

  • Sadlak, J., Merisotis, J., & Liu, N. C. (2008). University rankings: Seeking prestige, raising visibility and embedding quality—The editor’s view. Higher Education in Europe, 33(2/3), 195–199.

    Article  Google Scholar 

  • Salmi, J., & Saroyan, A. (2007). League tables as policy instruments: Uses and misuses. Higher Education Management & Policy, 19(2), 31–68.

    Google Scholar 

  • Steenkamp, F., de Looper, H., & Bliekendaal, M. (2006). Keuzegids Hoger Onderwijs 2006–2007: Verzamelgids. Leiden, The Netherlands: Keuzegids Hoger Onderwijs.

    Google Scholar 

  • Studentenspiegel. (2006). Methodik des Studentenspiegels. http://www.studentenspiegel1.de/methodik.pdf. Accessed February 14 2007.

  • swissUp. (2006a). Definitionen der gerankten Indikatoren: Ranking 2005 + 2006. http://www.rankingswissup.ch/pdf/Def_Indikatoren_Ranking06+05_D.pdf. Accessed January 29 2007.

  • swissUp. (2006b). swissUp Ranking 2006, Bemerkungen zur Methodologie. http://www.rankingswissup.ch/pdf/methodo06_D_DEF.pdf. Accessed January 29 2007.

  • Sydsvenska Industri-och Handelskammaren. (2007). Högskolerankingen. http://www.handelskammaren.com/backoffice/bakpress/ladda/H%C2%A8gskolerankingen%202007%20slutlig.pdf. Accessed May 14 2008.

  • Taylor, P., & Braddock, R. (2007). International university ranking systems and the idea of university excellence. Journal of Higher Education Policy and Management, 29(3), 245–260.

    Article  Google Scholar 

  • Terenzini, P. T., & Pascarella, B. T. (1994). Living with myths: Undergraduate education and what we can do about it. Change, 26(1), 28–32.

    Google Scholar 

  • UNESCO-CEPES. (2006). Second meeting of the International Rankings Expert Group: List of participants. http://www.cepes.ro/hed/meetings/berlin06/list.htm. Accessed October 3 2008.

  • Usher, A., & Savino, M. (2006). A world of difference: A global survey of university league tables. Toronto, Canada: Educational Policy Institute.

    Google Scholar 

  • Usher, A., & Savino, M. (2007). A global survey of university ranking and league tables. Higher Education in Europe, 32(1), 5–15.

    Article  Google Scholar 

  • van Dyke, N. (2005). Twenty years of university reports cards. Higher Education in Europe, 30(2), 103–124.

    Article  Google Scholar 

  • Yorke, M. (1997). A good league table guide? Quality Assurance in Education, 5(2), 61–72.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Organizational Leadership, Policy, and Development, University of Minnesota, 330 Wulling Hall, 86 Pleasant Street S.E., Minneapolis, MN, 55455-0221, USA

    Ingo Stolz, Darwin D. Hendel & Aaron S. Horn

Authors
  1. Ingo Stolz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Darwin D. Hendel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aaron S. Horn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Darwin D. Hendel.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Stolz, I., Hendel, D.D. & Horn, A.S. Ranking of rankings: benchmarking twenty-five higher education ranking systems in Europe. High Educ 60, 507–528 (2010). https://doi.org/10.1007/s10734-010-9312-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10734-010-9312-z

Keywords

Search

Navigation