What does U-multirank tell us about knowledge transfer and research?

Abstract

The economic and social need to spread knowledge between universities and industry has become increasingly evident in recent years. This paper presents a ranking based partly on research and knowledge transfer indicators from U-multirank data but using data-driven weights. The choice of specific weights and the comparison between ranks remain a sensitive topic. A restricted version of the benefit of the doubt method is implemented to build a new university ranking that includes an endogenous weighting scheme. Furthermore, a novel procedure is presented to compare the principal method with U-multirank. At the best of my knowledge, the U-multirank data set has been unapplied to achieve alternative rankings that include research and knowledge transfers dimensions. A significant result arises from the benefit of the doubt: the highest importance weight is assigned to the co-publications with industrial partners and interdisciplinary publication indicators. This paper fills a bit of the existing gap on the role of co-publications with industrial partners in the university efficiency around the world.

This is a preview of subscription content, access via your institution.

Fig. 1

Source: Author. U-Multirank (2018)

Fig. 2

Author. U-Multirank (2018)

Notes

  1. 1.

    https://www.umultirank.org. Performance groups with numbers: A = 1, B = 2, C = 3, D = 4, and E = 5.

  2. 2.

    The logical function is =MATCH($A2;$C$2:$C$819;0). The MATCH function searches for a specified item in a range of cells, and then returns the relative position of that item in the range.For instance, if the range A1:A3 contains the values 5, 25, and 38, then the formula =MATCH(25,A1:A3,0) returns the number 2, because 25 is the second item in the range.

  3. 3.

    As an example for the BODR \(=IF\)(Y($\(C3>0\);$\(C3<=3\));”1”;IF(Y($\(C3>3\);$\(C3<=42\)); ”2”;IF(Y($\(C3>42\);$\(C3<=53\));”3”; IF(Y($\(C3>53\);$\(C3<=54\));”4”; IF(Y($\(C3>54\);$\(C3<=84\));”5”;IF(Y($\(C3>84\);$\(C3<=110\)); “6”;IF(Y($\(C3>110\);$\(C3<=112\)); “7”)))))))

References

  1. Agasisti, T., Barra, C., & Zotti, R. (2019). Research, knowledge transfer and innovation: The effect of Italian universities’ efficiency on local economic development 2006–2012. Journal of Regional Science, 59(5), 819–849. https://doi.org/10.1111/jors.12427.

    Article  Google Scholar 

  2. Agasisti, T., & Pérez-Esparrells, C. (2009). Comparing efficiency in a cross-country perspective: The case of Italian and Spanish state universities. Higher Education, 59(1), 85–103. https://doi.org/10.1007/s10734-009-9235-8.

    Article  Google Scholar 

  3. Akoglu, H. (2018). User’s guide to correlation coefficients. Turkish Journal of Emergency Medicine, 18(3), 91–93. https://doi.org/10.1016/j.tjem.2018.08.001.

    Article  Google Scholar 

  4. Barros, M. V., Ferreira, M. B., do Prado, G.F., Moro-Piekarski, C., & Picinin, C.T. (2020). The interaction between knowledge management and technology transfer: A current literature review between 2013 and 2018. The Journal of Technology Transfer. https://doi.org/10.1007/s10961-020-09782-w.

  5. Bengoa, A., Maseda, A., Iturralde, T., & Aparicio, G. (2020). A bibliometric review of the technology transfer literature. The Journal of Technology Transfer,. https://doi.org/10.1007/s10961-019-09774-5.

    Article  Google Scholar 

  6. Bougnol, M. L., & Dulá, J. H. (2014). Technical pitfalls in university rankings. Higher Education, 69(5), 859–866. https://doi.org/10.1007/s10734-014-9809-y.

    Article  Google Scholar 

  7. Borge, L., & Bräring, S. (2017). Exploring effectiveness of technology transfer in interdisciplinary settings: The case of the bioeconomy. Creativity and Innovation Management, 26(3), 311–322. https://doi.org/10.1111/caim.12222.

    Article  Google Scholar 

  8. Borge, L., & Bräring, S. (2018). What affects technology transfer in emerging knowledge areas? A multi-stakeholder concept mapping study in the bioeconomy. The Journal of Technology Transfer,. https://doi.org/10.1007/s10961-018-9702-4.

    Article  Google Scholar 

  9. Bowman, N. A., & Bastedo, M. N. (2010). Anchoring effects in world university rankings: Exploring biases in reputation scores. Higher Education, 61(4), 431–444. https://doi.org/10.1007/s10734-010-9339-1.

    Article  Google Scholar 

  10. Breschi, S., & Catalini, C. (2010). Tracing the links between science and technology: An exploratory analysis of scientists’ and inventors’ networks. Research Policy, 39(1), 14–26. https://doi.org/10.1016/j.respol.2009.11.004.

    Article  Google Scholar 

  11. Carrión, A., Garía-Gutiérrez, V. R., Bas, M. C., & Carot, J. M. (2012). A new methodology for measuring third mission activities of universities. In INTED2012 proceedings (pp. 1218–1223). Retrieved October 01, 2020, from https://library.iated.org/view/CARRION2012ANE.

  12. Chang Lee, K., Lee, S., & Kang, I. W. (2005). KMPI: Measuring knowledge management performance. Information and Management, 42(3), 469–482. https://doi.org/10.1016/j.im.2004.02.003.

    Article  Google Scholar 

  13. Cherchye, L., Moesen, W., Rogge, N., & Van Puyenbroeck, T. (2007). An introduction to ‘Benefit of the doubt’ composite indicators. Social Indicators Research, 82, 111–145. https://doi.org/10.1007/s11205-006-9029-7.

    Article  Google Scholar 

  14. Choudhry, V., & Ponzio, T. A. (2020). Modernizing federal technology transfer metrics. The Journal of Technology Transfer, 45, 544–559. https://doi.org/10.1007/s10961-018-09713-w.

    Article  Google Scholar 

  15. Collins, F. L., & Park, G. (2016). Ranking and the multiplication of reputation: Reflections from the frontier of globalizing higher education. Higher Education, 72, 115–129. https://doi.org/10.1007/s10734-015-9941-3.

    Article  Google Scholar 

  16. Crescenzi, R., Filippetti, A., & Iammarino, S. (2017). Academic inventors: Collaboration and proximity with industry. The Journal of Technology Transfer, 42, 730–762. https://doi.org/10.1007/s10961-016-9550-z.

    Article  Google Scholar 

  17. David, S. A. (2018). The relationship between the third mission and university ranking: Exploring the outreach of the top ranked universities in BRICS countries. Retrieved October 01, 2020, from https://third-mission.org/files/publications2018/Third_mission_and_university_ranking_Solomon_A._David.pdf.

  18. de Wit-de Vries, E., Dolfsma, W. A., van der Windt, H. J., & Gerkema, M. P. (2019). Knowledge transfer in university–industry research partnerships: A review. The Journal of Technology Transfer, 44, 1236–1255. https://doi.org/10.1007/s10961-018-9660-x.

    Article  Google Scholar 

  19. De Witte, K., & Hudrlikova, L. (2013). What about excellence in teaching? A benevolent ranking of universities. Scientometrics, 96, 337–364. https://doi.org/10.1007/s11192-013-0971-2.

    Article  Google Scholar 

  20. De Witte, K., & Rogge, N. (2010). To publish or not to publish? On the aggregation and drivers of research performance. Scientometrics, 85, 657–680. https://doi.org/10.1007/s11192-010-0286-5.

    Article  Google Scholar 

  21. Etzkowitz, H., & Leydesdorff, L. (1999). The future location of research and technology transfer. Journal of Engineering and Technology Management, 24, 111–123. https://doi.org/10.1023/A:1007807302841.

    Article  Google Scholar 

  22. Finne, H., Day, A., Piccaluga, A., Spithoven, A., Walter, P., & Wellen, D. (2011). A composite indicator for knowledge transfer. In Report from the European Commission’s expert group on knowledge transfer indicators. https://ec.europa.eu/research/innovation-union/pdf/kti-report-final.pdf.

  23. Fisch, C. O., Hassel, T. M., Sandner, P. G., & Block, J. H. (2015). University patenting: A comparison of 300 leading universities worldwide. The Journal of Technology Transfer, 40, 318–345. https://doi.org/10.1007/s10961-014-9355-x.

    Article  Google Scholar 

  24. Fombuena, A. (2019). Evaluación de la transferencia de conocimiento e innovación de las universidades españolas. Revista Española de Documentación Científica, 42(3), e240. https://doi.org/10.3989/redc.2019.3.1596.

    Article  Google Scholar 

  25. Fusco, E., Vidoli, F., & Sahoo, B. K. (2018). Spatial heterogeneity in composite indicator: A methodological proposal. Omega, 77, 1–14. https://doi.org/10.1016/j.omega.2017.04.007.

    Article  Google Scholar 

  26. Govind, M., & Küttim, M. (2016). International knowledge transfer from university to industry: A systematic literature review. Research in Economics and Business: Central and Eastern Europe, 8(2), 5–25.

    Google Scholar 

  27. Gupta, T., & Panda, S. P. (2019). Clustering validation of CLARA and K-means using Silhouette & DUNN measures on Iris dataset. In 2019 International conference on machine learning, big data, cloud and parallel computing (COMITCon). https://doi.org/10.1109/comitcon.2019.8862199.

  28. Hermans, J. (2013). Knowledge transfer in or through clusters: Outline of a situated approach. Advances in Spatial Science,. https://doi.org/10.1007/978-3-642-33194-7_3.

    Article  Google Scholar 

  29. Ho, M. H. C., Liu, J. S., Lu, W. M., & Huang, C. C. (2013). A new perspective to explore the technology transfer efficiencies in US universities. The Journal of Technology Transfer, 39(2), 247–275. https://doi.org/10.1007/s10961-013-9298-7.

    Article  Google Scholar 

  30. Holgersson, M., & Aaboen, L. (2019). A literature review of intellectual property management in technology transfer offices: From appropriation to utilization. Technology in Society, 59, 101132. https://doi.org/10.1016/j.techsoc.2019.04.008.

    Article  Google Scholar 

  31. Huang, M., Yang, H., & Chen, D. (2015). Industry-academia collaboration in fuel cells: A perspective from paper and patent analysis. Scientometrics, 105, 1301–1318. https://doi.org/10.1007/s11192-015-1748-6.

    Article  Google Scholar 

  32. Johnes, J. (2018). University rankings: What do they really show? Scientometrics, 115, 585–606. https://doi.org/10.1007/s11192-018-2666-1.

    Article  Google Scholar 

  33. Landinez, L., Kliewe, T., & Diriba, H. (2019). Entrepreneurial University indicators in global university rankings. In Developing engaged and entrepreneurial universities. Theories, concepts and empirical findings (pp. 57–86). https://doi.org/10.1007/978-981-13-8130-0.

  34. Lee, J., Vance, H., Stensaker, B., & Ghosh, S. (2020). Global rankings at a local cost? The strategic pursuit of status and the third mission. Comparative Education, 56(2), 236–256. https://doi.org/10.1080/03050068.2020.1741195.

    Article  Google Scholar 

  35. Marhl, M., & Pausits, A. (2011). Third mission indicators for new ranking methodologies. Evaluation in Higher Education, 5(1), 43–64.

    Google Scholar 

  36. Moed, H. F. (2017). A critical comparative analysis of five world university rankings. Scientometrics, 110, 967–990. https://doi.org/10.1007/s11192-016-2212-y.

    Article  Google Scholar 

  37. Montesinos, P., Carot, J. M., Martinez, J. M., & Mora, F. (2008). Third mission ranking for world class universities: Beyond teaching and research. Higher Education in Europe, 33(2–3), 259–271. https://doi.org/10.1080/03797720802254072.

    Article  Google Scholar 

  38. Mukherji, N., & Silberman, J. (2019). Knowledge flows between universities and industry: The impact of distance, technological compatibility, and the ability to diffuse knowledge. The Journal of Technology Transfer,. https://doi.org/10.1007/s10961-019-09770-9.

    Article  Google Scholar 

  39. OECD. (2004). The OECD-JRC handbook on practices for developing composite indicators. In The OECD committee on statistics, Conference, 7–8 June 2004. OECD, Paris. https://stats.oecd.org/glossary/detail.asp?ID=6278.

  40. Olcay, G. A., & Bulu, M. (2017). Is measuring the knowledge creation of universities possible? A review of university rankings. Technological Forecasting and Social Change, 123, 153–160. https://doi.org/10.1016/j.techfore.2016.03.029.

    Article  Google Scholar 

  41. Perez-Esparrells, C., & Orduna-Malea, E. (2018). Do the technical universities exhibit distinct behaviour in global university rankings? A Times Higher Education (THE) case study. Journal of Engineering and Technology Management, 48, 97–108. https://doi.org/10.1016/j.jengtecman.2018.04.007.

    Article  Google Scholar 

  42. Ramos-Vielba, I., & Fernández-Esquinas, M. (2011). Beneath the tip of the iceberg: Exploring the multiple forms of university-industry linkages. Higher Education, 64(2), 237–265. https://doi.org/10.1007/s10734-011-9491-2.

    Article  Google Scholar 

  43. Rossi, F., & Rosli, A. (2014). Indicators of university-industry knowledge transfer performance and their implications for universities: Evidence from the United Kingdom. Studies in Higher Education, 40(10), 1970–1991. https://doi.org/10.1080/03075079.2014.914914.

    Article  Google Scholar 

  44. Safón, V. (2019). Inter-ranking reputational effects: An analysis of the Academic Ranking of World Universities (ARWU) and the Times Higher Education World University Rankings (THE) reputational relationship. Scientometrics, 121, 897–915. https://doi.org/10.1007/s11192-019-03214-9.

    Article  Google Scholar 

  45. Shehatta, I., & Mahmood, K. (2016). Correlation among top 100 universities in the major six global rankings: Policy implications. Scientometrics, 109, 1231–1254. https://doi.org/10.1007/s11192-016-2065-4.

    Article  Google Scholar 

  46. Schmoch, U. (1993). Tracing the knowledge transfer from science to technology as reflected in patent indicators. Scientometrics, 26, 193–211. https://doi.org/10.1007/BF02016800.

    Article  Google Scholar 

  47. Sun, Y., Zhang, C., & Kok, R. A. W. (2020). The role of research outcome quality in the relationship between university research collaboration and technology transfer: Empirical results from China. Scientometrics, 122, 1003–1026. https://doi.org/10.1007/s11192-019-03330-6.

    Article  Google Scholar 

  48. Szuwarzyński, A. (2019). Benefit of the doubt approach to assessing the research performance of Australian universities. Higher Education Quarterly, 73, 235–250. https://doi.org/10.1111/hequ.12184.

    Article  Google Scholar 

  49. U-Mulirank. (2018). Indicator book. Press Media. Retrieved September 09, 2019, from https://www.umultirank.org/export/sites/default/press-media/documents/Indicator-Book-2018.pdf.

  50. Uslu, B. (2020). A path for ranking success: What does the expanded indicator-set of international university rankings suggest? Higher Education,. https://doi.org/10.1007/s10734-020-00527-0.

    Article  Google Scholar 

  51. Van Puyenbroeck, T. (2018). On the output orientation of the benefit-of-the-doubt-model. Social Indicators Research, 139, 415–431. https://doi.org/10.1007/s11205-017-1734-x.

    Article  Google Scholar 

  52. van Vught, F., & Ziegele, F. (Eds.). (2011). Design and testing the feasibility of a multidimensional global university ranking. In Final Report: Consortium for Higher Education and Research Performance Assessment, CHERPA-Network. Retrieved October 01, 2020, from https://ris.utwente.nl/ws/portalfiles/portal/5145063/Vught11umulti.pdf.

  53. Vidoli, F., & Fusco, E. (2018). Compind: Composite indicators functions based on frontiers in R. R package version 2.0. Retrieved January 9, 2020, from https://cran.r-project.org/web/packages/Compind/vignettes/Compind_vignette.pdf.

  54. Vidoli, F., & Mazziotta, C. (2013). Robust weighted composite indicators by means of frontier methods with an application to european infrastructure endowment. Statistica Applicata-Italian Journal of Applied Statistics, 23(2), 259–282.

    Google Scholar 

  55. Wang, X., Zhao, Y., Liu, R., & Zhang, J. (2013). Knowledge-transfer analysis based on co-citation clustering. Scientometrics, 97, 859–869. https://doi.org/10.1007/s11192-013-1077-6.

    Article  Google Scholar 

  56. Waaijer, C. J. F. (2018). Why the public nature of university rankings torpedoes their value for benchmarking: An institutional researcher’s opinion. In Proceedings of the 23rd international conference on science and technology indicators (STI 2018). Retrieved September 09, 2019, from http://sti2018.cwts.nl/proceedings.

  57. Wiel, van de, M. A., & Di Bucchianico, A. (1998). Fast computation of the exact null distribution of Spearman’s rho and Page’s L statistic for samples with and without ties. (Memorandum COSOR; Vol. 9817). Eindhoven: Technische Universiteit Eindhoven.

Download references

Acknowledgements

Thanks to U-multirank consortium for generously providing the official database for this academic paper. I want to thank the anonymous referees for their valuable comments which helped to improve the manuscript.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Juan Antonio Dip.

Ethics declarations

Conflict of interest

The author declares that he has no conflict of interest.

Appendices

Appendix 1: Figures

See Figs. 3, 4, 5, 6, 7, 8, 9, 10 and 11.

Fig. 3
figure3

Author. U-Multirank (2018)

Universities in Europe.

Fig. 4
figure4

Universities in USA

Fig. 5
figure5

Universities in Asia

Fig. 6
figure6

Author. U-Multirank (2018)

The Silhouette method. See Gupta and Panda (2019).

Fig. 7
figure7

Author. U-Multirank (2018)

CLARA: Kendall’s correlation coefficient, density and Scatter plot.

Fig. 8
figure8

Author. U-Multirank (2018)

Importance weights mean. Whole sample. BODR.

Fig. 9
figure9

Author.U-Mutirank 2018

U-Multirank. Construction.

Fig. 10
figure10

Author.U-Mutirank 2018

BODR. BODRC. Construction.

Fig. 11
figure11

Author.U-Mutirank 2018.

Spearman’s orden rank correlation. BODR with different \(\alpha \) values. All significant at 1% (p < 0:01)

Appendix 2: Tables

See Tables 9 and 10.

Table 9 BODR-score—descriptive statistics
Table 10 BODRC, BODRF and U-multirank

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dip, J.A. What does U-multirank tell us about knowledge transfer and research?. Scientometrics (2021). https://doi.org/10.1007/s11192-020-03838-2

Download citation

Keywords

  • Knowledge transfer
  • U-multirank
  • Benefit of the doubt
  • University rankings

JEL Classification

  • O30
  • I23
  • C14
  • L20