Introduction

In modern, knowledge-based societies, higher education institutions (HEIs) are becoming increasingly embedded in their local, regional, national, and international environments (Degl’Innocenti et al. 2019; Kenney and Goe 2004; Riviezzo et al. 2020; de Wit 2010) by forming knowledge-related collaborations with external actors such as businesses, governments, other universities, and students as consumers (Sánchez-Barrioluengo 2014; Perkmann et al. 2013; Scandura 2016). University collaborations can be considered both catalysts and engines of their operation supported by policies and laws, programs for enhancing collaboration and mobility, and financial incentives and tax regulations (Geisler 1995; Thune 2007). Donors, both private and public, governments and research agencies seek ways to facilitate and promote university-society synergies based on the assumption that collaborating in a networked environment is advantageous for achieving complex policy goals (OECD 1998, 2007; Thomson et al. 2009). Specifically, the European Economic Community (EEC) extensively fostered university collaboration by launching the Framework Programmes for research, development and demonstration in 1984 (European Economic Community 1983) and the Erasmus mobility program in 1987 (European Economic Community 1987). The European Commission’s (EC) 2011 Modernisation Agenda identified the aim of enhancing the internationalization and openness of higher education systems, strengthening collaboration between member states and between universities and firms, and fostering international networks for excellence (European Comission 2011). The EC allocated 3.1 billion EUR (0.36%) of its budget to support the Erasmus (2008–2014) program and 45.8 billion EUR (5.3%) to support the FP7 (2007–2013) program (EU 2013).

Given the significant role of universities in economic growth and social progress, as well as the increasing demand for greater transparency regarding the use of public money (Dolowitz and Marsh 2000; Molas-Gallart and Castro-Martínez 2007; Janger et al. 2017), valid assessment of the universities’ performance becomes crucial for various stakeholders, including government, industry, funding agencies, and society at large (Bölling and Eriksson 2016; Olcay and Bulu 2017; Ramos-Vielba et al. 2010).

In this respect, university rankings serve as useful instruments for public accountability, supplying information to both consumers and policy makers on measurable differences in university performance (Adler and Harzing 2009; Dill and Soo 2005; Teichler 2011). A number of global rankings, such as the Times Higher Education (THE) World University Ranking, the Academic Ranking of World Universities (ARWU), the U-Multirank, and the CWTS Leiden Ranking, measure and compare university performance each year. These global rankings rely on internationally accessible indicators and bibliometric/webometric data (Çakır et al. 2015; Rauhvargers 2011). In addition, several countries have developed their own national ranking systems, often using sets of more comprehensive, but less accessible/reproducible, indicators (Olcay and Bulu 2017; Saisana et al. 2011).

In our view, existing ranking methods raise several problems. First, indicator-based ranking methods do not consider the interactions, such as collaboration and mobility, between HEIs. From a network point of view, these methods rank nodes (i.e., HEIs) by their properties without considering the links (i.e., collaboration and mobility) between them. Second, the size effect is a substantial shortcoming of indicator-based ranking methods (Van Raan 2005; Szücs 2018). Most rankings, except for ARWU and CWTS, do not consider size, by which many indicators’ measurements would naturally increase with the size of the HEIs, indicating that the larger a HEI is, the higher the ranking. This fact resonates with the current debates on the Matthew effect in Science (see, for instance, Azoulay et al. 2014; Ayoubi et al. 2019; Bol et al. 2018). Third, except for few rankings, such as CWTS, several indicators are not transparent (e.g., based on surveys), or their reliability is questionable (e.g., based on self-declaration); therefore, it is difficult to reproduce the ranking orders of the HEIs. Finally, due to different HEI financial and legal environments, several indicators cannot be measured or are not relevant for smaller HEIs (Kosztyán et al. 2019). Therefore, these rankings have bias and cannot usually ensure fair results.

The primary purpose of this paper is to complete the existing rankings by adding a new perspective related to HEIs’ core missions, such as embeddedness in collaboration networks, which no current ranking method addresses. This paper proposes a multilayer network-based method to measure and compare the embeddedness of HEIs in mobility and collaboration networks. The innovativeness of this approach is that instead of individual properties of HEIs, the multiple connections between academic and non-academic actors are analyzed and measured by multilayer network analysis. Second, we aim to decrease the size effect of HEIs on their ranking positions by focusing only on their network embeddedness (measured by multilayer eigenvector centrality) instead of on different indicators. Third, we aim to provide feedback for policy makers related to the synergy between EU-funded programs intended for HEIs by analyzing the correlations between the mobility and research collaboration networks’ properties. Finally, our objective is to transparently use methods and datasets that are freely available, and to ensure that all of our calculations can be reproduced.

The article is organized as follows. "Data sources and methods employed" section describes the applied data and the proposed multilayer model, followed by the section on results ("Result"). Finally, in "Summary and conclusion" section, the paper concludes by summarizing the main findings and identifying further research opportunities.

Data sources and methods employed

The aim of merging different data sources was twofold. First, for inclusion in the database, the studied universities were required to have noteworthy activities in the first two missions (teaching and research). Therefore, inclusion in this study was limited to those European universities represented in the Scimago rankingFootnote 1 and the Leiden ranking,Footnote 2 which have reported institutional data for their teaching activities in the ETER database.Footnote 3 The second and most important aspect was to collect as much information about the HEIs as possible from freely available data sources to maintain transparency. Therefore, multiple data sources were combined. The merged data focused on HEIs. ETER, Erasmus and the GRIDFootnote 4 databases have already been connected and geo-coded by Gadár et al. (2020). We extended this database and linked FP7 projects from the CORDIS database, Leiden’s indicators from the CWTS databaseFootnote 5 and the Scimago ranking of HEIs.Footnote 6 Following entity reasoning, in which the institutions are identified, Erasmus student and teacher mobility networks and FP7 research projects represent one layer each for each studied year in the multilayer structure (see Fig. 1). Further properties, such as collaboration and publication impact (from Leiden’s ranking system), organizational data (from the ETER database), and the number of foreign students, are also assigned to the HEIs. The data sources mentioned above contain data related to different universities and cover different time periods. We extracted the data for the 307 European HEIs identified in all of these databases for 2011–2013—the only period for which all data were available.

Mobility networks

To build up the networks related to Erasmus mobility of students and teachers/researchers, we used the EU Open Data Portal,Footnote 7 which includes data on 1M student and 180k teacher travels among 5267 HEIs in eight subject areas between 2008 and 2014.

The network nodes are the HEIs, and two types of edges are included: student mobility and teacher mobility. The directed edge weights are the number of travels between institutions. In this study, only short-term (so-called credit mobility) student and teacher mobility (henceforth, Erasmus database) were analyzed; however, other staff travel and contract networks could be analyzed in future research.

Research collaboration network

The research collaboration databaseFootnote 8 of projects funded under the 7th Framework Programme (furthermore CORDIS-FP7 database) consists of thirty thousand organizations from EU and non-EU countries and more than twenty thousand projects started between 2007 and 2014. To compare mobility and research collaboration networks, only HEIs from the EU-28 countries are included in our study. However, node (i.e., institutional) properties, such as the number of connections and their partners, are retained. Therefore, we obtain a so-called multilayer group ego network, which contains the studied 307 HEIs and their partners in all layers. The edge weights are the number of earned projects.

Other databases

The European Tertiary Education Register (henceforth the ETER database)Footnote 9 is used to unify institution names and to compare the proportions of foreign teachers and foreign students out of the number of Erasmus travels. The citation and collaboration data are obtained from Leiden’s ranking database (henceforth the CWTS-LEIDEN database).Footnote 10 In this study, we focused only on the period of 2011–2013 because all data were available only for these three years.

Note that all 307 of the studied HEIs are included in the CWTS Ranking System, are involved in both in the FP7 and the Erasmus programs, and have noteworthy social impact because all of them are in the first tercile of the Scimago Social Impact Rankings of HEIs (2018), which was the earliest ranking related to the third mission of the HEIs.

Data representation

The proposed multilayer network structure consists of 3 × 3 non-interconnected layers. Layers 1–3 and layers 4–6 consist of organizations in Erasmus student and teacher mobility networks, while layers 7–9 consist of the organizations in the FP7 research collaboration network (see Fig. 1). This proposed structure allows researchers to compare the embeddedness of the HEIs in different networks. Comparing the embeddedness of HEIs in different networks indicates participation interdependencies in parallel EU-funded programs.

Fig. 1
figure 1

The structure of the multilayer network

Full size image

The database of the analyzed multilayer networks is separated into three parts.

  1. 1.

    The set of nodes consists of 307 HEIs with all of the properties from the ETER, CWTS-LEIDEN, CORDIS-FP7 and Erasmus databases between 2011 and 2013.

  2. 2.

    The set of edges includes the pairs of nodes and the weights of occurrences. In mobility networks, directed edges specify the direction of travel between the sending and host HEIs, while in research collaboration networks, the edges represent common projects and are directed from the coordinator to the partner HEI.

  3. 3.

    The set of (non-interconnected) layers consists of the Erasmus mobility and FP7 research collaboration networks.

Future research could attach additional layers, such as collaborations on publications and patents.

Methods employed

The merging of multiple databases, such as the ETER, Erasmus, CORDIS, and CWTS-LEIDEN databases, allows researchers to compare and analyze HEIs and their connections, not only with standard statistical methods but also with multilayer analysis techniques, such as multilayer eigenvector centrality.

To analyze synergies between collaboration and mobility and also between publication and collaboration indicators, correlation analysis is applied to explore the relations among the publication, mobility and collaboration activities of European HEIs. In addition, multilayer centrality detects HEIs, which are embedded in mobility and collaboration networks, and offers size invariant indicators for HEI performance.

Formal description of the applied multilayer analysis

A multilayer network is a pair \(\mathcal {M}=(\mathcal {G},\mathcal {C})\) where \(\mathcal {G}=\{G_{\alpha }=(V_{\alpha },E_{\alpha }),\alpha \in \{1,\ldots ,m\}\}\) is a family of (directed or undirected, weighted or unweighted) graphs (called layers of \(\mathcal {M}\)), where \(V_{\alpha }\) is the set of vertices (set of nodes) and \(E_{\alpha }\) \(\subseteq V_{\alpha }\times V_{\alpha }\) is the set of edges (links) of graph \(G_{\alpha }\) in layer \(\alpha \) and

$$\begin{aligned} \mathcal {C}=\{E_{\alpha ,\beta }\subseteq V_{\alpha } \times V_{\beta },\alpha ,\beta \in \{1,\ldots ,m\}, \alpha \ne \beta \} \end{aligned}$$
(1)

is the set of interconnections between nodes of different layers \(G_{\alpha }, G_{\beta }\in \mathcal {M}\) with \(\alpha \ne \beta \).

In this study, the set of interconnections is not specified; therefore, it is assumed that \(\mathcal {C}=\emptyset \). Eigenvector centrality is one of the most common methods used to detect the embeddedness of a node in a graph. The simplest way to calculate eigenvector-like centralities in multilayer networks is to consider the eigenvector centrality \(c_{\alpha }=\left( c_1^{[\alpha ]},\ldots ,c_n^{[\alpha ]}\right) \) in each layer \(1\le \alpha \le m\) separately, where n is the number of nodes. For a given graph \(G_{\alpha }:=(V_{\alpha },E_{\alpha })\) with \(|V_{\alpha }|\) vertices, let \(A_{\alpha } = (a^{[\alpha ]}_{k^{[\alpha ]},l^{[\alpha ]}})\) be the adjacency matrix. The eigenvector centrality, \(c_k^{[\alpha ]}\), and the score of vertex \(k^{[\alpha ]}\) can be defined as:

$$\begin{aligned} c_{k}^{[\alpha ]}={\frac{1}{\lambda ^{[\alpha ]} }}\sum _{l^{[\alpha ]}\in N(k^{[\alpha ]})}c_{l^{[\alpha ]}}^{[\alpha ]}={\frac{1}{\lambda ^{[\alpha ]} }}\sum _{l^{[\alpha ]}\in G_{\alpha }}a^{[\alpha ]}_{k^{[\alpha ]},l^{[\alpha ]}}c_{l^{[\alpha ]}}^{[\alpha ]} \end{aligned}$$
(2)

where \(N(k^{[\alpha ]})\) is a set of the neighbors of \(k^{[\alpha ]}\) in layer \(\alpha \), and \(\lambda ^{[\alpha ]}\) is a constant.

The centrality measure has several important properties, including that the score value is independent from the node properties, and it depends only on the network’s structure. The eigenvector centrality has additional important properties, e.g., anonymity,Footnote 11 symmetry,Footnote 12 positive homogeneity,Footnote 13 and robustness.Footnote 14

A numeric centrality measure of each node can be obtained using an aggregation measure (f), such as the sum or the maximum. We used Bonacich -Kleinberg ’s aggregation function because with its additional important properties, such as natural consistency, group scores are obtained from individual scores by summing the scores (Kitti 2016). Therefore, multilayer centrality can be calculated in an aggregated graph \(G=(V,E)\), where \(E=\bigcup _{\alpha \in \{1,\ldots ,m\}}E_{\alpha }\) and \(V=f(V_{\alpha })\). Because of the properties of eigenvector centrality, in cases of non-interconnected, multilayer networks, varying network sizes in the layers does not affect or bias the scoring.

Results

The results show how to analyze the embeddedness of universities in a multilayer network and how we can rank these universities by their embeddedness.

Descriptive statistics of the explored multilayer network

All 307 European HEIs are at the intersection of the layers and other applied databases, and all partners in the research collaboration and mobility networks, as nodes, and all the connections between partners, as arcs, are saved. Therefore, 162–320 more institutions and organizations are included in the given layers (see Table 1). The number of edges in Table 1 represents projects in research collaboration networks (FP7 2011-FP7 2013) and the travels of teachers (ET 2011-ET 2013) and students (ES 2011-ES 2013) between HEIs via the Erasmus program. Because of the significantly larger number of edges, the mobility network is much denser than the research collaboration network. However, both are very fragmented, and even the student mobility layers (ES 2011-ES 2013) contain 136–139 unconnected components (weak components). The maximum diameter of these separated components is 7. The mean path length for the student mobility network is 2.3, but the mean for the collaboration network is 3.2.

Table 1 Descriptive statistics of multilayer group ego networks
Full size table

Rank correlation between node properties

The intersection of data sources consists of 307 HEIs. The number of FP7 research projects is strongly correlated with Leiden’s publication and collaboration impacts. Embeddedness (measured by eigenvector centralities) is moderately and positively correlated with Leiden’s indicators (see Table 2), while a weak connection exists between the embeddedness of mobility and collaboration (see Fig. 2 and Table 2).

Fig. 2
figure 2

Correlation between centralities

Full size image

Instead of using absolute values, such as the number of collaborations and the number of mobility exchanges, we used embeddedness, which reduces the size effect. We performed (k-means) clustering of HEIs with respect to mobility and the collaboration embeddedness with eigenvector centrality. The clustering result calculated on centralities shows that 3 clusters can be identified (see Fig. 2). It is immediately clear that the “lower left, third quadrant”, where both the rate of visits and projects are low, is well populated, while the “upper right, first quadrant” is almost empty. The second and fourth quadrants are moderately populated, showing that, if there is any correlation between the indicators, it results only from the elements in the third quadrant. Despite HEIs included in our study are represented in the Leiden ranking system, which means that they have a considerable number of publications with collaborations, the degree of simultaneous participation in the Erasmus and FP7 programs is rather low (Cluster 3).

Several universities concentrate on participating in FP7 projects (Cluster 2), but other universities are more attractive for students (Cluster 1). Interestingly, very few HEIs can be found for which both the number of incoming Erasmus students and the number of earned projects are substantial. Since the Erasmus Programme is not the only source of internationalization, the results of our analysis do not mean that internationalization is not vital for HEIs in Cluster 2, which are involved in many FP7 projects but receive foreign students and teachers through other means. The FP7 also offers longer-term mobility programs for researchers (FP7-PEOPLE), which are more attractive for research-oriented teachers than are short-term (usually 5-day) Erasmus teaching trips.

Figure 3 depicts the locations of the clusters. We can observe that very few institutions are embedded in both the collaboration and mobility networks. HEIs are embedded in either the collaboration network, such as elite English and German universities (Cluster 2), or the mobility network, such as several Spanish, Italian, Scandinavian and a few Eastern European universities located in attractive travel destinations (Cluster 1). The remaining universities, which are less embedded in either network, are dispersed across several countries and cannot be linked to a specific geographic location (Cluster 3).

Fig. 3
figure 3

Locations of specified clusters

Full size image

Our calculations demonstrate that mobility and research collaboration networks differ greatly. Since eigenvector centralities are normalized and relative values, they should be compared with each other. Based on our data, the correlation between the centralities of student’s mobility and the collaboration network is only 0.15; nevertheless, the centralities between teachers’ mobility and collaboration is insignificant. The results of the correlation between absolute numbers paint a similar picture. Collaboration and publication impacts are positively correlated with one another, but these properties are not or are very weakly correlated with mobility parameters, such as indegree (number of incoming teachers/students), outdegree (number of outgoing teachers/students) or embeddedness (eigenvector centralities). There is a strong, positive correlation (see dark blue boxes in Table 2) between collaboration properties, such as Leiden’s rankings of collaborated publications, and embeddedness in a collaboration network (measured by eigenvector centrality) (see Table 2).

Table 2 Correlation table between collaboration and mobility network properties
Full size table

The proposed aggregated (multilayer) centrality strongly correlates with mobility networks’ centrality and weakly correlates with publication impacts, rather constituting a property of research (the second mission of universities). The eigenvector centrality of the multilayer network allows us to specify new rankings based on the embeddedness of the multilayer network. Table 3) displays the rankings of the top 20 universities by their embeddedness in research collaboration and mobility networks.

Table 3 Top 20 universities based on multilayer eigenvector centrality
Full size table

Summary and conclusion

The study presents a method for assessing universities’ performance from a new perspective (i.e., embeddedness in mobility and collaboration networks) using existing, freely available data and social network analysis tools.

Our paper contributes to the relevant literature in various ways. To the best of our knowledge, this study is the first investigating universities’ performance using a network-based approach. Evidence from the scientific literature has shown that participation in collaboration networks fosters the fulfilment of HEIs’ core missions; thus, measuring their networking capacity (as we call it: embeddedness in collaboration networks) could reveal new aspects related to overall HEI performance. To test the proposed ranking method, we used data regarding the beneficiaries of the EU-funded Erasmus and the FP7 programs, both of which are based on collaboration with other academic and non-academic actors. Embeddedness in Erasmus mobility networks is primarily a measure of the universities’ first mission (education), and embeddedness in the FP7 network is mainly a measure of the second mission (research). However, both measures relate to the universities’ third mission, as strong embeddedness in these networks implies high engagement of the respective HEI in interaction, cooperation and knowledge diffusion. In particular, studying multilayer networks, such as for mobility and scientific project collaboration, can provide new insights into the patterns and consequences of students’ and teachers’ activities.

Second, we propose a multilayer eigenvector centrality method that offers a size-invariant indicator, which characterizes HEIs’ embeddedness.

Third, our results are policy relevant. By analyzing the correlation between mobility and research collaboration networks, we found that Erasmus mobility does not have a substantial effect on scientific and FP7 project collaborations. While the FP7 and Erasmus program aims are closely connected—they both are intended to support the EU in becoming a world-leading knowledge-based society and foster transnational cooperation (European Union 2006a, b)—the dynamics of the mobility and research collaboration networks differ. This difference is already partly reflected in the networks’ descriptive analysis since the fragmentation of the two networks, the number of nodes (i.e., partners) and the number of edges also differ. Nevertheless, only multilayer analysis of the group ego networks can reveal the differences that embeddedness highlights for the investigated universities. We found complementarity, but not synergy, between the Erasmus and the FP7 programs, although we would expect mobility, especially teacher mobility, to lead to research collaboration. If the EU wishes to increase synergy between its mobility and research collaboration programs, it should also consider longer-term Erasmus mobility, which would encourage the establishment of international research teams. Comparing mobility and collaboration network embeddedness with scientific impact demonstrates to policy makers that these programs’ use should have a mutually reinforcing effect, leading to increased synergy between these programs.

Finally, the proposed method and the data used are freely available and transparent; therefore, all of the calculations can be reproduced.

The authors argue that social impacts should not be measured without considering social network properties. The proposed multilayer network approach enables the integration of further networks (i.e., patents obtained by university-industry consortia), thus opening up opportunities for a more comprehensive assessment of universities’ performance in achieving their core missions.