An Ensemble Perspective on Multi-layer Networks

Abstract

We study properties of multi-layered, interconnected networks from an ensemble perspective, i.e. we analyze ensembles of multi-layer networks that share similar aggregate characteristics. Using a diffusive process that evolves on a multi-layer network, we analyze how the speed of diffusion depends on the aggregate characteristics of both intra- and inter-layer connectivity. Through a block-matrix model representing the distinct layers, we construct transition matrices of random walkers on multi-layer networks, and estimate expected properties of multi-layer networks using a mean-field approach. In addition, we quantify and explore conditions on the link topology that allow to estimate the ensemble average by only considering aggregate statistics of the layers. Our approach can be used when only partial information is available, like it is usually the case for real-world multi-layer complex systems.

Appendix

Note: Unless stated otherwise, here vectors are considered to be row-vectors and multiplication of vectors with matrices are left multiplications.

We assume a multi-layer network G consisting of L layers $G_{1},\ldots,G_{L}$ and n nodes. A single layer G _s contains n _s nodes and therefore $\sum _{s=1}^{L}n_{s} = n$. For a multi-layer network G we define the supra-transition matrix that can be represented in block structure according to the layers:

$$\displaystyle{ \mathbf{T} = \left (\begin{array}{c|c|c|c|c} \mathbf{T}_{1} & \ldots & \mathbf{T}_{1t} &\ldots &\mathbf{T}_{sL} \\ \hline \vdots &\ddots& \vdots &\ddots& \vdots\\ \hline \mathbf{T} _{ s1} & \ldots & \mathbf{T}_{st} &\ldots &\mathbf{T}_{sL} \\ \hline \vdots &\ddots& \vdots &\ddots& \vdots\\ \hline \mathbf{T} _{ L1} & \ldots & \mathbf{T}_{Lt}&\ldots & \mathbf{T}_{L} \end{array} \right )\,. }$$

Each T _st contains all the transition probabilities from nodes in G _s to nodes in G _t . Assuming Eq. (3.8) it follows that $\mathbf{T}_{st} =\alpha _{st}\mathbf{R}_{st}$ where R _st is a row stochastic matrix. This means that all T _st are scaled transition matrices. The factor α _st represents the weighted fraction of all links starting in G _s that end up in G _t .

In this respect we define the aggregated transition matrix $\mathfrak{T}$ of dimension L,

$$\displaystyle{ \mathfrak{T} = \left (\begin{array}{c|c|c|c|c} \alpha _{11} & \ldots & \alpha _{1t} &\ldots &\alpha _{sL} \\ \hline \vdots &\ddots& \vdots &\ddots& \vdots\\ \hline \alpha _{ s1} & \ldots & \alpha _{st} &\ldots &\alpha _{sL} \\ \hline \vdots &\ddots& \vdots &\ddots& \vdots\\ \hline \alpha _{ L1} & \ldots & \alpha _{Lt}&\ldots & \alpha _{L} \end{array} \right )\,. }$$

Each vector v of dimension n can be split according to the layer-separation given by G,

$$\displaystyle{ v = \left (v^{(1)},\ldots,v^{(k)},\ldots,v^{(L)}\right )\,. }$$

Each component v ^(k) has exactly dimension n _k . We define the layer-aggregated vector $\mathfrak{v} = (\mathfrak{v}_{1},\ldots,\mathfrak{v}_{L})$ of dimension L as follows

$$\displaystyle{ \forall k \in \{ 1,\ldots,L\}\quad \mathfrak{v}_{k} =\sum _{ i=1}^{n_{k} }\left [v^{(k)}\right ]_{ i}\,. }$$

We use the bracket notation [v] _i to represent the i-th entry of the vector v. Analogously, by [v M] _i we mean the i-th entry w _i that represents the multiplication of v with a matrix M, i.e. w = v M. Further, by | v | we indicate the sum of the entries of v, $\vert v\vert =\sum _{i}v_{i} =\sum _{i}[v]_{i}$.

Theorem 1

For a multi-layer network G consisting of L layers we assume the supra-transition matrix T to consist of block matrices T _st such that for all s,t ∈{ 1,…,L}, $\mathbf{T}_{st} =\alpha _{st}\mathbf{R}_{st}$ where $\alpha _{st} \in \mathbb{Q}$ and R _st is a row stochastic transition matrix. The multi-layer aggregation is defined by $\mathfrak{T} =\{\alpha _{st}\}_{st}$ . If an eigenvalue $\lambda$ of the matrix T corresponds to an eigenvector v with a layer-aggregation $\mathfrak{v}$ that satisfies $\mathfrak{v}\neq 0$ then $\lambda$ is also an eigenvalue of $\mathfrak{T}$.

Proof

Assume v is a left eigenvector of T corresponding to the eigenvalue $\lambda$. Therefore, it holds that $v\mathbf{T} =\lambda v$. Let v ^(k) be the k-th part of v that corresponds to the layer G _k . We can write the matrix multiplication in block structure.

$$\displaystyle{ \left (v^{(1)},\ldots,v^{(k)},\ldots,v^{(L)}\right )\mathbf{T} = \left (\sum _{ l}v^{(l)}\mathbf{T}_{ l1},\ldots,\sum _{l}v^{(l)}\mathbf{T}_{ lk},\ldots,\sum _{l}v^{(l)}\mathbf{T}_{ lL}\right )\,. }$$

Each v ^(k) is a row vector which length is equal to the amount of nodes n _k in G _k . The transformation $\sum _{l}v^{(l)}\mathbf{T}_{lk}$ is also a row vector with the same length as v ^(k). According to the eigenvalue equation it holds that for all k ∈ { 1, …, L}

$$\displaystyle{ \lambda v^{(k)} = \left (v\mathbf{T}\right )^{(k)} =\sum _{ l}v^{(l)}\mathbf{T}_{ lk}\,. }$$

Now let us denote the sum of the vector entries of v ^(k) as

$$\displaystyle{ \mathfrak{v}_{k} =\sum _{i}\left [v^{(k)}\right ]_{ i}\,. }$$

Further, we define layer-aggregated vector consisting of this sums by $\mathfrak{v} = \left (\mathfrak{v}_{1},\ldots,\mathfrak{v}_{n}\right )$. Note that for a general row stochastic matrix M and its multiplication with any vector v it holds that $\sum _{j}[v]_{j} =\sum _{j}[v\mathbf{M}]_{j}$. For the components after multiplication with T we can deduce

$$\displaystyle\begin{array}{rcl} \sum _{i}\left [\left (v\mathbf{T}\right )^{(k)}\right ]_{ i}& =& \sum _{i}\left [\sum _{l}v^{(l)}\mathbf{T}_{ lk}\right ]_{i} =\sum _{i}\left [\sum _{l}\alpha _{lk}v^{(l)}\mathbf{R}_{ lk}\right ]_{i} {}\\ & =& \sum _{l}\alpha _{lk}\sum _{i}\left [v^{(l)}\mathbf{R}_{ lk}\right ]_{i} =\sum _{l}\alpha _{lk}\sum _{i}\left [v^{(l)}\right ]_{ i} =\sum _{l}\alpha _{lk}\mathfrak{v}_{l}\,. {}\\ \end{array}$$

If we multiply $\mathfrak{v}$ with $\mathfrak{T}$ and look at a single entry of $\mathfrak{v}\mathfrak{T}$ we get

$$\displaystyle{ \left [\mathfrak{v}\mathfrak{T}\right ]_{k} =\sum _{l}\mathfrak{v}_{l}\mathfrak{T}_{lk} =\sum _{l}\mathfrak{v}_{l}\alpha _{lk}\,. }$$

Hence it holds that

$$\displaystyle{ \left [\mathfrak{v}\mathfrak{T}\right ]_{k} =\sum _{i}\left [\left (v\mathbf{T}\right )^{(k)}\right ]_{ i}\,, }$$

and therefore

$$\displaystyle{ \mathfrak{v}\mathfrak{T} = \left (\sum _{i}\left [\left (v\mathbf{T}\right )^{(1)}\right ]_{ i},\ldots,\sum _{i}\left [\left (v\mathbf{T}\right )^{(L)}\right ]_{ i}\right )\,. }$$

Finally since T is row stochastic and $\lambda v = v\mathbf{T}$ we have that

$$\displaystyle\begin{array}{rcl} \lambda \mathfrak{v}& =& \lambda \left (\mathfrak{v}_{1},\ldots,\mathfrak{v}_{n}\right ) {}\\ & =& \lambda \left (\sum _{i}[v^{(1)}]_{ i},\ldots,\sum _{i}[v^{(L)}]_{ i}\right ) {}\\ & =& \left (\sum _{i}\left [\left (v\mathbf{T}\right )^{(1)}\right ]_{ i},\ldots,\sum _{i}\left [\left (v\mathbf{T}\right )^{(L)}\right ]_{ i}\right ) {}\\ & =& \mathfrak{v}\mathfrak{T}\,. {}\\ \end{array}$$

Therefore, $\lambda$ is also an eigenvalue of $\mathfrak{T}$ to the eigenvector $\mathfrak{v}$ defined as before. It is important to note that this only holds if $\mathfrak{v}\neq 0$.

The procedure used in the proof of the previous theorem applies to several eigenvalues of T but at most L of them. Next we give a proposition for the reversed statement of Theorem 1.

Proposition 1

Let G be a multi-layer network that consists of L layers and fulfills all of the conditions of Theorem 1 . Let $\mathfrak{T} =\{\alpha _{st}\}_{st}$ be the multi-layer aggregation of T . If $\lambda$ is an eigenvalue of $\mathfrak{T}$ then $\lambda$ is also an eigenvalue of T.

Proof

Assume that $\lambda$ is an eigenvalue of $\mathfrak{T}$. For each matrix there exist a left and right eigenvector that correspond to the same eigenvalue $\lambda$. Assume the $\mathfrak{w}$ is the right eigenvector and therefore a column vector. Hence $\mathfrak{T}\mathfrak{w} =\lambda \mathfrak{w}$ and

$$\displaystyle{ \mathfrak{T}\mathfrak{w} = \left (\sum _{j}\alpha _{1j}\mathfrak{w}_{j},\ldots,\sum _{j}\alpha _{kj}\mathfrak{w}_{j},\ldots,\sum _{j}\alpha _{Lj}\mathfrak{w}_{j}\right )^{\top } =\lambda \mathfrak{w}\,. }$$

Now we generate a column vector w of dimension n such that for all the layer components w ^(k) it holds that

$$\displaystyle{ w^{(k)} = (\mathfrak{w}_{ k},\ldots,\mathfrak{w}_{k})^{\top }\,. }$$

Next we perform a right multiplication of w with T,

$$\displaystyle\begin{array}{rcl} \mathbf{T}w& =& \left (\sum _{l}\mathbf{T}_{1l}w^{(l)},\ldots,\sum _{ l}\mathbf{T}_{kl}w^{(l)},\ldots,\sum _{ l}\mathbf{T}_{Ll}w^{(l)}\right )^{\top } {}\\ & =& \left (\sum _{l}\alpha _{1l}\mathbf{R}_{1l}w^{(l)},\ldots,\sum _{ l}\alpha _{kl}\mathbf{R}_{kl}w^{(l)},\ldots,\sum _{ l}\alpha _{Ll}\mathbf{R}_{Ll}w^{(l)}\right )^{\top }\,. {}\\ \end{array}$$

Since all R _st are row stochastic matrices and w ^(l) contains only the value $\mathfrak{w}_{l}$ for each entry we get $\mathbf{R}_{st}w^{(l)} = w^{(l)}$. It follows that

$$\displaystyle\begin{array}{rcl} \mathbf{T}w& =& \left (\sum _{l}\alpha _{1l}w^{(l)},\ldots,\sum _{ l}\alpha _{kl}w^{(l)},\ldots,\sum _{ l}\alpha _{Ll}w^{(l)}\right )^{\top } {}\\ & =& \left (\lambda w^{(1)},\ldots,\lambda w^{(k)},\ldots,\lambda w^{(L)}\right )^{\top } {}\\ & =& \lambda w\,. {}\\ \end{array}$$

And therefore $\lambda$ is also an eigenvalue of T.

In the case of a diffusion process we are especially interested in the second-largest eigenvalue of T, denoted by $\lambda _{2}(\mathbf{T})$, which is related to algebraic connectivity of T. In this perspective the following corollary is useful:

Corollary 1

Let G be a multi-layer network consisting of L layers that fulfill all of the conditions of Theorem 1 . Further assume that G is partitioned according to a spectral partitioning, i.e. according to the eigenvector corresponding to $\lambda _{2}(\mathbf{T})$ , then $\lambda _{2}(\mathbf{T}) =\lambda _{2}(\mathfrak{T})$.

Proof

In general all the eigenvectors of a transition matrix, except the eigenvector corresponding to the largest eigenvalue that is equal to one, sum up to zero. However, these eigenvectors consist of positive and negative entries that allow for a spectral partitioning. Especially the eigenvector v ₂ that corresponds to the second-largest eigenvalue $\lambda _{2}(\mathbf{T})$, can be used for the partitioning of the network. This eigenvector is related to the Fiedler vector that is also used for spectral bisection [19]. Therefore if the layer-partition of G coincides with this spectral partitioning we assure that the layer-aggregated vector of v ₂ satisfies $\mathfrak{v}_{2}\neq 0$. Considering this and Proposition 1 the corollary follows directly from Theorem 1.

Given Eq. (3.8) we can fully describe the spectrum of T based on the intra-layers transition blocks T _i for i ∈ { 1, …, n} and the spectrum of $\mathfrak{T}$. Note that with uniform columns of a matrix M we mean that each column of M contains the same value in each entry. However, this value can be different for different columns.

Proposition 2

Let T be the supra-transition matrix of a multi-layer network G that consist of L layers and satisfies Eq. ( 3.8 ). If T has off-diagonal block matrices T _st , for s,t ∈{ 1,…,n} and s ≠ t, that all have uniform columns, then the spectrum of T can be decomposed as

$$\displaystyle{ Spec(\mathbf{T}) =\{ 1,\lambda _{2},\ldots,\lambda _{L}\} \cup \left (\bigcup _{s=1}^{L}Spec(\mathbf{T}_{ s})\setminus \{\lambda _{1}(\mathbf{T}_{s})\}\right )\,, }$$

where T _s are the block matrices of T corresponding to the single layers G _s and $\lambda _{1}(\mathbf{T}_{s})$ the largest eigenvalue of T _s . The eigenvalues $\lambda _{2},\ldots,\lambda _{L}$ are attributed to the interconnectivity of layers.

Proof

To prove this statement we just have to show that all eigenvalues (except the largest one) of T _s for s ∈ { 1, …, L} are also eigenvalues of T. We assume that $\lambda$ is any eigenvalue corresponding to the eigenvector u of some block matrix T _r , i.e. $\lambda u = u\mathbf{T}_{r}$. We define a row vector v that is zero everywhere except at the position where it corresponds to T _r . The vector v looks like v = (0, …, 0, u, 0, …, 0). Now we investigate what happens if we multiply this vector with the transition matrix T.

$$\displaystyle{ v\mathbf{T} = \left (v^{(1)},\ldots,v^{(k)},\ldots,v^{(L)}\right )\mathbf{T} = \left (\sum _{ l}v^{(l)}\mathbf{T}_{ l1},\ldots,\sum _{l}v^{(l)}\mathbf{T}_{ lk},\ldots,\sum _{l}v^{(l)}\mathbf{T}_{ lL}\right )\,. }$$

Let us take a look at the effect of the matrix multiplication on an arbitrary component v ^(k) with k ≠ r and recall that v ^(k) is equal to a zero vector 0 for k ≠ r.

$$\displaystyle{ (v\mathbf{T})^{(k)} =\sum _{ l}v^{(l)}\mathbf{T}_{ lk} =\sum _{l,l\neq r}\mathbf{0}\mathbf{T}_{lk} + u\mathbf{T}_{rk} = u\mathbf{T}_{rk}\,. }$$

Note that all eigenvectors u of a transition matrix that are not related to the largest eigenvalue sum up to zero. Therefore it holds that u T _rk  = 0 since T _rk has uniform columns and therefore u T _rk yields in a vector where each entry is equal to some multiple of | u | . In case of k = r it holds that v ^(k) = u and we get

$$\displaystyle{ (v\mathbf{T})^{(r)} =\sum _{ l}v^{(l)}\mathbf{T}_{ lr} =\sum _{l,l\neq r}\mathbf{0}\mathbf{T}_{lk} + u\mathbf{T}_{rr} = u\mathbf{T}_{r} =\lambda r\,. }$$

Hence, it holds that $v\mathbf{T} =\lambda v$, which means that $\lambda$ is also an eigenvalue of T. This way we get n − L eigenvalues of T apart from the largest eigenvalue that is equal to one. The remaining eigenvalues denoted by $\lambda _{2},\ldots,\lambda _{L}$ are not attributed to any block matrix of T. Therefore they are considered to be the interconnectivity eigenvalues.

Corollary 2

Let G be a multi-layer network consisting of L layers that satisfies Eq. ( 3.8 ) and the conditions of Proposition 2 . Then the aggregated matrix $\mathfrak{T} =\{\alpha _{st}\}_{st}$ has spectrum

$$\displaystyle{ Spec(\mathfrak{T}) =\{ 1,\lambda _{2},\ldots,\lambda _{L}\}\,, }$$

and it holds that $\lambda _{2},\ldots,\lambda _{L} \in Spec(\mathbf{T})$.

Proof

Note that every eigenvalue $\lambda \neq 1$ of some block matrix T _r with $\lambda u = u\mathbf{T}_{r}$ is by Proposition 2 also an eigenvalue of T. Furthermore, $\lambda$ is attributed to the eigenvector v = (0, …, 0, u, 0, …, 0) of T. However | v | = 0 since u is an eigenvector of a transition matrix, not related to the largest eigenvalue, and therefore sums up to zero. Hence all eigenvalues fulfilling this condition are by Theorem 1 not eigenvalues of $\mathfrak{T}$. Since $\mathfrak{T}$ contains at least L eigenvalues that by Proposition 1 also correspond to eigenvalues of T, the remaining eigenvalues $\lambda _{2},\ldots,\lambda _{L}$ have to also be eigenvalues of $\mathfrak{T}$.

In the following we provide a useful proposition for the eigenvalues arising from the inter-links in case of two layers. Note that by the function T(⋅ ) applied to matrix M we indicate that T(M) is the row-normalization of M.

Proposition 3

Let G be a multi-layer network that satisfies Eq. ( 3.8 ), consisting of two networks G ₁ and G ₂ in separate layers. Assume that the supra-transition matrix T has the form

$$\displaystyle{ \mathbf{T} = \left (\begin{array}{c|c} \mathbf{T}_{1} & \mathbf{T}_{12} \\ \hline \mathbf{T}_{21} & \mathbf{T}_{2} \end{array} \right ) = \left (\begin{array}{c|c} \beta T(\mathbf{A}_{1}) &(1-\beta )\mathbf{T}_{12}^{I} \\ \hline (1-\beta )\mathbf{T}_{21}^{I}& \beta T(\mathbf{A}_{2}) \end{array} \right )\,, }$$

where T ^I is the transition matrix of the layer G that only consists of the inter-layer links and $\beta \in \mathbb{Q}$ is a constant. Furthermore, assume that T ₁ and T ₂ have uniform columns.

Then for $\lambda \in Spec(\mathbf{T}^{I})$ and $\lambda \neq 1,-1$ it holds that $(1-\beta )\lambda \in Spec(\mathbf{T})$.

Proof

If v is an eigenvector to the eigenvalue $\lambda \neq 1,-1$ of T ^I it holds that $v\mathbf{T}^{I} =\lambda v$. Hence,

$$\displaystyle{ v\mathbf{T}^{I} = \left (v^{(1)},v^{(2)}\right )\mathbf{T}^{I} = \left (v^{(2)}\mathbf{T}_{ 21}^{I},v^{(1)}\mathbf{T}_{ 12}^{I},\right ) =\lambda \left (v^{(1)},v^{(2)}\right )\,. }$$

Because $\lambda v^{(2)} = v^{(1)}\mathbf{T}_{12}^{I}$, we get $\lambda ^{2}v^{(1)} = v^{(1)}\mathbf{T}_{12}^{I}\mathbf{T}_{21}^{I}$. Therefore, v ⁽¹⁾ is also an eigenvector of the transition matrix $\mathbf{T}_{12}^{I}\mathbf{T}_{21}^{I}$ to the eigenvalue $\lambda ^{2}$. Note that $\lambda \neq 1,-1$ hence $\lambda ^{2} <1$ which implies that v ⁽¹⁾ does not correspond to the largest eigenvalue and therefore its entries sum up to zero. The same holds for v ⁽²⁾ and the matrix $\mathbf{T}_{21}^{I}\mathbf{T}_{12}^{I}$. For the multiplication of v with T we deduce that

$$\displaystyle{ v\mathbf{T} = \left (v^{(1)},v^{(2)}\right )\mathbf{T} = \left (v^{(1)}\mathbf{T}_{ 1} + (1-\beta )v^{(2)}\mathbf{T}_{ 21}^{I},(1-\beta )v^{(1)}\mathbf{T}_{ 12}^{I} + v^{(2)}\mathbf{T}_{ 2},\right )\,. }$$

Since T ₁ and T ₂ have uniform columns we get $v^{(1)}\mathbf{T}_{1} = \mathbf{0}$ and $v^{(2)}\mathbf{T}_{2} = \mathbf{0}$. And therefore $v\mathbf{T} = (1-\beta )\lambda v$ and $(1-\beta )\lambda \in Spec(\mathbf{T})$.

Proposition 3 can be extended to multiple layers, however the proof is more involved and will be omitted.