A family of tractable graph metrics

A family of graph metrics

Optimization over Permutation Matrices. Our first result establishes that \(d_{\mathbb {P}^{n}}\) is a pseudometric over all weighted graphs when ∥·∥ is an arbitrary entry-wise or operator norm.

Hence, \(d_{\mathbb {P}^{n}}\) is a pseudometric under any entry-wise or operator norm over arbitrary directed, weighted graphs.

Optimization over the Birkhoff Polytope. Our second result states that the pseudometric property extends to the relaxed version of the chemical distance, in which permutations are replaced by doubly stochastic matrices.

Hence, if \(S=\mathbb {W}^{n}\) and ∥·∥ is an arbitrary entry-wise norm, then (12) defines a pseudometric over undirected graphs. The symmetry property (9c) breaks if ∥·∥ is an operator norm or graphs are directed. In both of these two cases d_S is a quasimetric over the quotient space Ω/ ∼_d, and symmetry is attained via the symmetric extension \(\bar {d}_{S}\).

Theorem 2 has significant practical implications. In contrast to \(d_{\mathbb {P}^{n}}\) and its extensions implied by Theorem 1, computing \(d_{{\mathbb {W}}^{n}}\) under any operator or entry-wise norm is tractable, in the sense that involves minimizing a convex function subject to linear constraints (Boyd and Vandenberghe 2004; Nesterov and Nemirovskii 1994; Bertsekas 1997).

Optimization over the Stiefler Manifold. A more limited result applies to the case when S is the Stiefel manifold \({\mathbb {O}}^{n}\):

Though (12) is not a convex problem when \(S=\mathbb {O}^{n}\), it is also tractable. Umeyama (Umeyama 1988) shows that the optimization can be solved exactly when ∥·∥=∥·∥_F and \(\Omega ={\mathbb {S}}^{n}\) (i.e., for undirected graphs) by performing a spectral decomposition on A and B. We extend this result, showing that the same procedure also applies when ∥·∥ is the operator 2-norm (see Thm. 7 in “Metric computation over the Stiefler manifold.” section). In the general case of directed graphs, (12) is a classic example of a problem that can be solved through optimization on manifolds (Absil et al. 2009).

Equivalence Classes. Observe that the equivalence of matrix norms, as stated by Eq. (3), implies that if d_S(A,B)=0 for one matrix norm ∥·∥ in (12), it will be so for all. As a result, pseudometrics d_S defined through (12) for a given S have the same quotient space \(\Omega /\!\sim _{d_{S}}\), irrespectively of norm ∥·∥. We therefore turn our attention to characterizing this quotient space in the three cases when S is the set of permutation, doubly stochastic, and orthononal matrices.

When \(S=\mathbb {P}^{n}\), \(\Omega /\!\sim _{d_{\mathbb {P}^{n}}}\) is the quotient space defined by graph isomorphism: any two adjacency matrices \(A,B\in {\mathbb {R}}^{n\times n}\) satisfy \(d_{\mathbb {P}^{n}} (A,B)= 0\) if and only if their (possibly weighted) graphs are isomorphic.

When \(S=\mathbb {W}^{n}\), the quotient space \(\Omega /\!\sim _{d_{\mathbb {W}^{n}}}\) has a connection to the Weisfeiler-Lehman (WL) algorithm (Weisfeiler and Lehman 1968) described in “The Weisfeiler-Lehman (WL) Algorithm.” section: Ramana et al. (Ramana et al. 1994) show that \(d_{{\mathbb {W}}^{n}}(A,B)=0\) if and only if G_A and G_B receive identical colors by the WL algorithm (see also (Tinhofer 1986) for another characterization of this quotient space). This equivalence relation is sometimes called called fractional linear isomorphism (Ramana et al. 1994).

Finally, if \(S=\mathbb {O}^{n}\) and \(\Omega =\mathbb {S}^{n}\), i.e., graphs are undirected, then \(\Omega /\!\sim _{d_{{\mathbb {O}}^{n}}}\) is determined by co-spectrality: \(d_{{\mathbb {O}}^{n}}(A,B)=0\) if and only if A,B have the same spectrum. When \(\Omega ={\mathbb {R}}^{n\times n}\), \(d_{{\mathbb {O}}^{n}}(A,B)=0\) implies that A,B are co-spectral, but co-spectral matrices A,B do not necessarily satisfy \(d_{{\mathbb {O}}^{n}}(A,B)=0\). Put differently, the quotient space \(\Omega /\!\sim _{d_{{\mathbb {O}}^{n}}}\) in this case is a refinement of the quotient space of co-spectrality.

Incorporating metric embeddings

We have seen that the chemical distance \(d_{\mathbb {P}^{n}}\) can be relaxed to \(d_{\mathbb {W}^{n}}\) or \(d_{{\mathbb {O}}^{n}}\), gaining tractability while still maintaining the metric property. In practice, nodes in a graph often contain additional attributes that one might wish to leverage when computing distances. In this section, we show that such attributes can be seamlessly incorporated in d_S either as soft or hard constraints, without violating the metric property.

Metric Embeddings. Given a graph G_A of size n, a metric embedding of G_A is a mapping \(\psi _{A}:[n]\to \tilde {\Omega } \) from the nodes of the graph to a metric space \((\tilde {\Omega },\tilde {d})\). That is, ψ_A maps nodes of the graph to \(\tilde {\Omega }\), where \(\tilde {\Omega }\) is endowed with a metric \(\tilde {d}\). We refer to a graph endowed with an embedding ψ_A as an embedded graph, and denote this by (A,ψ_A), where \(A\in {\mathbb {R}}^{n\times n}\) is the adjacency matrix of G_A. We list two examples:

Example 1: Node Attributes. Consider an embedding of a graph to \((\ensuremath {\mathbb {R}}^{k},\|\cdot \|_{2})\) in which every node v∈V is mapped to a k-dimensional vector describing “local” attributes. These can be exogenous: e.g., features extracted from a user’s profile (age, binarized gender, etc.) in a social network. Alternatively, attributes may be endogenous or structural, extracted from the adjacency matrix A, e.g., the node’s degree, the size of its k-hop neighborhood, its page-rank, etc.

Given a graph G_A, a mapping \(\psi _{A}:[n]\to \tilde {\Omega }\) is then a metric embedding. The values of \(\tilde {\Omega }\) are invariably called colors or labels, and a graph embedded in \(\tilde {\Omega }\) is a colored or labeled graph. Colors can again be exogenous or structural: e.g., if the graph represents an organic molecule, colors can correspond to atoms, while structural colors can be, e.g., the output of the WL algorithm (see “The Weisfeiler-Lehman (WL) Algorithm.” section) after k iterations.

As discussed below, node attributes translate to soft constraints in metric (12), while node colors correspond to hard constraints. The unified view through embeddings allows us to establish metric properties for both simultaneously (c.f. Theorems 4 and 5).

Note that, in the case of colored graphs and the Kronecker delta distance, minimizing (16) finds a \(P \in \mathbb {P}^{n}\) that maps nodes in A to nodes in B of equal color. It is not hard to verify that \(\min _{P\in \mathbb {P}^{n}} \mathop {\mathsf {tr}}\left (P^{\top } D_{\psi _{A},\psi _{B}} \right) \) induces a metric between graphs embedded in \((\tilde {\Omega },\tilde {d})\). In fact, this follows from the more general theorem we prove below (Theorem. 4) for A=B=0, i.e., for distances between embedded graphs with no edges.

Despite the combinatorial nature of \(\mathbb {P}^{n}\), the problem of minimizing (16) over \({\mathbb {P}}^{n}\) is a maximum weighted matching problem, which can be solved through, e.g., the Hungarian algorithm (Kuhn 1955), in O(n³) time (Jonker and Volgenant 1987). We note that this metric is not as expressive as (12): depending on the definition of the embeddings ψ_A, ψ_B, attributes may only capture “local” similarities between nodes, as opposed to the “global” view of a mapping attained by (12).

for some compact set \(S\subset \ensuremath {\mathbb {R}}^{n\times n}\) and matrix norm ∥·∥. Our next result states that incorporating this linear term does not affect the pseudometric property of d_S.

We stress here that this result is non-obvious: it is not true that adding any linear term to d_S leads to a quantity that satisfies the triangle inequality. It is precisely because \(\phantom {\dot {i}\!}D_{{\psi }_{A},{\psi }_{B}}\) contains pairwise distances that Theorem 4 holds. We can similarly extend Theorem 2:

Adding the linear term (16) in d_S has significant practical advantages. Beyond expressing exogenous attributes, a linear term involving colors, combined with a Kronecker distance, translates into hard constraints: any permutation attaining a finite objective value must map nodes in one graph to nodes of the same color. Theorem 5 therefore implies that such constraints can thus be added to the optimization problem, while maintaining the metric property. In practice, as the number of variables in optimization problem (12) is n², incorporating such hard constraints can significantly reduce the problem’s computation time; we illustrate this in “Experiments” section. Note that adding (16) to \(d_{\mathbb {O}^{n}}\) does not preserve the metric property.

Proof of Theorems 1–3

We define several properties that play a crucial role in our proofs.

The proofs of Theorems 1–3 rely on several common lemmas. The first three establish conditions under which (12) satisfies the triangle inequality (9d), symmetry (9c), and weak property (9e), respectively:

Both the set of permutation matrices \(\mathbb {P}^{n}\)and the Stiefel manifold \(\mathbb {O}^{n}\) are groups w.r.t. matrix multiplication: they are closed under multiplication, contain the identity I, and are closed under inversion. Hence, if they are also contractive w.r.t. a matrix norm ∥·∥, \(d_{{\mathbb {P}}^{n}}\) and \(d_{{\mathbb {O}}^{n}}\) defined in terms of this norm satisfy all assumptions of Lemmas 1–3. We therefore turn our attention to this property.

Hence, Theorem 1 follows as a direct corollary of Lemmas 1–4. Indeed, \(d_{{\mathbb {P}}^{n}}\) is non-negative, symmetric by Lemmas 2 and 4, satifies the triangle inequality by Lemmas 1 and 4, as well as property (9e) by Lemma 3; hence \(d_{{\mathbb {P}}^{n}}\) is a pseudometric over \({\mathbb {R}}^{n\times n}\). Our next lemma shows that the Stiefel manifold \({\mathbb {O}}^{n}\) is contractive for 2-norms:

Theorem 3 therefore follows from Lemmas 1–3 and Lemma 5, along with the fact that \({\mathbb {O}}^{n}\) is a group. Note that \({\mathbb {O}}^{n}\) is not contractive w.r.t. other norms, e.g., ∥·∥₁ or ∥·∥_∞.

To prove Theorem 2, we first show that Lemma 4 along with the Birkoff-von Neumann theorem imply that \({\mathbb {W}}^{n}\) is also contractive:

Unfortunately, the Birkhoff polytope \(\mathbb {W}^{n}\) is not a group, as it is not closed under inversion. Nevertheless, it is closed under transposition; in establishing (partial) symmetry of \(d_{{\mathbb {W}}^{n}}\), we leverage the following lemma:

The first part of Theorem 2 therefore follows from Lemmas 1, 3, 6, and 7: this is because \({\mathbb {W}}^{n}\) contains the identity I and is closed under both transposition and multiplication, while all entry-wise norms are transpose invariant.

To prove the second part, observe that operator norms are not transpose invariant. However, if ∥·∥ is an operator norm, or \(\Omega ={\mathbb {R}}^{n\times n}\), then Lemma 6 and Lemma 1 imply that \(d_{{\mathbb {W}}^{n}}\) satisfies non-negativity (9a) and the triangle inequality (9d), while Lemma 3 implies that it satisfies (9e). These properties are inherited by extension \(\bar {d}_{S}\), given by (10), which also satisfies symmetry (9c), and the second part of Theorem 2 follows.

Proof of Theorems 4 and 5

We begin by establishing conditions under which d_S satisfies the triangle inequality (9d). We note that, in contrast to Lemma 1, we require the additional condition that \(S\subseteq {\mathbb {W}}^{n}\), which is not satisfied by \({\mathbb {O}}^{n}\).

The weak property (9e) is again satisfied provided the identity is included in S.

To attain symmetry over \(\Omega =\ensuremath {\mathbb {R}}^{n\times n}\times \Psi _{\tilde {\Omega }}^{n}\), we again rely on closure under inversion, as in Lemma 2; nonetheless, in contrast to Lemma 2, due to the linear term, we also need to assume the orthogonality of elements of S.

Theorem 4 therefore follows from the above lemmas, as \(S=\mathbb {P}^{n}\) contains I, it is closed under multiplication and inversion, is a subset of \({\mathbb {W}}^{n}\cap {\mathbb {O}}^{n}\) by (7), and is contractive w.r.t. all operator and entrywise norms. Theorem 5 also follows by using the following lemma, along with Lemmas 8 and 9.

Metric computation over the Stiefler manifold.

In this section, we describe how to compute the metric d_S in polynomial time when \( S = \mathbb {O}^{n}\) and ∥·∥ is the Frobenius norm or the operator 2-norm. The algorithm for the Frobenius norm, and the proof of its correctness, is due to Umeyama (Umeyama 1988); we reprove it for completeness, along with its extension to the operator norm.

Both cases make use of the following lemma:

In both norm cases, for \(A,B \in \mathbb {S}^{n}\), we can compute d_S using a simple spectral decomposition, which dominates computations and can be performed in O(n³) time. Let A=UΣ_AU^T and B=VΣ_BV^T be the spectral decomposition of A and B. As A and B are real and symmetric, we can assume \(U,V \in \mathbb {O}^{n}\). Recall that U⁻¹=U^⊤ and V⁻¹=V^⊤, while Σ_A and Σ_B are diagonal and contain the eigenvalues of A and B sorted in increasing order; this ordering matters for computations below.

The following theorem establishes that this decomposition readily yields the distance d_S, as well as the optimal orthogonal matrix P^∗, when ∥·∥=∥·∥_F:

We can compute d_S when \(S= \mathbb {O}^{n}\) and ∥·∥ is the operator norm in the exact same way.

Let \(\tilde {B} = -B\) have eigenvalues \(\tilde {b}_{1} \leq \tilde {b}_{2} \leq...\leq \tilde {b}_{n}\) and let \(C = A + \tilde {B}\) have eigenvalues c₁≤c₂≤...≤c_n. We make use of the following lemma by Weyl (see Theorem 4.3.1 (Weyl), page 239, in (Horn and Johnson 2012)) to lower-bound c_n.

If we choose \(X = \tilde {B}\), Y=A and i=n we get \(a_{j} + \tilde {b}_{n+1-j} \leq c_{n}\) for all j=1,…,n. Since \(\tilde {b}_{n+1-j} = -{b}_{j}\) we get that a_j−b_j≤c_n, for any j. Similarly, by exchanging the role of A and B, we can lower bound the largest eigenvalue of B−A, say d_n, by b_j−a_j for any j. Notice that, by definition of the operator norm and the fact that A−B is Hermitian, ∥A−B∥₂≥|c_n| and ∥B−A∥₂≥|d_n|. Since ∥B−A∥₂=∥A−B∥₂ we have that ∥A−B∥₂≥ max{|c_n|,|d_n|}≥ max{c_n,d_n}≥ max{a_j−b_j,b_j−a_j}=|a_j−b_j| for all j. Taking the maximum over j we get that ∥A−B∥₂≥ maxj|a_j−b_j|, and the lemma follows.

Note again that if Σ_A and Σ_B are diagonal matrices with the ordered eigenvalues of A and B in the diagonal, then the conclusion of Lemma 14 can be written as ∥A−B∥₂≥∥Σ_A−Σ_B∥₂. The proof of Thm. 7 proceeds along the same steps as the above proof, using again the fact that, by Lemma 12, ∥M∥₂=∥MP∥₂=∥PM∥₂ for any \(P \in {\mathbb {O}}^{n}\) and any matrix M, along with Lemma 15.

Experimental setup

Graphs We use synthetic graphs from six classes summarized in Table 3: Barabasi Albert with degree d (Bd), Erdos Renyi with probablity p (Ep), Power Law Tree (P), Regular with degree d (Rd), Small World (S), Watts Strogatz with degree d (Ws). In addition, we use a dataset of small graphs, comprising all 853 connected graphs of 7 nodes. Finally, we use a collaboration graph with 5242 nodes and 14496 edges representing author collaborations.

We use all public algorithms as black boxes with their default parameters, as provided by the authors.

Experimental results

Triangle Inequality Violations. Given graphs A, B and C and a distance d, a Triangle Inequality Violation (TIV) occurs when d(A,C)>d(A,B)+d(B,C). Being metrics, none of our distances induce TIVs; this is not the case for the remaining algorithms in Table 2. Figure 2 shows the TIV fraction across the synthetic graphs of Table 3 while Fig. 3 shows the fraction of TIVs found on the 853 small graphs (n=7). NetAlignMR also produces no TIVs on the small graphs, but it does induce TIVs in synthetic graphs. We observe that it is easier to find TIVs when graphs are close: in synthetic graphs, TIVs abound for n=10. No algorithm performs well across all categories of graphs.

Open image in new window

Open image in new window

Effect of TIVs on Clustering. Next, to investigate the effect of TIVs on clustering, we artificially introduced triangle inequality violations into the pairs of distances between graphs. We then re-evaluated clustering performance for hierarchical agglomerative clustering using the Ward method, which performed best in Fig. 1. Figure 4 shows the fraction of misclassified graphs as the fraction of TIVs introduced increases. To incur as small a perturbation on distances as possible, we introduce TIVs as follows: For every three graphs, A,B,C, with probability p, we set d(A,C)=d(A,B)+d(B,C). Although this does not introduce a TIV w.r.t. A,B, and C, this distortion does introduce TIVs w.r.t. other triplets involving A and C. We repeat this 20 times for each algorithm and each value of p, and compute the average fraction of TIVs, shown in the x-axis, and the average fraction of misclassified graphs, shown in the y-axis. As little as 1% TIVs significantly deteriorate clustering performance. Note that the fraction of TIVs is computed over the total number of TIVs possible, which grows cubicly with the number of graphs being clustered. We also see that, even after introducing TIVs, clustering based on metrics outperforms clustering based on non-metrics.

Open image in new window

Comparison to Chemical Distance. We compare how different distance scores relate to the chemical distance EXACT through two experiments on the small graphs (computation on larger graphs is prohibitive). In Fig. 5a), we compare the distances between small graphs with 7 nodes produced by the different algorithms and EXACT using the DISTATIS method of (Abdi et al. 2005). Let \(D\in {\mathbb {R}}_{+}^{835\times 835}\) be the matrix of distances between graphs under an algorithm. DISTATIS computes the normalized Laplacian of this matrix, given by L=−UDU/∥UDU∥₂ where \(U = I - \frac {\textbf {1}\textbf {1}^{\top }}{n}\). The DISTATIS score is the cosine similarity of such Laplacians (vectorized). We see that our metrics produce distances attaining high similarity with EXACT, though NetAlignBP has the highest similarity. We measure proximity to EXACT with an additional test. Given D, we compute the nearest neighbor (NN) meta-graph by connecting a graph in D to every graph at distance less than its average distance to other graps. This results in a (labeled) meta-graph, which we can compare to the NN meta-graph induced by other algorithms, measuring the fraction of distinct edges. Figure 5b shows that our algorithms perform quite well, though Natalie yields the smallest distance to EXACT.

Open image in new window

Incorporating Constraints. Computation costs can be reduced through metric embeddings, as in (17). To show this, we produce a copy of the 5242 node collaboration graph with permuted node labels. We then run the WL algorithm (Weisfeiler and Lehman 1968) to produce structural colors, which induce coloring constraints on \(P\in {\mathbb {W}}^{n}\). The WL algorithm reaches a fixed point after k=5 iterations. The support of P (i.e., the number of variables in the optimization (12)), the support of AP−PA (i.e., the number of non-zero summation terms in the objective of (12)), as well as the execution time τ of the WL algorithm, are summarized in Table 4. The original unconstrained problem involves 5242²≈27.4M variables. However, after using WL and induced costraints, the effective dimension of the optimization problem (12) reduces considerably. This, in turn, speeds up convergence time, shown in Fig. 6: including the time to compute constraints, a solution is found 110 times faster after the introduction of the constraints.

Open image in new window

Table 4

Effect of coloring/hard constraints

k	∥P∥0	∥ AP − PA ∥0	τ
1	3,747,960	100.569	133s
2	239,048	3004	104s
3	182,474	2036	136s
4	182,016	2030	169s
5	182,006	2030	200s

Effect of coloring/hard constraints on the numbers of variables (∥P∥₀) and terms of objective (∥AP−PA∥₀) using k iterations of the WL coloring algorithm. The last column shows the execution time of WL on a 40 CPU machine using Apache Spark (Zaharia et al. 2010)