Top-k overlapping densest subgraphs

Abstract

Finding dense subgraphs is an important problem in graph mining and has many practical applications. At the same time, while large real-world networks are known to have many communities that are not well-separated, the majority of the existing work focuses on the problem of finding a single densest subgraph. Hence, it is natural to consider the question of finding the top-k densest subgraphs. One major challenge in addressing this question is how to handle overlaps: eliminating overlaps completely is one option, but this may lead to extracting subgraphs not as dense as it would be possible by allowing a limited amount of overlap. Furthermore, overlaps are desirable as in most real-world graphs there are vertices that belong to more than one community, and thus, to more than one densest subgraph. In this paper we study the problem of finding top-k overlapping densest subgraphs, and we present a new approach that improves over the existing techniques, both in theory and practice. First, we reformulate the problem definition in a way that we are able to obtain an algorithm with constant-factor approximation guarantee. Our approach relies on using techniques for solving the max-sum diversification problem, which however, we need to extend in order to make them applicable to our setting. Second, we evaluate our algorithm on a collection of benchmark datasets and show that it convincingly outperforms the previous methods, both in terms of quality and efficiency.

Appendices

Appendix: Proof of Proposition 1

Let us first define $h(x;\,Y) = \left[ f(x \cup Y) - f(Y)\right] / 2$ and

$$\begin{aligned} g(x;\,Y) = h(x;\,Y) + d \mathopen {}\left( Y \cup x\right) - d \mathopen {}\left( Y\right) = h(x;\,Y) + d \mathopen {}\left( x,\,Y\right) . \end{aligned}$$

For proving the proposition, we will need Lemma 1.

Lemma 1

Let $d$ be a c-relaxed metric. Let X and Y be two disjoint sets. Then

$$\begin{aligned} c({\left| X\right| } - 1) d \mathopen {}\left( X,\,Y\right) \ge {\left| Y\right| } d \mathopen {}\left( X\right) . \end{aligned}$$

Proof

Let $y \in Y$ and $x,\,z \in X.$ By definition,

$$\begin{aligned} c( d \mathopen {}\left( x,\,y\right) + d \mathopen {}\left( z,\,y\right) ) \ge d \mathopen {}\left( x,\,z\right) . \end{aligned}$$

For a given $x \in X,$ there are exactly ${\left| X\right| } - 1$ pairs $(x,\,z)$ such that $x \ne z \in X.$ Consequently, summing over all $x,\,z \in X$ such that $x \ne z$ gives us

$$\begin{aligned} 2c({\left| X\right| } - 1) d \mathopen {}\left( X,\,y\right) \ge 2 d \mathopen {}\left( X\right) . \end{aligned}$$

Summing over $y \in Y$ proves the lemma. $\square$

Proof of Proposition 1

Let $G_1 \subset \cdots \subset G_k$ be the sets during Greedy. Fix $1 \le i \le k.$ Then $G_i$ is the current solution after ith iteration of Greedy.

Let O be the optimal solution. Write $A = O \cap G_i,\,C = O \setminus A,$ and $B = G_i \setminus A.$ Lemma 1 implies that

$$\begin{aligned} c({\left| A\right| } - 1) d \mathopen {}\left( A,\,C\right) \ge {\left| C\right| } d \mathopen {}\left( A\right) , \end{aligned}$$

which in turn implies

$$\begin{aligned} \begin{aligned} {\left| C\right| }i( d \mathopen {}\left( A\right) + d \mathopen {}\left( A,\,C\right) )&\le ci({\left| A\right| } - 1) d \mathopen {}\left( A,\,C\right) + {\left| C\right| }i d \mathopen {}\left( A,\,C\right) \\&= ci({\left| A\right| } - 1 + {\left| C\right| }) d \mathopen {}\left( A,\,C\right) \\&= ci(k - 1) d \mathopen {}\left( A,\,C\right) . \\ \end{aligned} \end{aligned}$$

Moreover, Lemma 1 implies that

$$\begin{aligned} \begin{aligned} c({\left| C\right| } - 1) d \mathopen {}\left( B,\,C\right)&\ge {\left| B\right| } d \mathopen {}\left( C\right) , \\ c({\left| C\right| } - 1) d \mathopen {}\left( A,\,C\right)&\ge {\left| A\right| } d \mathopen {}\left( C\right) , \\ \end{aligned} \end{aligned}$$

which, together with ${\left| C\right| } = {\left| B\right| } + k - i,$ implies

$$\begin{aligned} \begin{aligned} {\left| C\right| }i d \mathopen {}\left( C\right)&= (k - i)i d \mathopen {}\left( C\right) + {\left| B\right| }i d \mathopen {}\left( C\right) \\&= (k - i)({\left| A\right| } + {\left| B\right| }) d \mathopen {}\left( C\right) + {\left| B\right| }i d \mathopen {}\left( C\right) \\&= (k - i){\left| A\right| } d \mathopen {}\left( C\right) + {\left| B\right| }k d \mathopen {}\left( C\right) \\&\le c(k - i)({\left| C\right| } - 1) d \mathopen {}\left( A,\,C\right) + ck({\left| C\right| } - 1) d \mathopen {}\left( B,\,C\right) \\&\le c(k - i)(k - 1) d \mathopen {}\left( A,\, C\right) + ck(k - 1) d \mathopen {}\left( B,\,C\right) .\\ \end{aligned} \end{aligned}$$

Combining these two inequalities leads us to

$$\begin{aligned} \begin{aligned} {\left| C\right| }i d \mathopen {}\left( O\right)&= {\left| C\right| }i d \mathopen {}\left( A\right) + {\left| C\right| }i d \mathopen {}\left( C\right) + {\left| C\right| }i d \mathopen {}\left( A,\,C\right) \\&\le ck(k - 1)( d \mathopen {}\left( A,\,C\right) + d \mathopen {}\left( B,\, C\right) ) \\&= ck(k - 1)d\left( G_i,\,C\right) . \\ \end{aligned} \end{aligned}$$

Submodularity and monotonicity imply

$$\begin{aligned} \begin{aligned} \sum _{v \in C} g\left( v;\,G_i\right)&= \sum _{v \in C} \left[ h\left( v;\,G_i\right) + d\left( \{v\},\, G_i\right) \right] \\&= \left( \sum _{v \in C} h\left( v;\, G_i\right) \right) + d\left( C,\,G_i\right) \\&\ge \frac{1}{2}\left[ f(O) - f\left( G_i\right) \right] + \frac{i{\left| C\right| }}{ck(k - 1)} d \mathopen {}\left( O\right) \\&\ge \frac{1}{2}\left[ f(O) - f\left( G_k\right) \right] + \frac{i{\left| C\right| }}{ck(k - 1)} d \mathopen {}\left( O\right) . \end{aligned} \end{aligned}$$

Let $u_i$ be the item added at the $i + 1$th step, $G_{i + 1} = \left\{ u_i\right\} \cup G_i.$ Then, since $g(u_i;\,G_i) \ge \alpha g(v;\,G_i)$ for any $v \in C,$

$$\begin{aligned} g\left( u_i;\, G_i\right) \ge \frac{\alpha }{2k}\left[ f(O) - f\left( G_k\right) \right] + \frac{i\alpha }{ck(k - 1)} d \mathopen {}\left( O\right) . \end{aligned}$$

Summing over i gives us

$$\begin{aligned} \frac{1}{2}f\left( G_k\right) + d\left( G_k\right) = \sum _{i = 0}^{k - 1}g\left( u_i;\, G_i\right) \ge \frac{\alpha }{2}\left[ f(O) - f\left( G_k\right) \right] + \frac{\alpha }{2c} d \mathopen {}\left( O\right) . \end{aligned}$$

Since $\alpha \le 1$ and $c \ge 1,$ we have

$$\begin{aligned} r\left( G_k\right) = f\left( G_k\right) + d\left( G_k\right) \ge \frac{\alpha }{2}f(O) + \frac{\alpha }{2c} d \mathopen {}\left( O\right) \ge \frac{\alpha }{2c} r \mathopen {}\left( O\right) , \end{aligned}$$

which completes the proof. $\square$

Proof of Proposition 4

To prove the proposition we need to first show that Modify does not decrease the gain of a set significantly.

Lemma 2

Assume a graph $G = (V,\,E).$ Assume a collection of k distinct subgraphs $\mathcal {W}$ of G,  and let $U \in \mathcal {W}.$ Assume that $k < {\left| V\right| }$ and G contains more than k wedges, i.e., connected subgraphs of size 3. Let $M = \mathsf{{Modify}} (U,\, G,\, \mathcal {W},\, \lambda ).$ Then $\chi \mathopen {}\left( V;\,\mathcal {W}\right) \ge 2/5 \times ( \chi \mathopen {}\left( U,\,\mathcal {W}\right) +\lambda ).$

Proof

Write $r = {\left| U\right| }$ and $\alpha = \frac{r}{r + 1}.$ We will split the proof in two cases. Case 1 assume that X,  as given in Algorithm 3, is not empty. Select $B \in X.$ We will show that

$$\begin{aligned} \mathrm {dens} \mathopen {}\left( B\right) \ge \alpha \mathrm {dens} \mathopen {}\left( U\right) \quad \text {and}\quad D \mathopen {}\left( B,\,W\right) \ge \alpha ( D \mathopen {}\left( U,\, W\right) + I[U = W]), \end{aligned}$$

for any $W \in \mathcal {W},$ where $I[U = W] = 1$ if $U = W,$ and 0 otherwise. This automatically guarantees that

$$\begin{aligned} \chi \mathopen {}\left( B;\, \mathcal {W},\, \lambda \right) \ge \alpha ( \chi \mathopen {}\left( U;\, \mathcal {W},\, \lambda \right) + \lambda ), \end{aligned}$$

proving the result since $\alpha \ge 1/2$ and the gain of M is at least as good as the gain of B.

To prove the first inequality, note that

$$\begin{aligned} \mathrm {dens} \mathopen {}\left( B\right) = \frac{{\left| E(B)\right| }}{r + 1} \ge \frac{{\left| E(U)\right| }}{r + 1} = \alpha \frac{{\left| E(U)\right| }}{r} = \alpha \mathrm {dens} \mathopen {}\left( U\right) . \end{aligned}$$

To prove the second inequality fix $W \in \mathcal {W},$ and let $p = {\left| W\right| },\,q = {\left| W \cap U\right| }.$ Define

$$\begin{aligned} \varDelta = D \mathopen {}\left( U,\,W\right) + I[U = W] = 2 - \frac{q^2}{rp} = \frac{2rp - q^2}{rp}. \end{aligned}$$

Let v be the only vertex in $B \setminus U.$ If $v \notin W,$ then $D \mathopen {}\left( B,\,W\right) \ge \varDelta .$ Hence, we can assume that $v \in W.$ This leads to

$$\begin{aligned} \begin{aligned} D \mathopen {}\left( B,\, W\right)&= 2 - \frac{{\left| B \cap W\right| }^2}{{\left| B\right| }{\left| W\right| }} \\&= 2 - \frac{ (1 + q)^2}{(1 + r)p} = \frac{2p(1 + r) - (1 + q)^2}{(1 + r)p}. \end{aligned} \end{aligned}$$

Let us define $\beta$ as the fraction of the numerators,

$$\begin{aligned} \beta = \frac{2p(1 + r) - (1 + q)^2}{2rp - q^2}. \end{aligned}$$

We wish to show that $\beta \ge 1.$ Since $p \ge q + 1,$

$$\begin{aligned} \begin{aligned} \beta&= \frac{2p(1 + r) - (1 + q)^2}{2rp - q^2} = \frac{2rp - q^2 + 2p - 2q - 1}{2rp - q^2} \\&\ge \frac{2rp - q^2 + 2(q + 1) - 2q - 1}{2rp - q^2} = \frac{2rp - q^2 + 1}{2rp - q^2} \ge 1. \end{aligned} \end{aligned}$$

The ratio of distances is now

$$\begin{aligned} \frac{ D \mathopen {}\left( B,\,W\right) }{\varDelta } = \beta \frac{r}{r + 1} \ge \frac{r}{r + 1} = \alpha . \end{aligned}$$

This proves the first case.

Case 2 assume that $X = \emptyset .$ Then we must have $Y \ne \emptyset$ and $r \ge 2,$ as otherwise ${\left| \mathcal {W}\right| } \ge {\left| V\right| },$ which violates the assumption of the lemma.

Assume that $\mathrm {dens} \mathopen {}\left( U\right) \ge 5/3.$ Let $B \in Y.$ Removing a single item of U decreases the density by 1, at most. This gives us

$$\begin{aligned} \frac{ \mathrm {dens} \mathopen {}\left( B\right) }{ \mathrm {dens} \mathopen {}\left( U\right) } \ge \frac{ \mathrm {dens} \mathopen {}\left( U\right) - 1}{ \mathrm {dens} \mathopen {}\left( U\right) } \ge \frac{5/3 - 1}{5 / 3} = \frac{2}{5}. \end{aligned}$$

To bound the distance term, fix $W \in \mathcal {W},$ and let $p = {\left| W\right| },\,q = {\left| W \cap U\right| }.$ Let v be the only vertex in $U \setminus V.$ Define $\varDelta = D \mathopen {}\left( U,\,W\right) + I[U = W].$ If $v \in W,$ then we can easily show that $D \mathopen {}\left( V,\,W\right) \ge \varDelta .$ Hence, assume that $v \notin W.$ This implies that $q \le \min p,\,r - 1,$ or $q^2 \le p(r - 1).$ As before, we can express the distance term as

$$\begin{aligned} \varDelta = 2 - \frac{q^2}{rp} = \frac{2rp - q^2}{rp}, \end{aligned}$$

and

$$\begin{aligned} \begin{aligned} D \mathopen {}\left( B,\, W\right)&= 2 - \frac{{\left| B \cap W\right| }^2}{{\left| B\right| }{\left| W\right| }} = 2 - \frac{ q^2}{(r - 1)p} = \frac{2p(r - 1) - q^2}{(r - 1)p}. \end{aligned} \end{aligned}$$

The ratio is then

$$\begin{aligned} \begin{aligned} \frac{ D \mathopen {}\left( B,\, W\right) }{\varDelta }&= \frac{2p(r - 1) - q^2}{2rp - q^2}\frac{r}{r - 1} \\&\ge \frac{2p(r - 1) - p(r - 1)}{2rp - p(r - 1)}\frac{r}{r - 1} = \frac{p(r - 1)}{rp + p}\frac{r}{r - 1} = \frac{r}{r + 1} \ge 1/2, \end{aligned} \end{aligned}$$

where the first inequality follows from the fact that the ratio is decreasing as function of q.

Assume that $\mathrm {dens} \mathopen {}\left( U\right) < 5/ 3.$ By assumption there is a wedge B outside $\mathcal {W}.$ Since $\mathrm {dens} \mathopen {}\left( B\right) \ge 2/3,$ we have $\mathrm {dens} \mathopen {}\left( B\right) / \mathrm {dens} \mathopen {}\left( U\right) \ge 2 / 5.$ The distance terms decrease by a factor of 1/2, since

$$\begin{aligned} D \mathopen {}\left( U,\, W\right) \le 2 = 2 \times 1 \le 2 D \mathopen {}\left( B,\, W\right) . \end{aligned}$$

Combining the inequalities proves that

$$\begin{aligned} \chi \mathopen {}\left( B;\, \mathcal {W},\, \lambda \right) \ge \frac{2}{5} \chi \mathopen {}\left( U;\, \mathcal {W},\, \lambda \right) , \end{aligned}$$

which proves the lemma. $\square$

Proof of Proposition 4

To prove the proposition, we will first form a new graph H,  and show that the density of a subgraph in H is closely related to the gain. This then allows us to prove the statement.

Let us first construct the graph $H{\text {:}}$ given a vertex v let us define

$$\begin{aligned} s(v) = {-}\sum _{v \in W_j} \frac{2\lambda }{{\left| W_j\right| }}. \end{aligned}$$

Let $H = (V,\, E^{\prime },\, c)$ be a fully connected weighted graph with self-loops where the weight of an edge $c(v,\, w)$ is

$$\begin{aligned} c(v,\, w) = I[(v,\, w) \in E] - \sum _{j \mid v,\, w \in W_j} \frac{4 \lambda }{{\left| W_j\right| }}, \end{aligned}$$

for $v \ne w,$ and $c(v,\, v) = s(v).$

Next, we connect the gain of set of vertices U (w.r.t. G) with the weighted density of U in H. Given an arbitrary set of vertices U,  we will write c(U) to mean the total weight of edges in H. Each $c(v,\, w),$ for $v \in w,$ participates in $\deg _{H}(v;\, U)$ and $\deg _{H}(w;\, U),$ and each $c(v,\, v) = s(v)$ participates (once) in $\deg _{H}(v;\, U).$ This leads to

$$\begin{aligned} 2c(U) = \sum _{v \in U} \deg _{H}(v;\, U) + s(v). \end{aligned}$$

We can express the (weighted) degree of a vertex in H as

$$\begin{aligned} \begin{aligned} \deg _{H}(v;\, U)&= s(v) + \sum _{\begin{array}{c} w \in U \\ w \ne v \end{array}} c(v,\, w) = \deg _G (v ;\, U) -\sum _{j \mid v \in W_j} \frac{2\lambda }{{\left| W_j\right| }} \\&\quad - \sum _{\begin{array}{c} w \in U \\ w \ne v \end{array}} \sum _{j \mid v,\, w \in W_j} \frac{4 \lambda }{{\left| W_j\right| }} \\&= \deg _G (v ;\, U) - \lambda \sum _{j \mid v \in W_j} \frac{4{\left| U \cap W_j\right| } - 2}{{\left| W_j\right| }}. \end{aligned} \end{aligned}$$

(3)

Write $k = {\left| \mathcal {W}\right| }.$ These equalities lead to the following identity,

$$\begin{aligned} \begin{aligned} \mathrm {dens} \mathopen {}\left( U;\,H\right) + 4\lambda k&= \frac{1}{{\left| U\right| }}c(U) + 4\lambda k \\&= 4\lambda k + \frac{1}{2{\left| U\right| }} \sum _{v \in U} \deg _{H}(v;\, U) + s(v) \\&= 4\lambda k + \mathrm {dens} \mathopen {}\left( U;\, G\right) - \frac{1}{2{\left| U\right| }} \sum _{v \in U} \lambda \sum _{j \mid v \in W_j} \frac{4{\left| U \cap W_j\right| }}{{\left| W_j\right| }} \\&= \mathrm {dens} \mathopen {}\left( U;\, G\right) - 2\lambda \sum _{j = 1}^k2 - \frac{{\left| U \cap W_j\right| }^2}{{\left| U\right| }{\left| W_j\right| }} \\&= 2 \chi \mathopen {}\left( U;\, \mathcal {W},\, \lambda \right) + \epsilon (U,\, \mathcal {W}), \\ \end{aligned} \end{aligned}$$

(4)

where $\epsilon (U,\, \mathcal {W})$ is a correction term, equal to 2$\lambda$ if $U \in \mathcal {W},$ and 0 otherwise.

Let O be the densest subgraph in H. Next we show that during the for-loop Peel finds a graph whose density close to $\mathrm {dens} \mathopen {}\left( O;\, H\right) .$ Let o be the first vertex in O deleted by Peel. We must have

$$\begin{aligned} \deg _H(o;\, O) \ge \mathrm {dens} \mathopen {}\left( O;\, H\right) , \end{aligned}$$

as otherwise we can delete o from O and obtain a better solution. Let $R = V_i$ be the graph at the moment when o is about to be removed. Let us compare $\deg _H(o;\,O)$ and $\deg _H(o;\, R).$ We can lower-bound of the second term of the right-hand side in Eq. (3) by ${-}4k\lambda - s(v).$ Since $O \subseteq R,$ this gives us

$$\begin{aligned} \begin{aligned} \deg _H(o;\, O)&\le \deg _G(o;\, O) \le \deg _G(o;\, R) \\&\le \deg _H(o;\, R) + s(o) + 4k\lambda . \end{aligned} \end{aligned}$$

To upper-bound the first two terms, note that by definition of Peel, the vertex o has the smallest $\deg _H(o;\, R) + s(o)$ among all the vertices in R. Hence,

$$\begin{aligned} \deg _H(o;\, R) + s(o) \le \sum _{v \in R} \frac{\deg _H(v;\, R) + s(v)}{{\left| R\right| }} = 2\frac{c(R)}{{\left| R\right| }} = 2 \mathrm {dens} \mathopen {}\left( R;\, H\right) . \end{aligned}$$

To complete the proof, let $O^{\prime }$ be the graph outside $\mathcal {W},$ maximizing the gain. Due to Eq. (4), we have

$$\begin{aligned} \begin{aligned} 2 \chi \mathopen {}\left( O^{\prime };\, \mathcal {W},\, \lambda \right)&= \mathrm {dens} \mathopen {}\left( O^{\prime };\, H\right) + 4k\lambda \le \mathrm {dens} \mathopen {}\left( O;\, H\right) + 4k\lambda \\&\le 2 \mathrm {dens} \mathopen {}\left( R;\, H\right) + 8k\lambda = 2( \mathrm {dens} \mathopen {}\left( R;\, H\right) + 4k\lambda ) \\&= 4 \chi \mathopen {}\left( R;\, \mathcal {W},\, \lambda \right) + 2\epsilon (R;\, \mathcal {W}). \end{aligned} \end{aligned}$$

Let S be the set returned by Peel.

If $R \notin \mathcal {W},$ then $\epsilon (R;\, \mathcal {W}) = 0.$ Moreover, R is not modified, and is one of the graphs that is tested for gain. Consequently, $\chi \mathopen {}\left( S;\, \mathcal {W}\right) \ge \chi \mathopen {}\left( R;\, \mathcal {W}\right) ,$ proving the statement.

If $R \in \mathcal {W},$ then it is modified by Modify to, say, $R^{\prime }.$ Lemma 2 implies that $5/2 \times \chi \mathopen {}\left( R^{\prime },\, \mathcal {W}\right) \ge \chi \mathopen {}\left( R;\, \mathcal {W}\right) + \epsilon (R;\, \mathcal {W}).$ Since, $\chi \mathopen {}\left( S;\, \mathcal {W}\right) \ge \chi \mathopen {}\left( R^{\prime };\, \mathcal {W}\right) ,$ this completes the proof. $\square$