Spectral clustering based on similarity and dissimilarity criterion

Abstract

The clustering assumption is to maximize the within-cluster similarity and simultaneously to minimize the between-cluster similarity for a given unlabeled dataset. This paper deals with a new spectral clustering algorithm based on a similarity and dissimilarity criterion by incorporating a dissimilarity criterion into the normalized cut criterion. The within-cluster similarity and the between-cluster dissimilarity can be enhanced to result in good clustering performance. Experimental results on toy and real-world datasets show that the new spectral clustering algorithm has a promising performance.

Appendix A

Proof:

To prove Theorem 1, we need to prove that the weight of the between-cluster similarity in (1 − m)y ^T Dy − m y ^T Qy is less than that in y ^T Dy. Thus, the between-cluster similarity would have less effect on the maximization of the within-cluster similarity.

Without loss of generality, assume that a graph G can be partitioned into two disjoint sub-graphs X ₁ and X ₂. Consider the continuous and loose form of the indicator vector y, and let y _i  ∊ {1, −b} with $b = \frac{{\sum\nolimits_{{z_{i} > 0}} {D_{i} } }}{{\sum\nolimits_{{z_{i} < 0}} {D_{ii} } }}$ and indicator vector z = [z ₁,…, z _n ]. First, we expand the denominator of the objective in (4) and simplify it, and have

$$\begin{aligned} {\mathbf{y}}^{T} {\mathbf{Dy}} = \left( {\sum\limits_{{\user2{\rm x}_{i} \in X_{1} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{1} }} {W_{ij} } } + b^{2} \sum\limits_{{\user2{\rm x}_{i} \in X_{2} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{2} }} {W_{ij} } } } \right) \hfill \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} + \left( {\sum\limits_{{\user2{\rm x}_{i} \in X_{1} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{2} }} {W_{ij} } } + b^{2} \sum\limits_{{\user2{\rm x}_{i} \in X_{2} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{1} }} {W_{ij} } } } \right) \hfill \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} = Sim_{w} + (1 + b^{2} )Sim_{b} , \hfill \\ \end{aligned}$$

(15)

where Sim _w is the sum of the first two terms in (15) which can be viewed as the within-cluster similarity, and $Sim_{b} = \sum\nolimits_{{x_{i} \in X_{2} }} {\sum\nolimits_{{x_{j} \in X_{1} }} {W_{ij} } }$ denotes the between-cluster similarity. Next, we expand the denominator of the objective in (12), and have

$$\begin{aligned} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} (1 - m){\mathbf{y}}^{T} {\mathbf{Dy}} - m{\mathbf{y}}^{T} {\mathbf{Qy}} \hfill \\ = (1 - m)\left( {\sum\limits_{{\user2{\rm x}_{i} \in X_{1} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{1} }} {W_{ij} } } } \right) \hfill \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} + (1 - m)\left( {b^{2} \sum\limits_{{\user2{\rm x}_{i} \in X_{2} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{2} }} {W_{ij} } } + (1 + b^{2} )\sum\limits_{{\user2{\rm x}_{i} \in X_{2} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{1} }} {W_{ij} } } } \right) \hfill \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} - m\left( {\sum\limits_{{\user2{\rm x}_{i} \in X_{1} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{1} }} {Q_{ij} } } + b^{2} \sum\limits_{{\user2{\rm x}_{i} \in X_{2} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{2} }} {Q_{ij} } } - 2b\sum\limits_{{\user2{\rm x}_{i} \in X_{1} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{2} }} {Q_{ij} } } } \right) \hfill \\ \end{aligned}$$

(16)

Since Q _ij  = 1 − W _ij , we substitute it into (16) and get

$$\begin{aligned} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} (1 - m){\mathbf{y}}^{T} {\mathbf{Dy}} - m{\mathbf{y}}^{T} {\mathbf{Dy}} \hfill \\ = \left( {\sum\limits_{{\user2{\rm x}_{i} \in X_{1} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{1} }} {W_{ij} } } + b^{2} \sum\limits_{{\user2{\rm x}_{i} \in X_{2} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{2} }} {W_{ij} } } } \right) \hfill \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} + \left( {\left( {\left( {1 + b^{2} } \right) - m(1 + b^{2} )} \right)\sum\limits_{{\user2{\rm x}_{i} \in X_{1} }} {\sum\limits_{{\user2{\rm x}_{j} \in X_{2} }} {W_{ij} } } } \right) \hfill \\ = Sim_{w} + \left( {\left( {1 + b^{2} } \right) - m\left( {1 + b^{2} } \right)} \right)Sim_{b} \hfill \\ \end{aligned}$$

(17)

Comparing (15) with (17), we can see that the difference between them is the weight of the between-cluster similarity. In addition, it is obvious that the following inequality

$$\left( {\left( {1 + b^{2} } \right) - m(1 + b^{2} )} \right) \le (1 + b^{2} )$$

(18)

holds true. If and only if m = 0, the inequality becomes equality.Thus, the between-cluster similarity in the denominator of (12) has a smaller weight than that of (4).

This completes the proof of Theorem 1.