Generalizing Distance Functions for Fuzzy c-Means Clustering

Abstract

Fuzzy $c$-means (FCM) is a well-known partitional clustering method, which allows an object to belong to two or more clusters with a membership grade between zero and one. Recently, due to the rich information conveyed by the membership grade matrix, FCM has been widely used in many real-world application domains where well-separated clusters are typically not available. In addition, people also recognize that the simple centroid-based iterative procedure of FCM is very appealing when dealing with large volume data.

Appendix

As previously mentioned in Sect. 3.4, the degeneration of $\varvec{U}$ may happen for GD-FCM using $f^\phi _{II}$, although the probability of occurrence is extremely low for real-world data. Here, we provide a solution for this degeneration case and show how the global convergence of GD-FCM can still be guaranteed.

During a GD-FCM iteration, assume that we get $(\varvec{U}^{de},\varvec{ v}^{de})=T_{mf}(\bar{\varvec{U}},\bar{\varvec{ V}})$, where $\varvec{U}^{de}$ is degenerate but $\bar{\varvec{U}}$ is nondegenerate. Without loss of generality, suppose there is only one $r$ such that $u^{de}_{rk}=0~\forall ~k$. Then, we let $\hat{\varvec{ v}}=(\hat{\varvec{ v}}_1,\hat{\varvec{ v}}_2,\ldots ,\hat{\varvec{ v}}_c)^T$ be

$$\begin{aligned} \left\{ \begin{array}{l} \hat{\varvec{{ v}}}_i=\varvec{ v}^{de}_{i},~\forall ~i\ne r,~{\mathrm and }\\ \hat{\varvec{ v}}_r=\varvec{x}\in {\mathrm Conv }(\fancyscript{X}),~\varvec{x}\ne \varvec{ v}^{de}_{r}. \end{array}\right. \end{aligned}$$

(3.44)

Next, we try to resume the iteration by having $\hat{\varvec{U}}=F(\hat{\varvec{V}})$. If $\hat{\varvec{U}}$ is nondegenerate, then we define $(\hat{\varvec{U}},\hat{\varvec{V}})\doteq T_{mf}(\bar{\varvec{U}},\bar{\varvec{V}})$, and resume the iteration based on $(\hat{\varvec{U}},\hat{\varvec{V}})$. Otherwise, we repeat choosing a new $\varvec{x}\in {\mathrm Conv }(\fancyscript{X})$ for $\hat{\varvec{v}}_r$ until we have a nondegenerate $\hat{\varvec{U}}=F(\hat{\varvec{V}})$. Typically, we choose $\varvec{x}$ from $\fancyscript{X}$ to ensure that $\exists ~k$ such that $u^{de}_{rk}\ne 0$.

Now, we establish the descent theorem when there is degeneration. Assume that $(\bar{\varvec{U}},\bar{\varvec{V}})$ is not in $\Omega ^{\prime }$ of Eq. (3.37). Since $u^{de}_{rk}=0~\forall ~k$, we have $J_{mf}(\varvec{U}^{de},\varvec{V}^{de})= J_{mf}(\varvec{U}^{de},\hat{\varvec{V}})\ge J_{mf}(\hat{\varvec{U}},\hat{\varvec{V}})$. Furthermore, by Theorem 3.6, we have $J_{mf}(\bar{\varvec{U}},\bar{\varvec{V}})>J_{mf}(\varvec{U}^{de},\varvec{V}^{de})$, which implies that $J_{mf}(\bar{\varvec{U}},\bar{\varvec{V}})>J_{mf}(\hat{\varvec{U}},\hat{\varvec{V}})$. The descent theorem therefore holds. In addition, since we can “skip” the degenerate solution by jumping from $(\bar{\varvec{U}},\bar{\varvec{V}})$ to $(\hat{\varvec{U}},\hat{\varvec{V}})$, we still have $T_{mf}:M_{fc}\times \fancyscript{S}^c\mapsto M_{fc}\times \fancyscript{S}^c$, and the closeness of $T_{mf}$ and the compactness of $M_{fc}\times {\mathrm Conv }(\fancyscript{X})^c$ still hold. By assembling the above results, we again get the global convergence by Zangwill’s convergence theorem.

Finally, in case that we cannot find any $\varvec{x}\in {\mathrm Conv }(\fancyscript{X})$ for $\hat{\varvec{ v}}_r$ such that $\hat{\varvec{U}}=F(\hat{\varvec{ V}})$ is nondegenerate, we simply return $(\bar{\varvec{U}},\bar{\varvec{ V}})$ as the solution.