Advertisement

International Journal of Fuzzy Systems

, Volume 21, Issue 8, pp 2632–2649 | Cite as

A Novel Fuzzy c-Means Clustering Algorithm Using Adaptive Norm

  • Yunlong Gao
  • Dexin Wang
  • Jinyan PanEmail author
  • Zhihao Wang
  • Baihua Chen
Article
  • 24 Downloads

Abstract

The fuzzy c-means (FCM) clustering algorithm is an unsupervised learning method that has been widely applied to cluster unlabeled data automatically instead of artificially, but is sensitive to noisy observations due to its inappropriate treatment of noise in the data. In this paper, a novel method considering noise intelligently based on the existing FCM approach, called adaptive-FCM and its extended version (adaptive-REFCM) in combination with relative entropy, are proposed. Adaptive-FCM, relying on an inventive integration of the adaptive norm, benefits from a robust overall structure. Adaptive-REFCM further integrates the properties of the relative entropy and normalized distance to preserve the global details of the dataset. Several experiments are carried out, including noisy or noise-free University of California Irvine (UCI) clustering and image segmentation experiments. The results show that adaptive-REFCM exhibits better noise robustness and adaptive adjustment in comparison with relevant state-of-the-art FCM methods.

Keywords

Fuzzy c-means clustering Adaptive norm Noise robustness Relative entropy 

Notes

Acknowledgements

This study was supported by the National Natural Science Foundation of China (61203176) and the Natural Science Foundation of Fujian Province (2013J05098, 2016J01756).

References

  1. 1.
    Bock, H.H.: Origins and extensions of the k-means algorithm in cluster analysis. Elect. J. 4, 2 (2008)MathSciNetGoogle Scholar
  2. 2.
    Zadeh, L.A.: Fuzzy logic = computing with words. Physica-Verlag, Heidelberg (1999)CrossRefGoogle Scholar
  3. 3.
    Bezdek, J.C., Ehrlich, R., Full, W.: FCM: the fuzzy c-means clustering algorithm. Comput. Geosci. 10(2–3), 191–203 (1984)CrossRefGoogle Scholar
  4. 4.
    Zadeh, L.A.: Fuzzy sets. Inf. Control 8(3), 338–353 (1965).  https://doi.org/10.1016/S0019-9958(65)90241-X CrossRefzbMATHGoogle Scholar
  5. 5.
    Dunn, J.C.: A fuzzy relative of the isodata process and its use in detecting compact well-separated clusters. J. Cybern. 3(3), 32–57 (1973).  https://doi.org/10.1080/01969727308546046 MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Jing, G., Jiao, L., Yang, S., Fang, L.: Fuzzy double c-means clustering based on sparse self-representation. IEEE Trans. Fuzzy Syst. 99, 1–1 (2018)Google Scholar
  7. 7.
    Keller, A., Klawonn, F.: Fuzzy clustering with weighting of data variables. Int. J. Uncertain. Fuzziness Knowl. Based Syst. 8(06), 735–746 (2000)CrossRefGoogle Scholar
  8. 8.
    Le, H.S., Tien, N.D.: Tune up fuzzy c-means for big data: some novel hybrid clustering algorithms based on initial selection and incremental clustering. Int. J. Fuzzy Syst. 19(5), 1–18 (2016).  https://doi.org/10.1007/s40815-016-0260-3 MathSciNetCrossRefGoogle Scholar
  9. 9.
    Hu, Z., Bodyanskiy, Y.V., Tyshchenko, O.K., Samitova, V.O.: Fuzzy clustering data given on the ordinal scale based on membership and likelihood functions sharing. Int. J. Intell. Syst. Appl. 9(2), 1–9 (2017)Google Scholar
  10. 10.
    Raja, S., Ramaiah, S.: An efficient fuzzy-based hybrid system to cloud intrusion detection. Int. J. Fuzzy Syst. 19(1), 62–77 (2017).  https://doi.org/10.1007/s40815-016-0147-3 CrossRefGoogle Scholar
  11. 11.
    Zhao, X., Yu, L., Zhao, Q.: A fuzzy clustering approach for complex color image segmentation based on gaussian model with interactions between color planes and mixture gaussian model. Int. J. Fuzzy Syst. 20(1), 309–317 (2018).  https://doi.org/10.1007/s40815-017-0411-1 MathSciNetCrossRefGoogle Scholar
  12. 12.
    Davarpanah, S.H., Liew, W.C.: Spatial possibilistic fuzzy c-mean segmentation algorithm integrated with brain mid-sagittal surface information. Int. J. Fuzzy Syst. 19(2), 1–15 (2017).  https://doi.org/10.1007/s40815-016-0247-0 MathSciNetCrossRefGoogle Scholar
  13. 13.
    Hung, C.C., Kulkarni, S., Kuo, B.C.: A new weighted fuzzy c-means clustering algorithm for remotely sensed image classification. IEEE J. Select. Topics Signal Process. 5(3), 543–553 (2011)CrossRefGoogle Scholar
  14. 14.
    Zhou, J., Chen, L., Chen, C.L.P., Zhang, Y.H., Li, H.X.: Fuzzy clustering with the entropy of attribute weights. Neurocomputing 198, 125–134 (2016).  https://doi.org/10.1016/j.neucom.2015.09.127 CrossRefGoogle Scholar
  15. 15.
    Kroger, P.: Outlier detection techniques. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining (2010)Google Scholar
  16. 16.
    Chang, X., Wang, Q., Liu, Y., Wang, Y.: Sparse regularization in fuzzy \(c\)-means for high-dimensional data clustering. IEEE Trans. Cybern. 47(9), 2616–2627 (2017).  https://doi.org/10.1109/TCYB.2016.2627686 CrossRefGoogle Scholar
  17. 17.
    Hamasuna, Y., Endo, Y., Miyamoto, S.: Comparison of tolerant fuzzy c-means clustering with \(l_{1}\) and \(l_{2}\) regularization. In: IEEE international conference on granular computing, pp. 197–202 (2009)Google Scholar
  18. 18.
    Yun-Xia, Y.U., Wang, S.T., Zhu, W.P.: On fuzzy c-means for data with tolerance. Comput. Eng. Des. 31(3), 612–615 (2010)Google Scholar
  19. 19.
    Rubio, E., Castillo, O.: Designing type-2 fuzzy systems using the interval type-2 fuzzy c-means algorithm. Stud. Comput. Intell. (2014).  https://doi.org/10.1007/978-3-319-05170-3_3 CrossRefGoogle Scholar
  20. 20.
    Yu, S.M., Wang, J., Wang, J.Q.: An interval type-2 fuzzy likelihood-based mabac approach and its application in selecting hotels on a tourism website. Int. J. Fuzzy Syst. 19(1), 47–61 (2017).  https://doi.org/10.1007/s40815-016-0217-6 MathSciNetCrossRefGoogle Scholar
  21. 21.
    Vu, M.N., Long, T.N.: A multiple kernels interval type-2 possibilistic c-means. Stud. Comput. Intell. (2016).  https://doi.org/10.1007/978-3-319-31277-4_6 CrossRefGoogle Scholar
  22. 22.
    Miyamoto, S.: Multisets and fuzzy multisets. Springer, Berlin (2000)CrossRefGoogle Scholar
  23. 23.
    Atanassov, K.T.: Intuitionistic fuzzy sets. Fuzzy Sets Syst. 20(1), 87–96 (1986).  https://doi.org/10.1016/S0165-0114(86)80034-3 CrossRefzbMATHGoogle Scholar
  24. 24.
    Liao, H., Xu, Z., Herrera-Viedma, E., Herrera, F.: Hesitant fuzzy linguistic term set and its application in decision making: a state-of-the-art survey. Int. J. Fuzzy Syst. 20(12), 1–27 (2017).  https://doi.org/10.1007/s40815-017-0432-9 MathSciNetCrossRefGoogle Scholar
  25. 25.
    Torra, V.: Hesitant fuzzy sets. Int. J. Intell. Syst. 25(6), 529–539 (2010).  https://doi.org/10.1002/int.20418 CrossRefzbMATHGoogle Scholar
  26. 26.
    Wang, J., Wang, J.Q., Zhang, H.Y., Chen, X.H.: Multi-criteria group decision-making approach based on 2-tuple linguistic aggregation operators with multi-hesitant fuzzy linguistic information. Int. J. Fuzzy Syst. 18(1), 81–97 (2016).  https://doi.org/10.1007/s40815-015-0050-3 MathSciNetCrossRefGoogle Scholar
  27. 27.
    Wen, F., Liu, P., Liu, Y., Qiu, R.C., Yu, W.: Robust sparse recovery for compressive sensing in impulsive noise using p-norm model fitting. In: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp. 4643–4647 (2016)Google Scholar
  28. 28.
    Tang, M., Nie, F., Jain, R.: Capped lp-norm graph embedding for photo clustering. In: Proceedings of the 24th ACM international conference on Multimedia, pp. 431–435. ACM (2016)Google Scholar
  29. 29.
    Ding, C.: A new robust function that smoothly interpolates between l1 and l2 error functions. Univerisity of Texas at Arlington Tech ReportGoogle Scholar
  30. 30.
    Nie, F., Wang, H., Huang, H., Ding, C.: Adaptive loss minimization for semi-supervised elastic embedding. In: International joint conference on artificial intelligence, pp. 1565–1571 (2013)Google Scholar
  31. 31.
    Krishnapuram, R., Keller, J.M.: A possibilistic approach to clustering. IEEE Trans. Fuzzy Syst. 1(2), 98–110 (1993)CrossRefGoogle Scholar
  32. 32.
    Zarinbal, M., Zarandi, M.H.F., Turksen, I.B.: Relative entropy fuzzy c-means clustering. Inf. Sci. 260(1), 74–97 (2014).  https://doi.org/10.1016/j.ins.2013.11.004 MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Gustafson, D.E., Kessel, W.C.: Fuzzy clustering with a fuzzy covariance matrix. In: 1978 IEEE conference on decision and control including the symposium on adaptive processes, pp. 761–766 (2007).  https://doi.org/10.1109/CDC.1978.268028
  34. 34.
    Tibshirani, R.: Regression shrinkage and selection via the lasso: a retrospective. J. R. Stat. Soc. Series B Stat. Methodol. 73(3), 273–282 (2011).  https://doi.org/10.1111/j.1467-9868.2011.00771.x MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Hui, Z., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. 67(5), 768–768 (2010).  https://doi.org/10.1111/j.1467-9868.2005.00527.x CrossRefGoogle Scholar
  36. 36.
    Liu, H.C., Jeng, B.C., Yih, J.M., Yu, Y.K.: Fuzzy c-means algorithm based on standard Mahalanobis distances. Proc. Int. Symp. Inf. Process 15, 581–595 (2009)Google Scholar
  37. 37.
    Zhao, X., Li, Y., Zhao, Q.: Mahalanobis distance based on fuzzy clustering algorithm for image segmentation. Digit. Signal Process. 43, 8–16 (2015)CrossRefGoogle Scholar
  38. 38.
    Corless, R.M., Gonnet, G.H., Knuth, D.: On the Lambertw function. In: Advances in computational mathematics, p. 329–359 (1996)  https://doi.org/10.1007/BF02124750 MathSciNetCrossRefGoogle Scholar
  39. 39.
    Pal, N.R., Bezdek, J.C.: Correction to on cluster validity for the fuzzy c-means model (1997)Google Scholar
  40. 40.
    Bezdek, J.C.: A physical interpretation of fuzzy isodata. IEEE Trans. Syst. Man Cybern. 6(5), 387–389 (2007)MathSciNetzbMATHGoogle Scholar
  41. 41.
    Dheeru, D., Karra Taniskidou, E.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml
  42. 42.
    Rand, W.M.: Objective criteria for the evaluation of clustering methods. Publ. Am. Stat. Assoc. 66(336), 846–850 (1971)CrossRefGoogle Scholar
  43. 43.
    Luo, M., Nie, F., Chang, X., Yang, Y., Hauptmann, A.G., Zheng, Q.: Adaptive unsupervised feature selection with structure regularization. IEEE Trans. Neural Netw. Learn. Syst. 29(4), 944–956 (2018)CrossRefGoogle Scholar
  44. 44.
    Wen, Z., Liu, X., Chen, Y., Wu, W., Wei, W., Li, X.: Feature-derived graph regularized matrix factorization for predicting drug side effects. Neurocomputing 287, 154 (2018)CrossRefGoogle Scholar

Copyright information

© Taiwan Fuzzy Systems Association 2019

Authors and Affiliations

  • Yunlong Gao
    • 1
  • Dexin Wang
    • 1
  • Jinyan Pan
    • 2
    Email author
  • Zhihao Wang
    • 1
  • Baihua Chen
    • 1
  1. 1.Department of AutomationXiamen UniversityXiamenChina
  2. 2.College of Information EngineerJimei UniversityXiamenChina

Personalised recommendations