Advertisement

Fuzzy Clustering

  • Rudolf Kruse
  • Christian Borgelt
  • Frank Klawonn
  • Christian Moewes
  • Matthias Steinbrecher
  • Pascal Held
Part of the Texts in Computer Science book series (TCS)

Abstract

After a brief overview of fuzzy methods in data analysis, this chapter focuses on fuzzy cluster analysis as the oldest fuzzy approach to data analysis. Fuzzy clustering comprises a family of prototype-based clustering methods that can be formulated as the problem of minimizing an objective function. These methods can be seen as “fuzzifications” of, for example, the classical c-means algorithm, which strives to minimize the sum of the (squared) distances between the data points and the cluster centers to which they are assigned. However, in order to “fuzzify” such a crisp clustering approach, it is not enough to merely allow values from the unit interval for the variables encoding the assignments of the data points to the clusters: the minimum is still obtained for a crisp data point assignment. As a consequence, additional means have to be employed in the objective function in order to obtain actual degrees of membership. This chapter surveys the most common fuzzification means and examines and compares their properties.

Keywords

Objective Function Cluster Center Fuzzy Cluster Membership Degree Fuzzy Data 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. G.H. Ball and D.J. Hall. A Clustering Technique for Summarizing Multivariate Data. Behavioral Science 12(2):153–155. J. Wiley & Sons, Chichester, United Kingdom, 1967 CrossRefGoogle Scholar
  2. H. Bandemer and W. Näther. Fuzzy Data Analysis. Kluwer, Dordrecht, Netherlands, 1992 MATHCrossRefGoogle Scholar
  3. J.C. Bezdek. Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York, NY, USA, 1981 MATHCrossRefGoogle Scholar
  4. J.C. Bezdek and R.J. Hathaway. Visual Cluster Validity (VCV) Displays for Prototype Generator Clustering Methods. Proc. 12th IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE 2003, Saint Louis, MO), 2:875–880. IEEE Press, Piscataway, NJ, USA, 2003 CrossRefGoogle Scholar
  5. J.C. Bezdek and N. Pal. Fuzzy Models for Pattern Recognition. IEEE Press, New York, NY, USA, 1992 Google Scholar
  6. J.C. Bezdek, J.M. Keller, R. Krishnapuram, and N. Pal. Fuzzy Models and Algorithms for Pattern Recognition and Image Processing. Kluwer, Dordrecht, Netherlands, 1999 MATHGoogle Scholar
  7. J. Bilmes. A Gentle Tutorial on the EM Algorithm and Its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models. Tech. Report ICSI-TR-97-021. University of Berkeley, CA, USA, 1997 Google Scholar
  8. A. Blanco-Fernández, M.R. Casals, A. Colubi, R. Coppi, N. Corral, S. Rosa de Sáa, P. D’Urso, M.B. Ferraro, M. García-Bárzana, M.A. Gil, P. Giordani, G. González-Rodríguez, M.T. López, M.A. Lubiano, M. Montenegro, T. Nakama, A.B. Ramos-Guajardo, B. Sinova, and W. Trutschnig. Arithmetic and Distance-Based Approach to the Statistical Analysis of Imprecisely Valued Data. In: Borgelt et al. (2013), 1–18 Google Scholar
  9. C. Borgelt. Prototype-based Classification and Clustering. Habilitationsschrift, Otto-von-Guericke-University of Magdeburg, Germany, 2005 Google Scholar
  10. C. Borgelt, M.A. Gil, J.M.C. Sousa, and M. Verleysen (eds.). Towards Advanced Data Analysis by Combining Soft Computing and Statistics. Studies in Fuzziness and Soft Computing, vol. 285. Springer-Verlag, Berlin/Heidelberg, Germany, 2013 MATHGoogle Scholar
  11. N. Boujemaa. Generalized Competitive Clustering for Image Segmentation. Proc. 19th Int. Meeting North American Fuzzy Information Processing Society (NAFIPS 2000, Atlanta, GA), 133–137. IEEE Press, Piscataway, NJ, USA, 2000 Google Scholar
  12. Z. Daróczy. Generalized Information Functions. Information and Control 16(1):36–51. Academic Press, San Diego, CA, USA, 1970 MathSciNetMATHCrossRefGoogle Scholar
  13. R.N. Davé and R. Krishnapuram. Robust Clustering Methods: A Unified View. IEEE Transactions on Fuzzy Systems 5(1997):270–293. IEEE Press, Piscataway, NJ, USA, 1997 CrossRefGoogle Scholar
  14. A.P. Dempster, N. Laird and D. Rubin. Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society. Series B 39:1–38. Blackwell, Oxford, United Kingdom, 1977 MathSciNetMATHGoogle Scholar
  15. C. Döring, C. Borgelt, and R. Kruse. Effects of Irrelevant Attributes in Fuzzy Clustering. Proc. 14th IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE’05, Reno, NV), 862–866. IEEE Press, Piscataway, NJ, USA, 2005 Google Scholar
  16. D. Dubois. Statistical Reasoning with set-Valued Information: Ontic vs. Epistemic Views. In: Borgelt et al. (2013), 119–136 Google Scholar
  17. J.C. Dunn. A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters. Journal of Cybernetics 3(3):32–57, 1973. American Society for Cybernetics, Washington, DC, USA. Reprinted in Bezdek and Pal (1992), 82–101 MathSciNetMATHCrossRefGoogle Scholar
  18. B.S. Everitt. Cluster Analysis. Heinemann, London, United Kingdom, 1981 Google Scholar
  19. B.S. Everitt and D.J.Hand. Finite Mixture Distributions. Chapman & Hall, London, United Kingdom, 1981 MATHCrossRefGoogle Scholar
  20. H. Frigui and R. Krishnapuram. Clustering by Competitive Agglomeration. Pattern Recognition 30(7):1109–1119. Pergamon Press, Oxford, United Kingdom, 1997 CrossRefGoogle Scholar
  21. I. Gath and A.B. Geva. Unsupervised Optimal Fuzzy Clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 11:773–781, 1989. IEEE Press, Piscataway, NJ, USA. Reprinted in Bezdek and Pal (1992), 211–218 CrossRefGoogle Scholar
  22. E.E. Gustafson and W.C. Kessel. Fuzzy Clustering with a Fuzzy Covariance Matrix. Proc. of the IEEE Conf. on Decision and Control (CDC 1979, San Diego, CA), 761–766. IEEE Press, Piscataway, NJ, USA, 1979. Reprinted in Bezdek and Pal (1992), 117–122 Google Scholar
  23. J.A. Hartigan and M.A. Wong. A k-Means Clustering Algorithm. Applied Statistics 28:100–108. Blackwell, Oxford, United Kingdom, 1979 MATHCrossRefGoogle Scholar
  24. K. Honda and H. Ichihashi. Regularized Linear Fuzzy Clustering and Probabilistic PCA Mixture Models. IEEE Transactions on Fuzzy Systems 13(4):508–516. IEEE Press, Piscataway, NJ, USA, 2005 CrossRefGoogle Scholar
  25. F. Höppner, F. Klawonn, R. Kruse, and T. Runkler. Fuzzy Cluster Analysis. J. Wiley & Sons, Chichester, United Kingdom, 1999 MATHGoogle Scholar
  26. E. Hüllermeier. Fuzzy-Methods in Machine Learning and Data Mining: Status and Prospects. Fuzzy Sets and Systems 156(3):387–407. Elsevier, Amsterdam, Netherlands, 2005 MathSciNetCrossRefGoogle Scholar
  27. E. Hüllermeier. Fuzzy Sets in Machine Learning and Data Mining. Applied Soft Computing 11(2):1493–1505. Elsevier, Amsterdam, Netherlands, 2011 CrossRefGoogle Scholar
  28. H. Ichihashi, K. Miyagishi and K. Honda. Fuzzy c-Means Clustering with Regularization by K-L Information. Proc. 10th IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE 2001, Melbourne, Australia), 924–927. IEEE Press, Piscataway, NJ, USA, 2001 Google Scholar
  29. A.K. Jain and R.C. Dubes. Algorithms for Clustering Data. Prentice Hall, Englewood Cliffs, NJ, USA, 1988 MATHGoogle Scholar
  30. K. Jajuga. L 1-norm Based Fuzzy Clustering. Fuzzy Sets and Systems 39(1):43–50. Elsevier, Amsterdam, Netherlands, 2003 MathSciNetCrossRefGoogle Scholar
  31. N.B. Karayiannis. MECA: Maximum Entropy Clustering Algorithm. Proc. 3rd IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE 1994, Orlando, FL), I:630–635. IEEE Press, Piscataway, NJ, USA, 1994 CrossRefGoogle Scholar
  32. L. Kaufman and P. Rousseeuw. Finding Groups in Data: An Introduction to Cluster Analysis. J. Wiley & Sons, New York, NY, USA, 1990 CrossRefGoogle Scholar
  33. F. Klawonn and F. Höppner. What is Fuzzy about Fuzzy Clustering? Understanding and Improving the Concept of the Fuzzifier. Proc. 5th Int. Symposium on Intelligent Data Analysis (IDA 2003, Berlin, Germany), 254–264. Springer-Verlag, Berlin, Germany, 2003 Google Scholar
  34. R. Krishnapuram and J.M. Keller. A Possibilistic Approach to Clustering. IEEE Transactions on Fuzzy Systems 1(2):98–110. IEEE Press, Piscataway, NJ, USA, 1993 CrossRefGoogle Scholar
  35. R. Krishnapuram and J.M. Keller. The Possibilistic c-Means Algorithm: Insights and Recommendations. IEEE Transactions on Fuzzy Systems 4(3):385–393. IEEE Press, Piscataway, NJ, USA, 1996 CrossRefGoogle Scholar
  36. R. Kruse. On the Variance of Random Sets. Journal of Mathematical Analysis and Applications 122:469–473. Elsevier, Amsterdam, Netherlands, 1987 MathSciNetMATHCrossRefGoogle Scholar
  37. R. Kruse and K.D. Meyer. Statistics with Vague Data. D. Reidel Publishing Company, Dordrecht, Netherlands, 1987 MATHCrossRefGoogle Scholar
  38. R. Kruse, M.R. Berthold, C. Moewes, M.A. Gil, P. Grzegorzewski, and O. Hryniewicz (eds). Synergies of Soft Computing and Statistics for Intelligent Data Analysis. Advances in Intelligent Systems and Computing, vol. 190. Springer-Verlag, Heidelberg/Berlin, Germany, 2012 Google Scholar
  39. S. Kullback and R.A. Leibler. On Information and Sufficiency. Annals of Mathematical Statistics 22:79–86. Institute of Mathematical Statistics, Hayward, CA, USA, 1951 MathSciNetMATHCrossRefGoogle Scholar
  40. H. Kwakernaak. Fuzzy Random Variables—I. Definitions and Theorems. Information Sciences 15:1–29. Elsevier, Amsterdam, Netherlands, 1978 MathSciNetMATHCrossRefGoogle Scholar
  41. H. Kwakernaak. Fuzzy Random Variables—II. Algorithms and Examples for the Discrete Case. Information Sciences 17:252–278. Elsevier, Amsterdam, Netherlands, 1979 MathSciNetCrossRefGoogle Scholar
  42. R.P. Li and M. Mukaidono. A Maximum Entropy Approach to Fuzzy Clustering. Proc. 4th IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE 1994, Yokohama, Japan), 2227–2232. IEEE Press, Piscataway, NJ, USA, 1995 Google Scholar
  43. S. Lloyd. Least Squares Quantization in PCM. IEEE Transactions on Information Theory 28:129–137. IEEE Press, Piscataway, NJ, USA, 1982 MathSciNetMATHCrossRefGoogle Scholar
  44. S. Miyamoto and M. Mukaidono. Fuzzy c-Means as a Regularization and Maximum Entropy Approach. Proc. 7th Int. Fuzzy Systems Association World Congress (IFSA’97, Prague, Czech Republic), II:86–92, 1997 Google Scholar
  45. S. Miyamoto and K. Umayahara. Fuzzy Clustering by Quadratic Regularization. Proc. IEEE Int. Conf. on Fuzzy Systems/IEEE World Congress on Computational Intelligence (WCCI 1998, Anchorage, AK), 2:1394–1399. IEEE Press, Piscataway, NJ, USA, 1998 Google Scholar
  46. Y. Mori, K. Honda, A. Kanda, and H. Ichihashi. A Unified View of Probabilistic PCA and Regularized Linear Fuzzy Clustering. Proc. Int. Joint Conf. on Neural Networks (IJCNN 2003, Portland, OR), I:541–546. IEEE Press, Piscataway, NJ, USA, 2003 CrossRefGoogle Scholar
  47. D. Özdemir and L. Akarun. A Fuzzy Algorithm for Color Quantization of Images. Pattern Recognition 35:1785–1791. Pergamon Press, Oxford, United Kingdom, 2002 MATHCrossRefGoogle Scholar
  48. M. Puri and D. Ralescu. Fuzzy Random Variables. Journal of Mathematical Analysis and Applications 114:409–422. Elsevier, Amsterdam, Netherlands, 1986 MathSciNetMATHCrossRefGoogle Scholar
  49. E.H. Ruspini. A New Approach to Clustering. Information and Control 15(1):22–32, 1969. Academic Press, San Diego, CA, USA. Reprinted in Bezdek and Pal (1992), 63–70 MATHCrossRefGoogle Scholar
  50. C.E. Shannon. The Mathematical Theory of Communication. The Bell System Technical Journal 27:379–423. Bell Laboratories, Murray Hill, NJ, USA, 1948 MathSciNetMATHGoogle Scholar
  51. H. Timm, C. Borgelt, C. Döring, and R. Kruse. An Extension to Possibilistic Fuzzy Cluster Analysis. Fuzzy Sets and Systems 147:3–16. Elsevier Science, Amsterdam, Netherlands, 2004 MathSciNetMATHCrossRefGoogle Scholar
  52. R. Viertl. Statistical Methods for Fuzzy Data. John Wiley & Sons, Chichester, UK, 2011 MATHCrossRefGoogle Scholar
  53. C. Wei and C. Fahn. The Multisynapse Neural Network and Its Application to Fuzzy Clustering. IEEE Transactions on Neural Networks 13(3):600–618. IEEE Press, Piscataway, NJ, USA, 2002 CrossRefGoogle Scholar
  54. M.S. Yang. On a Class of Fuzzy Classification Maximum Likelihood Procedures. Fuzzy Sets and Systems 57:365–375. Elsevier, Amsterdam, Netherlands, 1993 MathSciNetMATHCrossRefGoogle Scholar
  55. M. Yasuda, T. Furuhashi, M. Matsuzaki and S. Okuma. Fuzzy Clustering Using Deterministic Annealing Method and Its Statistical Mechanical Characteristics. Proc. 10th IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE 2001, Melbourne, Australia), 2:797–800. IEEE Press, Piscataway, NJ, USA, 2001 Google Scholar
  56. J. Yu and M.S. Yang. A Generalized Fuzzy Clustering Regularization Model with Optimality Tests and Model Complexity Analysis. IEEE Transactions on Fuzzy Systems 15(5):904–915. IEEE Press, Piscataway, NJ, USA, 2007 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Rudolf Kruse
    • 1
  • Christian Borgelt
    • 2
  • Frank Klawonn
    • 3
  • Christian Moewes
    • 1
  • Matthias Steinbrecher
    • 4
  • Pascal Held
    • 1
  1. 1.Faculty of Computer ScienceOtto-von-Guericke University MagdeburgMagdeburgGermany
  2. 2.Intelligent Data Analysis & Graphical Models Research UnitEuropean Centre for Soft ComputingMieresSpain
  3. 3.FB InformatikOstfalia University of Applied SciencesWolfenbüttelGermany
  4. 4.SAP Innovation CenterPotsdamGermany

Personalised recommendations