Abstract
While fuzzy c-means (FCM) and its variants have become popular tools in many application fields, their fuzzy partition natures were often discussed only from the empirical viewpoints without theoretical insight. This chapter reviews some fuzzy clustering models induced by probabilistic mixture concepts and discusses the effects of introduction of adjustable fuzziness penalties into statistical models. First, the entropy regularization-based FCM proposed by Miyamoto et al. is revisited from the Gaussian mixtures viewpoint and the fuzzification mechanism is compared with the standard FCM. Second, the regularization concept is discussed in fuzzy co-clustering context and a multinomial mixtures-induced clustering model is reviewed. Some illustrative examples demonstrate the characteristics of fuzzy clustering algorithms with adjustable fuzziness penalties, and the interpretability of object partition is shown to be improved. Finally, a possible future direction of fuzzy clustering research is discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
M. R. Anderberg, Cluster Analysis for Applications, Academic Press, 1981.
J. B. MacQueen, “Some methods of classification and analysis of multivariate observations,” Proc. of 5th Berkeley Symposium on Math. Stat. and Prob., pp. 281–297, 1967.
J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum Press, 1981.
E. H. Ruspini, “A new approach to clustering,” Information and Control, vol. 15, no. 1, pp. 22–32, July 1969.
S. Miyamoto and M. Mukaidono, “Fuzzy \(c\)-means as a regularization and maximum entropy approach,” Proc. of the 7th Int. Fuzzy Syst. Assoc. World Cong., vol. 2, pp. 86–92, 1997.
S. Miyamoto, H. Ichihashi and K. Honda, Algorithms for Fuzzy Clustering, Springer, 2008.
S. Miyamoto and K. Umayahara, “Fuzzy clustering by quadratic regularization,” Proc. 1998 IEEE Int. Conf. Fuzzy Systems and IEEE World Congr. Computational Intelligence, vol. 2, pp. 1394–1399, 1998.
R. J. Hathaway, “Another interpretation of the EM algorithm for mixture distributions,” Statistics & Probability Letters, vol. 4, pp. 53–56, 1986.
R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis, John Wiley & Sons, 1973.
C. M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press, 1995.
H. Ichihashi, K. Miyagishi and K. Honda, “Fuzzy \(c\)-means clustering with regularization by K-L information,” Proc. of 10th IEEE International Conference on Fuzzy Systems, vol. 2, pp. 924–927, 2001.
K. Honda and H. Ichihashi, “Regularized linear fuzzy clustering and probabilistic PCA mixture models,” IEEE Trans. Fuzzy Systems, vol. 13, no. 4, pp. 508–516, 2005.
C.-H. Oh, K. Honda and H. Ichihashi, “Fuzzy clustering for categorical multivariate data,” Proc. of Joint 9th IFSA World Congress and 20th NAFIPS International Conference, pp. 2154–2159, 2001.
K. Kummamuru, A. Dhawale and R. Krishnapuram, “Fuzzy co-clustering of documents and keywords,” Proc. 2003 IEEE Int’l Conf. Fuzzy Systems, vol. 2, pp. 772–777, 2003.
L. Rigouste, O. Cappé and F. Yvon, “Inference and evaluation of the multinomial mixture model for text clustering,” Information Processing and Management, vol. 43, no. 5, pp. 1260–1280, 2007.
K. Honda, S. Oshio and A. Notsu, “Fuzzy co-clustering induced by multinomial mixture models,” Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 19, no. 6, pp. 717–726, 2015.
R. Krishnapuram and J. M. Keller, “A possibilistic approach to clustering,” IEEE Trans. on Fuzzy Systems, vol. 1, pp. 98–110, 1993.
T. Needham, “A visual explanation of Jensen’s inequality,” American Mathematical Monthly, vol. 100, no. 8, pp. 768–771, 1993.
A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. of the Royal Statistical Society, Series B, vol. 39, pp. 1–38, 1977.
Y. Kanzawa, “Fuzzy co-clustering algorithms based on fuzzy relational clustering and TIBA imputation,” Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 18, no. 2, pp. 182–189, 2014.
Y. Kanzawa, “On Bezdek-type fuzzy clustering for categorical multivariate data,” Proc. of Joint 7th Int. Conf. Soft Computing and Intelligent Systems and 15th Int. Sympo. Advanced Intelligent Systems, pp. 694–699, 2014.
N. R. Pal and J. C. Bezdek, “On cluster validity for the fuzzy \(c\)-mean model,” IEEE Trans. Fuzzy Systems, vol. 3, pp. 370–379, Aug. 1995.
J. Yu, Q. Cheng and H. Huang, “Analysis of the weighting exponent in the FCM,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 34, no. 1, pp. 634–639, 2004.
K. Rose, E. Gurewitz and G. Fox, “A deterministic annealing approach to clustering,” Pattern Recognition Letters, vol. 11, pp. 589–594, 1990.
S. Oshio, K. Honda, S. Ubukata and A. Notsu, “Deterministic annealing framework in MMMs-induced fuzzy co-clustering and its applicability,” International Journal of Computer Science and Network Security, vol. 16, no. 1, pp. 43–50, 2016.
K. Sjölander, K. Karplus, M. Brown, R. Hughey, A. Krogh, I. Saira Mian and D. Haussler, “Dirichlet mixtures: a method for improved detection of weak but significant protein sequence homology,” Computer Applications in the Biosciences, vol. 12, no. 4, pp. 327–345, 1996.
X. Ye, Y.-K. Yu and S. F. Altschul, “Compositional adjustment of Dirichlet mixture priors,” Journal of Computational Biology, vol. 17, no. 12, pp. 1607–1620, 2010.
M. E. Tipping and C. M. Bishop, “Mixtures of probabilistic principal component analysers,” Neural Computation, vol. 11, no. 2, pp. 443–482, 1999.
J. Takeuchi, T. Kawabata, and A. R. Barron, “Properties of Jeffreys mixture for markov sources,” IEEE Transactions on Information Theory, vol. 59, no. 1, pp. 438–457, January 2013.
Acknowledgements
This work was supported in part by the Ministry of Education, Culture, Sports, Science and Technology, Japan, under Grant-in-Aid for Scientific Research (JSPS KAKENHI Grant Number JP26330281).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Honda, K. (2017). Fuzzy Clustering/Co-clustering and Probabilistic Mixture Models-Induced Algorithms. In: Torra, V., Dahlbom, A., Narukawa, Y. (eds) Fuzzy Sets, Rough Sets, Multisets and Clustering. Studies in Computational Intelligence, vol 671. Springer, Cham. https://doi.org/10.1007/978-3-319-47557-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-47557-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47556-1
Online ISBN: 978-3-319-47557-8
eBook Packages: EngineeringEngineering (R0)