Nonparametric Fisher Kernel Using Fuzzy Clustering
The Fisher kernel, which refers to the inner product in the feature space of the Fisher score, has been known to be a successful tool for feature extraction using a probabilistic model. If an appropriate probabilistic model for given data is known, the Fisher kernel provides a discriminative classifier such as support vector machines with good generalization. However, if the distribution is unknown, it is difficult to obtain an appropriate Fisher kernel. In this paper, we propose a new nonparametric Fisher-like kernel derived from fuzzy clustering instead of a probabilistic model, noting that fuzzy clustering methods such as a family of fuzzy c-means are highly related to probabilistic models, e.g., entropy-based fuzzy c-means and a Gaussian mixture distribution model. The proposed kernel is derived from observing the last relationship. Numerical examples show the effectiveness of the proposed method.
KeywordsSupport Vector Machine Feature Space Probabilistic Model Fuzzy Cluster Neural Information Processing System
Unable to display preview. Download preview PDF.
- 2.Chapelle, O., Weston, J., Schölkopf, B.: Cluster kernels for semi-supervised learning. Advances in Neural Information Processing Systems 15, 585–592 (2003)Google Scholar
- 3.Ichihashi, H., Honda, K., Tani, N.: Gaussian mixture PDF approximation and fuzzy c-means clustering with entropy regularization. In: Proc. of the 4th Asian Fuzzy System Symposium, Tsukuba, Japan, May 31-June 3, pp. 217–221 (2000)Google Scholar
- 4.Jaakkola, T., Haussler, D.: Exploiting generative models in discriminative classifiers. In: Proc. of Neural Information Processing Systems. NIPS (1998)Google Scholar
- 5.Miyamoto, S., Mukaidono, M.: Fuzzy c-means as a regularization and maximum entropy approach. In: Proc. of the 7th International Fuzzy Systems Association World Congress (IFSA 1997), Prague, Czech, June 25-30, vol. II, pp. 86–92 (1997)Google Scholar
- 6.Miyamoto, S.: Introduction to Cluster Analysis: Theory and Applications of Fuzzy Clustering. Morikita-Shuppan, Tokyo (1999) (in Japanese)Google Scholar
- 7.Seeger, M.: Covariance kernels from bayesian generative models. Advances in Neural Information Processing Systems 14, 905–912 (2001)Google Scholar
- 8.Miyamoto, S., Suizu, D.: Fuzzy c-means clustering using kernel functions in support vector machines. J. of Advanced Computational Intelligence and Intelligent Informatics 7(1), 25–30 (2003)Google Scholar
- 9.Tsuda, K., Kawanabe, M., Muller, K.R.: Clustering with the Fisher score. Advances in Neural Information Processing Systems 15, 729–736 (2003)Google Scholar