Recently Multiple Kernel Learning (MKL) has gained increasing attention in constructing a combinational kernel from a number of basis kernels. In this paper, we proposed a novel approach of multiple kernel learning for clustering based on the kernel k-means algorithm. Rather than using a convex combination of multiple kernels over the whole input space, our method associates to each cluster a localized kernel. We assign to each cluster a weight vector for feature selection and combine it with a Gaussian kernel to form a unique kernel for the corresponding cluster. A locally adaptive strategy is used to localize the kernel for each cluster with the aim of minimizing the within-cluster variance of the corresponding cluster. We experimentally compared our methods to kernel k-means and spectral clustering on several data sets. Empirical results demonstrate the effectiveness of our method.


kernel methods kernel k-means multiple kernel clustering localized kernel 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press (2002)Google Scholar
  2. 2.
    Filippone, M., Camastra, F., Masulli, F., Rovetta, S.: A Survey of Kernel and Spectral Methods for Clustering. Pattern Recognition 41(1), 176–190 (2008)zbMATHCrossRefGoogle Scholar
  3. 3.
    Wu, Z.D., Xie, W.X., Yu, J.P.: Fuzzy C-means Clustering Algorithm Based on Kernel Method. In: Proceedings of the International Conference on Computational Intelligence and Multimedia Applications, pp. 49–54 (2003)Google Scholar
  4. 4.
    Yu, K., Ji, L., Zhang, X.: Kernel Nearest-Neighbor Algorithm. Neural Processing Letters 15(2), 147–156 (2002)zbMATHCrossRefGoogle Scholar
  5. 5.
    Zhang, D.Q., Chen, S.C.: Kernel Based Fuzzy and Possibilistic C-means Clustering. In: Proceedings of the International Conference Artificial Neural Network, Turkey, pp. 122–125 (2003)Google Scholar
  6. 6.
    Schölkopf, B., Smola, A.J., Müller, K.R.: Nonlinear Component Analysis as a Kernel Eigenvalue Problem. Neural Computation 10, 1299–1319 (1998)CrossRefGoogle Scholar
  7. 7.
    Inokuchi, R., Miyamoto, S.: LVQ Clustering and SOM Using a Kernel Function. In: Proceedings of IEEE International Conference on Fuzzy Systems, vol. 3, pp. 1497–1500 (2004)Google Scholar
  8. 8.
    Qinand, A.K., Suganthan, P.N.: Kernel Neural Gas Algorithms with Application to Cluster Analysis. In: 17th International Conference on Pattern Recognition (ICPR 2004), vol. 4, pp. 617–620 (2004)Google Scholar
  9. 9.
    Xu, L., Neufeld, J., Larson, B., Schuurmans, D.: Maximum Margin Clustering. In: Advances in Neural Information Processing Systems, vol. 17, pp. 1537–1544 (2005)Google Scholar
  10. 10.
    Camastra, F., Verri, A.: A Novel Kernel Method for Clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(5), 801–804 (2005)CrossRefGoogle Scholar
  11. 11.
    Hur, A.B., Horn, D., Siegelmann, H.T., Vapnik, V.: Support vector clustering. Journal of Machine Learning Research 2, 125–137 (2001)Google Scholar
  12. 12.
    Dhillon, I.S., Guan, Y., Kulis, B.: A Unified View of Kernel K-means. Spectral Clustering and Graph Cuts. Computational Complexity, 1–20 (2005)Google Scholar
  13. 13.
    Lanckriet, G., Cristianini, N., Ghaoui, L., Bartlett, P., Jordan, M.: Learning the Kernel Matrix with Semidefinite Programming. Journal of Machine Learning Research 5, 27–72 (2004)zbMATHGoogle Scholar
  14. 14.
    Sonnenburg, S., Rätsch, G., Schäfer, C., Schölkopf, B.: Large Scale Multiple Kernel Learning. Journal of Machine Learning Research 7, 1531–1565 (2006)zbMATHGoogle Scholar
  15. 15.
    Bach, F., Lanckriet, G., Jordan, M.: Multiple Kernel Learning, Conic Duality, and the SMO Algorithm. In: Proceedings of the 21th International Conference on Machine Learning, pp. 6–13 (2004)Google Scholar
  16. 16.
    Gehler, P.V., Nowozin, S.: Infinite Kernel Learning. Technical Report No. TR-178, Max Planck Institute for Biological Cybernetics (2008)Google Scholar
  17. 17.
    Rakotomamonjy, A., Bach, F., Canu, S., Grandvalet, Y.: Simple MKL. Journal of Machine Learning Research 9, 2491–2521 (2008)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Kloft, M., Brefeld, U., Sonnenburg, S., Laskov, P., Müller, K.R., Zien, A.: Efficient and Accurate l p-norm Multiple Kernel Learning. In: Advances in Neural Information Processing Systems, vol. 22, pp. 997–1005 (2009)Google Scholar
  19. 19.
    Jin, R., Hoi, S.C.H., Yang, T.: Online Multiple Kernel Learning: Algorithms and Mistake Bounds. In: Hutter, M., Stephan, F., Vovk, V., Zeugmann, T. (eds.) ALT 2010. LNCS, vol. 6331, pp. 390–404. Springer, Heidelberg (2010)Google Scholar
  20. 20.
    Zhao, B., Kwok, J.T., Zhang, C.: Multiple Kernel Clustering. In: Proceedings of the 9th SIAM International Conference on Data Mining (SDM 2009), pp. 638–649 (2009)Google Scholar
  21. 21.
    Lewis, D.P., Jebara, T., Noble, W.S.: Nonstationary Kernel Combination. In: Proceedings of the 23th International Conference on Machine Learning, pp. 553–560 (2006)Google Scholar
  22. 22.
    Gönen, M., Alpaydin, E.: Localized multiple kernel learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 352–359 (2008)Google Scholar
  23. 23.
    Ng, A.Y., Jordan, M.I., Weiss, Y.: On Spectral Clustering: Analysis and an Algorithm. In: Advances in Neural Information Processing Systems, vol. 14, pp. 849–856 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Lujiang Zhang
    • 1
  • Xiaohui Hu
    • 1
    • 2
  1. 1.School of Automation Science and Electrical EngineeringBeijing University of Aeronautics & AstronauticsBeijingChina
  2. 2.Institute of SoftwareChinese Academy of SciencesBeijingChina

Personalised recommendations