Kernel Learning for Local Learning Based Clustering

  • Hong Zeng
  • Yiu-ming Cheung
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5768)


For most kernel-based clustering algorithms, their performance will heavily hinge on the choice of kernel. In this paper, we propose a novel kernel learning algorithm within the framework of the Local Learning based Clustering (LLC) (Wu & Schölkopf 2006). Given multiple kernels, we associate a non-negative weight with each Hilbert space for the corresponding kernel, and then extend our previous work on feature selection (Zeng & Cheung 2009) to select the suitable Hilbert spaces for LLC. We show that it naturally renders a linear combination of kernels. Accordingly, the kernel weights are estimated iteratively with the local learning based clustering. The experimental results demonstrate the effectiveness of the proposed algorithm on the benchmark document datasets.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wu, M., Schölkopf, B.: A Local Learning Approach for Clustering. In: NIPS, pp. 1529–1536 (2006)Google Scholar
  2. 2.
    Zeng, H., Cheung, Y.M.: Feature Selection for Local Learning based Clustering. In: PAKDD, pp. 414–425 (2009)Google Scholar
  3. 3.
    Scholköpf, B., Smola, A.J.: Learning With Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (2002)Google Scholar
  4. 4.
    Zelnik-Manor, L., Perona, P.: Self-tuning Spectral Clustering. In: NIPS, pp. 1601–1608 (2004)Google Scholar
  5. 5.
    Bach, F.R., Jordan, M.I.: Learning Spectral Clustering, with Application to Speech Separation. JMLR 7, 1963–2001 (2006)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Lanckriet, G.R.G., Cristianini, N., Bartlett, P., Ghaoui, M.I.E., Jordan, M.I.: Learning the Kernel Matrix with Semidefinite Programming. JMLR 5, 27–72 (2004)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Valizadegan, H., Jin, R.: Generalized Maximum Margin Clustering and Unsupervised Kernel Learning. In: NIPS, pp. 1417–1424 (2007)Google Scholar
  8. 8.
    Lange, T., Buhmann, J.: Fusion of Similarity Data in Clustering. In: NIPS, pp. 723–730 (2005)Google Scholar
  9. 9.
    Ng, A., Jordan, M., Weiss, Y.: On Spectral Clustering: Analysis and an Algorithm. In: NIPS, pp. 849–856 (2001)Google Scholar
  10. 10.
    Yu, S.X., Shi, J.: Multiclass Spectral Clustering. In: ICCV, pp. 313–319 (2003)Google Scholar
  11. 11.
    Calamai, P.H., Moré, J.J.: Projected Gradients Methods for Linearly Constrained Problems. Math. Prog., 93–116 (1987)Google Scholar
  12. 12.
    Rakotomamonjy, A., Bach, F., Canu, S., Grandvalet, Y.: More Efficiency in Multiple Kernel Learning. In: ICML, pp. 775–782 (2007)Google Scholar
  13. 13.
    Bonnans, J.F., Sharpiro, A.: Perturbation Analysis of Optimization Problems (2000)Google Scholar
  14. 14.
    Chapelle, O., Vanpnik, V., Bousquet, O., Mukherjee, S.: Choosing Multiple Parameters for Surport Vector Machines. Mach. Learn., 131–159 (2002)Google Scholar
  15. 15.
    Karypis, G.: CLUTO-A Clustering Toolkit (2002),

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Hong Zeng
    • 1
  • Yiu-ming Cheung
    • 1
  1. 1.Department of Computer ScienceHong Kong Baptist UniversityHong KongChina

Personalised recommendations