Fast Multiway Maximum Margin Clustering Based on Genetic Algorithm via the NystrÖm Method

  • Ying Kang
  • Dong Zhang
  • Bo YuEmail author
  • Xiaoyan Gu
  • Weiping Wang
  • Dan Meng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9098)


Motivated by theories of support vector machine, the concept of maximum margin has been extended to the applications in the unsupervised scenario, developing a novel clustering methodmaximum margin clustering (MMC). MMC shows an outstanding performance in computational accuracy, which is superior to other traditional clustering methods. But the integer programming of labels of data instances induces MMC to be a hard non-convex optimization problem to settle. Currently, many techniques like semi-definite programming, cutting plane etc. are embedded in MMC to tackle this problem. However, the increasing time complexity and premature convergence of these methods limit the analytic capability of MMC for large datasets. This paper proposes a fast multiway maximum margin clustering method based on genetic algorithm (GAM3C). GAM3C initially adopts the NystrÖm method to generate a low-rank approximate kernel matrix in the dual form of MMC, reducing the scale of original problem and speeding up the subsequent analyzing process; and then makes use of the solution-space alternation of genetic algorithm to compute the non-convex optimization of MMC explicitly, obtaining the multiway clustering results simultaneously. Experimental results on real world datasets reflect that GAM3C outperforms the state-of-the-art maximum margin clustering algorithms in terms of computational accuracy and running time.


Maximum margin clustering NystrÖm method Genetic algorithm 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Jain, A., Dubes, R.: Algorithms for clustering data. Englewood Cliffs (1988)Google Scholar
  2. 2.
    Kanungo, T., Mount, D.M., Netanyahu, N.S., et al.: An efficient k-means clustering algorithm. IEEE Trans. on PAMI 24(7), 881–892 (2002)CrossRefGoogle Scholar
  3. 3.
    Ng, A. Y., Jordan, M. I., Weiss, Y.: On spectral clustering: analysis and an algorithm. Advances in NIPS, pp. 849–856 (2001)Google Scholar
  4. 4.
    Xu, L., Neufeld, J., Larson, B., Schuurmans, D.: Maximum margin clustering. In: Advances in NIPS, pp. 1537–1544 (2004)Google Scholar
  5. 5.
    Nesterov, Y., Nimirovskii, A.: Interior-point polynomial algorithms in convex programming. SIAM (1994)Google Scholar
  6. 6.
    Bezdek, J., Hathaway, R.: Convergence of alternating optimization. Journal Neural, Parallel & Scientific Computations 11(4), 351–368 (2003)zbMATHMathSciNetGoogle Scholar
  7. 7.
    Kelley, J.E.: The cutting-plane method for solving convex programs. Journal of SIAM 8(4), 703–712 (1960)MathSciNetGoogle Scholar
  8. 8.
    Wang, F., Zhao, B., Zhang, C.S.: Linear time maximum margin clustering. IEEE Trans. on Neural Network 21(2), 319–332 (2010)CrossRefGoogle Scholar
  9. 9.
    Xu, L., Schuurmans, D.: Unsupervised and semi-supervised muliti-class support vector macine. Proc. of NCAI 2, 904–910 (2005)Google Scholar
  10. 10.
    Valizadegan, H., Jin, R.: Generalized maximum margin clustering and unsupervised learning. In: Advances in NIPS, pp. 1417–1424 (1994)Google Scholar
  11. 11.
    Zhang, K., Tsang, I. W., Kwok, J. T.: Maximum margin clustering made practical. In: Proc. of ICML, pp. 1119–1126 (2007)Google Scholar
  12. 12.
    Zhao, B., Wang, F., Zhang, C.: Efficient multiclass maximum margin clustering. In: Proc. of ICML, pp. 1248–1255 (2008b)Google Scholar
  13. 13.
    Gieseke, F., Pahikkala, T., Kramer, O.: Fast evolutionary maximum margin clustering. In: Proc. of ICML, pp. 361–368 (2009)Google Scholar
  14. 14.
    Xianchao Z., Quanzeng Y.: Clusterability analysis and incremental sampling for nyström extension based spectral clustering. ICDM, pp. 942–951 (2011)Google Scholar
  15. 15.
    Choromanska A., Jebara T., Kim H., et al.: Fast spectral clustering via the nyström method. In: Algorithmic Learning Theory, pp. 367–381 (2013)Google Scholar
  16. 16.
    Crammer, K., Singer, Y.: On the algorithmic implementation of multiclass kerner-based vector machine. Journal of MLR 2, 265–292 (2001)Google Scholar
  17. 17.
    Lee, C.P., Lin, C.J.: A study on L2-loss (square hinge-loss) mulit-class SVM. Neural Computation 25(5), 1302–1323 (2013)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Drineas, P., Mahoney, M.W.: On the Nyström method for approximating a Gram matrix for improved kernel-based learning. Journal of MLR 6, 2153–2175 (2005)zbMATHMathSciNetGoogle Scholar
  19. 19.
    Pizzuti, C.: GA-Net: A genetic algorithm for community detection in social networks. In: Proc. of ICPPSNX, pp. 1081–1090 (2008)Google Scholar
  20. 20.
    Alzate, C., Suykens, J.A.K.: Multiway spectral clustering with out-of-sample extensions through weighted kernel PCA. IEEE Trans. on PAMI 32(2), 335–347 (2010)CrossRefGoogle Scholar
  21. 21.
    Labatut, V.: Generalized measures for the evaluation of community detection methods. Journal of CoRR (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Ying Kang
    • 1
    • 2
  • Dong Zhang
    • 3
    • 4
  • Bo Yu
    • 1
    Email author
  • Xiaoyan Gu
    • 1
  • Weiping Wang
    • 1
  • Dan Meng
    • 1
  1. 1.Institute of Information EngineeringChinese Academy of SciencesBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina
  3. 3.State Key Laboratory of High-end Server and Storage TechnologyJinanChina
  4. 4.Inspur Group Corporation Ltd.JinanChina

Personalised recommendations