Discriminative K-Means Laplacian Clustering

Article
  • 46 Downloads

Abstract

Recently, more and more multi-source data are widely used in many real world applications. This kind of data is high dimensional and comes from different resources, which are often the attribute information and similarity information of the same data. It is challenging to use these two types of information to deal with the high dimensional problem simultaneously. A natural way to adopt is a two-step procedure: it utilizes feature integration or kernel integration to combine these two types of information first and then perform dimensional reduction like principal component analysis or various manifold learning algorithms. Different from that, we proposed to deal with these problems in a unified framework which combines discriminative K-means clustering and spectral clustering together. Compared with those separate two-step procedure, information integration and dimension reduction can benefit from each other in our method to promote clustering performance.In addition, discriminative K-means clustering has incorporated K-means and linear discriminant analysis to promote clustering and tackle high dimensional problem. Spectral clustering can reduce the original dimension easily due to the singular value decomposition. Thus it is a good way to combine discriminative K-means and spectral clustering to improve clustering and deal with high dimensional problem. Experimental results on multiple real world data sets verified its effectiveness.

Keywords

K-means clustering Laplacian Linear discriminant analysis Dimension reduction 

References

  1. 1.
    Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15:1373–1396CrossRefMATHGoogle Scholar
  2. 2.
    Bishop C (2006) Pattern recognition and machine learning. Springer, BerlinMATHGoogle Scholar
  3. 3.
    Cai D, He X, Han J (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560CrossRefGoogle Scholar
  4. 4.
    Chao G, Sun S (2012) Applying a multitask feature sparsity method for the classification of semantic relations between nominals. In: 2012 international conference on machine learning and cybernetics (ICMLC), vol 1. IEEE, pp 72–76Google Scholar
  5. 5.
    Chao G, Sun S (2012) Semi-supervised multitask learning via self-training and maximum entropy discrimination. In: International conference on neural information processing. Springer, pp 340–347Google Scholar
  6. 6.
    Chao G, Sun S (2016) Alternative multiview maximum entropy discrimination. IEEE Trans Neural Netw Learn Syst 27(7):1445–1456MathSciNetCrossRefGoogle Scholar
  7. 7.
    Chao G, Sun S (2016) Consensus and complementarity based maximum entropy discrimination for multi-view classification. Inf Sci 367:296–310CrossRefGoogle Scholar
  8. 8.
    Chao G, Sun S (2016) Multi-kernel maximum entropy discrimination for multi-view learning. Intell Data Anal 20(3):481–493CrossRefGoogle Scholar
  9. 9.
    Chao G, Sun S, Bi J (2017) A survey on multi-view clustering. arXiv preprint arXiv:1712.06246 (2017)
  10. 10.
    Chen X, Cai D (2011) Large scale spectral clustering with landmark-based representation. In: AAAI, vol 5, p 14Google Scholar
  11. 11.
    Cortes C, Mohri M, Rostamizadeh A (2009) Learning non-linear combination of kernels. In: Proceedings of the 22th annual conference on neural information processing systems, pp 396–404Google Scholar
  12. 12.
    Ding C, Li T (2007) Adaptive dimension reduction using discriminant analysis and k-means clustering. In: Proceedings of the 24th international conference on machine learning, pp 521–528Google Scholar
  13. 13.
    Fletcher R (2002) Principle component analysis, 2nd edn. Springer, BerlinGoogle Scholar
  14. 14.
    Hardoon DR, Szedmak S, Shawe-Taylor J (2004) Canonical correlation analysis: an overview with application to learning methods. Neural Comput 16(12):2639–2664CrossRefMATHGoogle Scholar
  15. 15.
    Hong C, Yu J, Tao D, Wang M (2015) Image-based three-dimensional human pose recovery by multiview locality-sensitive sparse retrieval. IEEE Trans Ind Electron 62(6):3742–3751Google Scholar
  16. 16.
    Hong C, Yu J, Wan J, Tao D, Wang M (2015) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24(12):5659–5670MathSciNetCrossRefGoogle Scholar
  17. 17.
    Hong C, Yu J, You J, Chen X, Tao D (2015) Multi-view ensemble manifold regularization for 3d object recognition. Inf Sci 320:395–405MathSciNetCrossRefGoogle Scholar
  18. 18.
    Jieping Y, Zhao Z, Wu M (2007) Discriminative k-means for clustering. In: Proceedings of the 21th annual conference on neural information processing systemsGoogle Scholar
  19. 19.
    Liu W, Yang X, Tao D, Cheng J, Tang Y (2018) Multiview dimension reduction via hessian multiset canonical correlations. Inf Fusion 41:119–128CrossRefGoogle Scholar
  20. 20.
    Liu W, Zha ZJ, Wang Y, Lu K, Tao D (2016) \( p \)-laplacian regularized sparse coding for human activity recognition. IEEE Trans Ind Electron 63(8):5120–5129Google Scholar
  21. 21.
    Luo Y, Thompson WK, Herr TM, Zeng Z, Berendsen MA, Jonnalagadda SR, Carson MB, Starren J (2017) Natural language processing for ehr-based pharmacovigilance: a structured review. Drug Saf 40(11):1075–1089CrossRefGoogle Scholar
  22. 22.
    MaLachlan GJ (2005) Discriminant analysis and statistical pattern recognition. Wiley-Interscience, LondonGoogle Scholar
  23. 23.
    Nene SA, Nayar SK, Murase H et al (1996) Columbia object image library (COIL-20). Technical report CUCS-005-96Google Scholar
  24. 24.
    Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. In: Proceedings of the 15th annual conference on neural information processing systems, pp 849–856Google Scholar
  25. 25.
    Sun S, Chao G (2013) Multi-view maximum entropy discrimination. In: IJCAI, pp 1706–1712Google Scholar
  26. 26.
    Tao D, Li X, Wu X, Maybank SJ (2007) General tensor discriminant analysis and gabor features for gait recognition. IEEE Trans Pattern Anal Mach Intell 29(10):1700–1715CrossRefGoogle Scholar
  27. 27.
    de la Torre F, Kanade T (2006) Discriminative cluster analysis. In: Proceedings of the 23th international conference on machine learning, pp 241–248Google Scholar
  28. 28.
    Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 22:2323–2326CrossRefGoogle Scholar
  29. 29.
    Wang F, Ding C, Li T (2009) Integrated kl (k-means–Laplacian) clustering: a new clustering approach by combing attribute data and pairwise relations. In: Proceedings of the 2009 SIAM data mining conference, pp 38–48Google Scholar
  30. 30.
    Ye J, Zhao Z, Liu H (2007) Adaptive distance metric learning for clustering. In: Proceedings of IEEE conference on computer vision and pattern recognition in 2007, pp 1–7Google Scholar
  31. 31.
    Yu J, Hong C, Rui Y, Tao D (2017) Multi-task autoencoder model for recovering human poses. IEEE Trans Ind Electron 99:1.  https://doi.org/10.1109/tie.2017.2739691
  32. 32.
    Yu J, Tao D, Wang M, Rui Y (2015) Learning to rank using user clicks and visual features for image retrieval. IEEE Trans Cybern 45(4):767–779CrossRefGoogle Scholar
  33. 33.
    Yu J, Yang X, Gao F, Tao D (2017) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern 47(12):4014–4024CrossRefGoogle Scholar
  34. 34.
    Yu Z, Wu F, Yang Y, Tian Q, Luo J, Zhuang Y (2014) Discriminative coupled dictionary hashing for fast cross-media retrieval. In: Proceedings of the 37th international ACM SIGIR conference on research & development in information retrieval. ACM, pp 395–404Google Scholar
  35. 35.
    Yu Z, Yu J, Fan J, Tao D (2017) Multi-modal factorized bilinear pooling with co-attention learning for visual question answering. In: Proceedings of IEEE international conference on computer vision, vol 3Google Scholar
  36. 36.
    Zha H, He X, Ding C, Simon H, Gu M (2001) Spectral relaxation for k-means clustering. In: Proceedings of the 15th annual conference on neural information processing systems, pp 1057–1064Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringUniversity of ConnecticutStorrsUSA

Personalised recommendations