Abstract
Recently, more and more multi-source data are widely used in many real world applications. This kind of data is high dimensional and comes from different resources, which are often the attribute information and similarity information of the same data. It is challenging to use these two types of information to deal with the high dimensional problem simultaneously. A natural way to adopt is a two-step procedure: it utilizes feature integration or kernel integration to combine these two types of information first and then perform dimensional reduction like principal component analysis or various manifold learning algorithms. Different from that, we proposed to deal with these problems in a unified framework which combines discriminative K-means clustering and spectral clustering together. Compared with those separate two-step procedure, information integration and dimension reduction can benefit from each other in our method to promote clustering performance.In addition, discriminative K-means clustering has incorporated K-means and linear discriminant analysis to promote clustering and tackle high dimensional problem. Spectral clustering can reduce the original dimension easily due to the singular value decomposition. Thus it is a good way to combine discriminative K-means and spectral clustering to improve clustering and deal with high dimensional problem. Experimental results on multiple real world data sets verified its effectiveness.
Similar content being viewed by others
References
Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15:1373–1396
Bishop C (2006) Pattern recognition and machine learning. Springer, Berlin
Cai D, He X, Han J (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560
Chao G, Sun S (2012) Applying a multitask feature sparsity method for the classification of semantic relations between nominals. In: 2012 international conference on machine learning and cybernetics (ICMLC), vol 1. IEEE, pp 72–76
Chao G, Sun S (2012) Semi-supervised multitask learning via self-training and maximum entropy discrimination. In: International conference on neural information processing. Springer, pp 340–347
Chao G, Sun S (2016) Alternative multiview maximum entropy discrimination. IEEE Trans Neural Netw Learn Syst 27(7):1445–1456
Chao G, Sun S (2016) Consensus and complementarity based maximum entropy discrimination for multi-view classification. Inf Sci 367:296–310
Chao G, Sun S (2016) Multi-kernel maximum entropy discrimination for multi-view learning. Intell Data Anal 20(3):481–493
Chao G, Sun S, Bi J (2017) A survey on multi-view clustering. arXiv preprint arXiv:1712.06246 (2017)
Chen X, Cai D (2011) Large scale spectral clustering with landmark-based representation. In: AAAI, vol 5, p 14
Cortes C, Mohri M, Rostamizadeh A (2009) Learning non-linear combination of kernels. In: Proceedings of the 22th annual conference on neural information processing systems, pp 396–404
Ding C, Li T (2007) Adaptive dimension reduction using discriminant analysis and k-means clustering. In: Proceedings of the 24th international conference on machine learning, pp 521–528
Fletcher R (2002) Principle component analysis, 2nd edn. Springer, Berlin
Hardoon DR, Szedmak S, Shawe-Taylor J (2004) Canonical correlation analysis: an overview with application to learning methods. Neural Comput 16(12):2639–2664
Hong C, Yu J, Tao D, Wang M (2015) Image-based three-dimensional human pose recovery by multiview locality-sensitive sparse retrieval. IEEE Trans Ind Electron 62(6):3742–3751
Hong C, Yu J, Wan J, Tao D, Wang M (2015) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24(12):5659–5670
Hong C, Yu J, You J, Chen X, Tao D (2015) Multi-view ensemble manifold regularization for 3d object recognition. Inf Sci 320:395–405
Jieping Y, Zhao Z, Wu M (2007) Discriminative k-means for clustering. In: Proceedings of the 21th annual conference on neural information processing systems
Liu W, Yang X, Tao D, Cheng J, Tang Y (2018) Multiview dimension reduction via hessian multiset canonical correlations. Inf Fusion 41:119–128
Liu W, Zha ZJ, Wang Y, Lu K, Tao D (2016) \( p \)-laplacian regularized sparse coding for human activity recognition. IEEE Trans Ind Electron 63(8):5120–5129
Luo Y, Thompson WK, Herr TM, Zeng Z, Berendsen MA, Jonnalagadda SR, Carson MB, Starren J (2017) Natural language processing for ehr-based pharmacovigilance: a structured review. Drug Saf 40(11):1075–1089
MaLachlan GJ (2005) Discriminant analysis and statistical pattern recognition. Wiley-Interscience, London
Nene SA, Nayar SK, Murase H et al (1996) Columbia object image library (COIL-20). Technical report CUCS-005-96
Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. In: Proceedings of the 15th annual conference on neural information processing systems, pp 849–856
Sun S, Chao G (2013) Multi-view maximum entropy discrimination. In: IJCAI, pp 1706–1712
Tao D, Li X, Wu X, Maybank SJ (2007) General tensor discriminant analysis and gabor features for gait recognition. IEEE Trans Pattern Anal Mach Intell 29(10):1700–1715
de la Torre F, Kanade T (2006) Discriminative cluster analysis. In: Proceedings of the 23th international conference on machine learning, pp 241–248
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 22:2323–2326
Wang F, Ding C, Li T (2009) Integrated kl (k-means–Laplacian) clustering: a new clustering approach by combing attribute data and pairwise relations. In: Proceedings of the 2009 SIAM data mining conference, pp 38–48
Ye J, Zhao Z, Liu H (2007) Adaptive distance metric learning for clustering. In: Proceedings of IEEE conference on computer vision and pattern recognition in 2007, pp 1–7
Yu J, Hong C, Rui Y, Tao D (2017) Multi-task autoencoder model for recovering human poses. IEEE Trans Ind Electron 99:1. https://doi.org/10.1109/tie.2017.2739691
Yu J, Tao D, Wang M, Rui Y (2015) Learning to rank using user clicks and visual features for image retrieval. IEEE Trans Cybern 45(4):767–779
Yu J, Yang X, Gao F, Tao D (2017) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern 47(12):4014–4024
Yu Z, Wu F, Yang Y, Tian Q, Luo J, Zhuang Y (2014) Discriminative coupled dictionary hashing for fast cross-media retrieval. In: Proceedings of the 37th international ACM SIGIR conference on research & development in information retrieval. ACM, pp 395–404
Yu Z, Yu J, Fan J, Tao D (2017) Multi-modal factorized bilinear pooling with co-attention learning for visual question answering. In: Proceedings of IEEE international conference on computer vision, vol 3
Zha H, He X, Ding C, Simon H, Gu M (2001) Spectral relaxation for k-means clustering. In: Proceedings of the 15th annual conference on neural information processing systems, pp 1057–1064
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chao, G. Discriminative K-Means Laplacian Clustering. Neural Process Lett 49, 393–405 (2019). https://doi.org/10.1007/s11063-018-9823-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-018-9823-7