Unsupervised Feature Selection for Multi-cluster Data via Smooth Distributed Score

  • Furui Liu
  • Xiyan Liu
Part of the Communications in Computer and Information Science book series (CCIS, volume 304)


Unsupervised feature selection is one of the key topics in data engineering. Previous studies usually use a score vector which has the same length as the feature number to measure the discriminating power of each feature, and the top ranked features are considered to represent the intrinsic multi-cluster structure of the original data. Among different algorithms, Multi-Cluster Feature Selection(MCFS) is one well designed algorithm for its superior performance in feature selection tasks. However, in practice the score vector of MCFS is often sparse, and it brings a problem that only few features are well evaluated about the discriminating power while most others’ are still ambiguous. In this paper, by simultaneously solving one L1-regularized regression and one L2-regularized regression, we propose a novel Multi-Cluster Feature Selection via Smooth Distributed Score(MCFS-SDS), which combines the two results to clearly evaluate the discriminating power of most features via smooth distributed score vector. It is extremely efficient when cluster number is small. Experimental results over various real-life data demonstrate the effectiveness of the proposed algorithm.


Unsupervised feature selection clustering spectral regression 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, Hoboken (2000)Google Scholar
  2. 2.
    He, X., Cai, D., Niyogi, P.: Laplacian Score for Feature Selection. In: Advances in Neural Information Processing Systems 18 (NIPS 2005) (2005)Google Scholar
  3. 3.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York (2001)zbMATHGoogle Scholar
  4. 4.
    Stewart, G.W.: Matrix Algorithms: Eigensystems, vol. II. SIAM, Philadelphia (2001)zbMATHCrossRefGoogle Scholar
  5. 5.
    Chung, F.R.K.: Spectral Graph Theory. Regional Conference Series in Mathematics, vol. 92. AMS (1997)Google Scholar
  6. 6.
    Wolf, L., Shashua, A.: Feature Selection for Unsupervised and Supervised Infe-rence: The Emergence of Sparsity in A Weight-Based Approach. Journal of Machine Learning Research 6, 1855–1887 (2005)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Cai, D., Zhang, C., He, X.: Unsupervised Feature Selection for Multi-Cluster Data. In: 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2010), pp. 333–342 (2010)Google Scholar
  8. 8.
    Tenenbaum, J., Silva, V.D.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science 290(5500), 2319–2323 (2000)CrossRefGoogle Scholar
  9. 9.
    Paige, C.C., Saunders, M.A.: LSQR: An Algorithm for Sparse Linear Equations And Sparse Least Squares. ACM Transactions on Mathematical Software 8(1), 43–71 (1982)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Furui Liu
    • 1
  • Xiyan Liu
    • 2
  1. 1.State Key Lab of CAD&CG, College of Computer ScienceZhejiang UniversityChina
  2. 2.College of Computer Science and Information EngineeringChongqing Technology and Business UniversityChina

Personalised recommendations