Most existing feature selection methods focus on ranking individual features based on a utility criterion, and select the optimal feature set in a greedy manner. However, the feature combinations found in this way do not give optimal classification performance, since they neglect the correlations among features. In an attempt to overcome this problem, we develop a novel unsupervised feature selection technique by using hypergraph spectral embedding, where the projection matrix is constrained to be a selection matrix designed to select the optimal feature subset. Specifically, by incorporating multidimensional interaction information (MII) for higher order similarities measure, we establish a novel hypergraph framework which is used for characterizing the multiple relationships within a set of samples. Thus, the structural information latent in the data can be more effectively modeled. Secondly, we derive a hypergraph embedding view of feature selection which casting the feature discriminant analysis into a regression framework that considers the correlations among features. As a result, we can evaluate joint feature combinations, rather than being confined to consider them individually, and are thus able to handle feature redundancy. Experimental results demonstrate the effectiveness of our feature selection method on a number of standard datasets.


Hypergraph representation Hypergraph subspace learning 


  1. 1.
    Scholkopf, B., Smola, A., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10(5), 1299–1319 (1998)CrossRefGoogle Scholar
  2. 2.
    He, X., Niyogi, P.: Locality preserving projections (LPP). In: Proc. NIPS (2004)Google Scholar
  3. 3.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Proc. NIPS, pp. 585–592 (2002)Google Scholar
  4. 4.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. John Wiley & Sons, New York (2001)zbMATHGoogle Scholar
  5. 5.
    Zhao, Z., Liu, H.: Spectral feature selection for supervised and unsupervised learning. In: Proc. ICML, pp. 1151–1157 (2007)Google Scholar
  6. 6.
    He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: Proc. NIPS (2005)Google Scholar
  7. 7.
    Belhumeur, P.N., Kriegman, D.J.: What is the set of images of an object under all possible illumination conditions? IJCV 28(3), 245–260 (1998)CrossRefGoogle Scholar
  8. 8.
    Cai, D., Zhang, C., He, X.: Unsupervised feature selection for multi-cluster data. In: Proc. ACM SIGKDD, pp. 333–342 (2010)Google Scholar
  9. 9.
    Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. The Annals of Statistics 32(2), 407–499 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Yang, Y., Shen, H.T., Ma, Z., Huang, Z., Zhou, X.: L21-norm regularized discriminative feature selection for unsupervised learning. In: Proc. IJCAI, pp. 1589–1594 (2011)Google Scholar
  11. 11.
    Agarwal, S., Lim, J., Zelnik-Manor, L., Perona, P., Kriegman, D., Belongie, S.: Beyond pairwise clustering. In: Proc. CVPR, pp. 838–845 (2005)Google Scholar
  12. 12.
    Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. on PAMI 27(8), 1226–1238 (2005)CrossRefGoogle Scholar
  13. 13.
    Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Review 51(3), 455–500 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Shashua, A., Zass, R., Hazan, T.: Multi-way Clustering Using Super-Symmetric Non-negative Tensor Factorization. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 595–608. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. 15.
    Björck, A.: Numberical methods for least squares problems. In: Proc. SIAM (1996)Google Scholar
  16. 16.
    Zhang, Z., Hancock, E.R.: Hypergraph based Information-theoretic Feature Selection. Pattern Recognition Letters (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Zhihong Zhang
    • 1
  • Edwin R. Hancock
    • 1
  1. 1.Department of Computer ScienceUniversity of YorkUK

Personalised recommendations