Advertisement

Coupled-Feature Hypergraph Representation for Feature Selection

  • Zhihong ZhangEmail author
  • Jianbing Xiahou
  • Lu Bai
  • Edwin R. Hancock
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9069)

Abstract

Real-world objects and their features tend to exhibit multiple relationships rather than simple pairwise ones, and as a result basic graph representation can lead to substantial loss of information. Hypergraph representations, on the other hand, allow vertices to be multiply connected by hyperedges and can hence capture multiple or higher order relationships among features. Due to their effectiveness in representing multiple relationships, in this paper, we draw on recent work on hypergraph clustering to select the relevant feature subset (RFS) from a set of features using high-order (rather than pairwise) similarities. Specifically, we first devise a coupled feature representation to represent the data by utilizing self-coupled and inter-feature coupling relationships, which can be more effective to capture the intrinsic linear and nonlinear information on data structure. Regarding the new data representation, we use a new information theoretic criterion referred to as multivariate mutual information to measure the high-order feature combinations with respect to the class labels. Therefore, we construct a coupled feature hypergraph to model the high-order relations among features. Finally, we locate the relevant feature subset (RFS) from feature hypergraph by maximizing features’ average relevance, which has both low redundancy and strong discriminating power. The size of the relevant feature subset (RFS) is determined automatically. Experimental results demonstrate the effectiveness of our feature selection method on a number of standard data-sets.

Keywords

Hypergraph Coupled feature analysis Feature selection 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Zhou, D., Huang, J., Scholkopf, B.: Learning with hypergraphs: Clustering, classification, and embedding. Advances in Neural Information Processing Systems 19, 1601–1608 (2006)Google Scholar
  2. 2.
    Sun, L., Ji, S., Ye, J.: Hypergraph spectral learning for multi-label classification. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 668–676 (2008)Google Scholar
  3. 3.
    Chung, F.: Spectral Graph Theory. American Mathematical Society (1992)Google Scholar
  4. 4.
    Huang, Y., Liu, Q., Lv, F., Gong, Y., Metaxas, D.N.: Unsupervised image categorization by hypergraph partition. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(6), 1266–1273 (2011)CrossRefGoogle Scholar
  5. 5.
    Zhou, D., Huang, J., Scholkopf, B.: Learning with hypergraphs: Clustering, classification, and embedding. Advances in Neural Information Processing Systems 19, 1601–1608 (2006)Google Scholar
  6. 6.
    Klimmek, R., Wagner, F.: A simple hypergraph min cut algorithm. Technical Report B 96-02, University Berlin, Germany (March 1996)Google Scholar
  7. 7.
    Bollegala, D., Matsuo, Y., Ishizuka, M.: Relation adaptation: learning to extract novel relations with minimum supervision. In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, vol. 3, pp. 2205–2210 (2011)Google Scholar
  8. 8.
    Wang, C., Cao, L.B., Wang, M.C., Li, J.J., Wei, W., Ou, Y.M.: Coupled nominal similarity in unsupervised learning. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, pp. 973–978 (2011)Google Scholar
  9. 9.
    Wang, C., She, Z., Cao, L.B.: Coupled attribute analysis on numerical data. In: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, pp. 1736–1742 (2013)Google Scholar
  10. 10.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(2), 210–227 (2009)CrossRefGoogle Scholar
  11. 11.
    MacKay, D.: Information theory, inference and learning algorithms. Cambridge Univ. Press, Cambridge (2003)zbMATHGoogle Scholar
  12. 12.
    Baum, L.E., Eagon, J.A.: An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology. Bull. Amer. Math. Soc. 73(3), 360–363 (1967)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Battiti, R.: Using Mutual Information for Selecting Features in Supervised Neural Net Learning. IEEE Transactions on Neural Networks, 537–550 (2002)Google Scholar
  14. 14.
    Peng, H., Long, F., Ding, C.: Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(8), 1226–1238 (2005)CrossRefGoogle Scholar
  15. 15.
    Tibshirani, R.: Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society. Series B (Methodological) 58(1), 267–288 (1996)zbMATHMathSciNetGoogle Scholar
  16. 16.
    Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 68(1), 49–67 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Zou, H.: The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101, 1418–1429 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67(2), 301–320 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Zhang, Z., Hancock, E.R.: Hypergraph based Information-theoretic Feature Selection. Pattern Recognition Letters 33, 1991–1999 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Zhihong Zhang
    • 1
    Email author
  • Jianbing Xiahou
    • 1
  • Lu Bai
    • 2
  • Edwin R. Hancock
    • 3
  1. 1.Software schoolXiamen UniversityXiamenChina
  2. 2.School of InformationCentral University of Finance and EconomicsBeijingChina
  3. 3.Department of Computer ScienceUniversity of YorkYorkUK

Personalised recommendations