Feature Selection via lp-Norm Support Vector Machines

  • Yong Shi
  • Yingjie Tian
  • Gang Kou
  • Yi Peng
  • Jianping Li
Part of the Advanced Information and Knowledge Processing book series (AI&KP)


Though support vector machine has been a promising tool in machine learning, but it does not directly obtain the feature importance. Identifying a subset of features which contribute most to classification is also an important task in classification. The benefit of feature selection is twofold. It leads to parsimonious models that are often preferred in many scientific problems, and it is also crucial for achieving good classification accuracy in the presence of redundant features. We can combine SVM with various feature selection strategies. Some of them are “filters”: general feature selection methods independent of SVM; on the other hand, some are wrapper-type methods: modifications of SVM which choose important features as well as conduct training/testing. In the machine learning literature, there are several proposals for feature selection to accomplish the goal of automatic feature selection in the SVM, in some of which they applied the l 0-norm, l 1-norm SVM and got competitive performance. We proposed two models in this chapter, l p -norm C-support vector classification (l p -SVC) and l p -norm proximal support vector machine (l p -PSVM), which separately combines C-SVC and PSVM with feature selection strategy by introducing the l p -norm (0<p<1).


Feature Selection Nonzero Entry Smoothing Function Local Optimal Solution Sparse Solution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 26.
    Bradley, P., Mangasarian, O.: Feature selection via concave minimization and support vector machines. In: International Conference on Machine Learning. Morgan Kaufmann, San Mateo (1998) Google Scholar
  2. 30.
    Candes, E., Wakin, M., Boyd, S.: Enhancing sparsity by reweighted l 1 minimization. J. Fourier Anal. Appl. 14, 877–905 (2008) MathSciNetMATHCrossRefGoogle Scholar
  3. 35.
    Chen, X., Xu, F., Ye, Y.: Lower bound theory of nonzero entries in solutions of l 2l p minimization. Technical report, Department of Applied Mathematics, The Hong Kong Polytechnic University (2009) Google Scholar
  4. 36.
    Chen, X., Zhou, W.: Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. Preprint, Department of Applied Mathematics, The Hong Kong Polytechnic University (2008) Google Scholar
  5. 77.
    Friedman, J., Hastie, T., Rosset, S., Tibshirani, R., Zhu, J.: Discussion of “Consistency in boosting” by W. Jiang, G. Lugosi, N. Vayatis and T. Zhang. Ann. Stat. 32, 102–107 (2004) Google Scholar
  6. 79.
    Fung, G., Mangasarian, O.: Proximal support vector machine classifiers. In: Proceedings of International Conference of Knowledge Discovery and Data Mining, pp. 77–86 (2001) Google Scholar
  7. 99.
    Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46, 389–422 (2002) MATHCrossRefGoogle Scholar
  8. 216.
    Weston, J., Mukherjee, S., Chapelle, O., Pontil, M., Poggio, T., Vapnik, V.: Feature selection for SVMs. In: Advances in Neural Information Processing Systems, vol. 13 (2001) Google Scholar
  9. 252.
    Zhu, J., Rosset, S., Hastie, T., Tibshirani, R.: 1-norm support vector machines. In: Advances in Neural Information Processing Systems, vol. 16 (2004) Google Scholar
  10. 254.
    Zou, H., Yuan, M.: The f norm support vector machine. Stat. Sin. 18, 379–398 (2008) MathSciNetMATHGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Yong Shi
    • 1
    • 2
  • Yingjie Tian
    • 1
  • Gang Kou
    • 3
  • Yi Peng
    • 3
  • Jianping Li
    • 4
  1. 1.Research Center on Fictitious Economy and Data ScienceChinese Academy of SciencesBeijingChina
  2. 2.College of Information Science & TechnologyUniversity of Nebraska at OmahaOmahaUSA
  3. 3.School of Management and EconomicsUniversity of Electronic Science and Technology of ChinaChengduChina
  4. 4.Institute of Policy and ManagementChinese Academy of SciencesBeijingChina

Personalised recommendations