Abstract
Feature selection is considered to be a key preprocessing step in machine learning and pattern recognition. Feature evaluation is one of the key issues for constructing a feature selection algorithm. In this work, we propose a new concept of neighborhood margin and neighborhood soft margin to measure the minimal distance between different classes. We use the criterion of neighborhood soft margin to evaluate the quality of candidate features and construct a forward greedy algorithm for feature selection. We conduct this technique on eight classification learning tasks. Compared with the raw data and other three feature selection algorithms, the proposed technique is effective in most of the cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Wang, H., Bell, D., Murtagh, F.: Axiomatic Approach to Feature Subset Selection Based on Relevance. IEEE Transactions on Pattern Analysis and Machine Intelligence 21, 271–277 (1999)
Yu, L., Liu, H.: Efficient Feature Selection via Analysis of Relevance and Redundancy. Journal of Machine Learning Research 5, 1205–1224 (2004)
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. Journal of Machine Learning Research 3, 1157–1182 (2003)
Liu, H., Yu, L.: Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering 17, 491–502 (2005)
Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artificial Intelligence 97, 273–324 (1997)
Abe, S., Thawonmas, R., Kobayashi, Y.: Feature selection by analyzing class regions approximated by ellipsoids. IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews 28, 282–287 (1998)
Neumann, J., Schnorr, C., Steidl, G.: Combined SVM-based feature selection and classification. Machine learning 61, 129–150 (2005)
Ho, T.K., Basu, M.: Complexity measures of supervised classification problems. IEEE Trans. Pattern Analysis and Machine Intelligence 24, 289–300 (2002)
Hu, Q.H., Yu, D., Liu, J.F., Wu, C.: Neighborhood rough set based heterogeneous feature subset selection. Information sciences 178, 3577–3594 (2008)
Hall, M.A.: Correlation-based feature selection for discrete and numeric class machine learning. In: Proc. 17th Int’l. Conf. Machine Learning, pp. 359–366 (2000)
Hu, Q.H., Yu, D.R., Xie, Z.X.: Neighborhood classifier. Expert Systems with Applications 34, 866–876 (2008)
Dash, M., Liu, H.: Consistency-based search in feature selection. Artificial Intelligence 151, 155–176 (2003)
Crammer, K., Gilad-Bachrach, R., Navot, A., Tishby, N.: Margin analysis of the LVQ algorithm. In: Proc. 17th Conference on Neural Information Processing Systems (2002)
Sun, Y., Li, J.: Iterative RELIEF for Feature Weighting. In: Proc. 23rd Int’l. Conf. Machine Learning, pp. 913–920 (2006)
Kononenko, I.: Estimating Attributes: Analysis and Extensions of RELIEF. In: Bergadano, F., De Raedt, L. (eds.) ECML 1994. LNCS, vol. 784, pp. 171–182. Springer, Heidelberg (1994)
Li, Y., Lu, B.-L.: Feature selection based on loss-margin of nearest neighbor classification. Pattern Recognition (2008), doi:10.1016/j.patcog. 2008.10.011
Cortes, C., Vapnik, V.: Support-Vector Networks. Machine Learning 20, 273–297 (1995)
Vapnik, V.: Statistical learning theory. Wiley, NY (1998)
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Gilad-Bachrach, R., Navot, A., Tishby, N.: Margin based feature selection- theory and algorithm. In: Proceddings of 21st international conference on machine learning, Banff, Canada (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hu, Q., Che, X., Liu, J. (2009). Feature Selection via Maximizing Neighborhood Soft Margin. In: Zhou, ZH., Washio, T. (eds) Advances in Machine Learning. ACML 2009. Lecture Notes in Computer Science(), vol 5828. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05224-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-05224-8_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-05223-1
Online ISBN: 978-3-642-05224-8
eBook Packages: Computer ScienceComputer Science (R0)