Skip to main content
SpringerLink
Log in

A Weighted Feature Selection Method for Instance-Based Classification

  • Conference paper
  • First Online:
Artificial Intelligence: Methodology, Systems, and Applications (AIMSA 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9883))

Abstract

The paper presents a new method for selecting features that is suited for the instance-based classification. The selection is based on the ReliefF estimation of the quality of features in the orthogonal feature space obtained after PCA transformation, as well as on the interpretation of these weights as values proportional to the amount of explained concept changes. The user sets a threshold defining what percent of the whole concept variability the selected features should explain and only the first “stronger” features, which combine weights together exceed this threshold, are selected. During the classification phase the selected features are used along with their weights. The experiment results on 12 benchmark databases have shown the advantages of the proposed method in comparison with traditional ReliefF.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    https://archive.ics.uci.edu/ml/datasets.html.

  2. 2.

    http://datam.i2r.a-star.edu.sg/datasets/krbd/.

References

  1. Bins, J., Draper, B.: Feature selection from huge feature sets. In: Proceedings of the Eighth IEEE International Conference on Computer Vision, vol. 2, pp. 159–165 (2001)

    Google Scholar 

  2. Chang, C.-C.: Generalized iterative RELIEF for supervised distance metric learning. Pattern Recogn. 43(8), 2971–2981 (2010)

    Article  MATH  Google Scholar 

  3. Dhanabal, S., Chandramathi, S.: A review of various k-nearest neighbor query processing techniques. Intern. J. Comput. Appl. 31(7), 14–22 (2011)

    Google Scholar 

  4. Diamataras, K.I., Kung, S.J.: Principal Component Neural Networks. Theory and Applications. Wiley, New York (1996)

    Google Scholar 

  5. Florez-lopez, R.: Reviewing RELIEF and its extensions: a new approach for estimating attributes considering high-correlated features. In: Proceedings of IEEE International Conference on Data Mining, Maebashi, Japan, pp. 605–608 (2002)

    Google Scholar 

  6. Freitag, D., Caruana, R.: Greedy attribute selection. In: Proceedings of Eleven International Conference on Machine Learning, pp. 28–36 (1994)

    Google Scholar 

  7. Hall, M.A.: Correlation-based feature selection of discrete and numeric class machine learning. In: Proceedings of International Conference on Machine Learning (ICML-2000), San Francisco, CA, pp. 359–366. Morgan Kaufmann, San Francisco (2000)

    Google Scholar 

  8. Jolliffe, I.T.: Principal Component Analysis. Springer, New York (1986)

    Book  MATH  Google Scholar 

  9. Kira, K., Rendell, L.A.: The feature selection problem: traditional methods and a new algorithm. In: Proceedings of AAAI 1992, San Jose, USA, pp. 129–134 (1992)

    Google Scholar 

  10. Kononenko, I.: Estimating attributes: analysis and extensions of RELIEF. In: Proceedings of European Conference on Machine Learning, Catania, Italy, vol. 182, pp. 171–182 (1994)

    Google Scholar 

  11. Ordonez, C., Mohanam, N., Garcia-Alvarado, C.: PCA for large data sets with parallel data summarization. Distrib. Parallel Databases 32(3), 377–403 (2014)

    Article  Google Scholar 

  12. Pechenizkiy, M.: The impact of feature extraction on the performance of a classifier: kNN, Naïve Bayes and C4.5. In: Kégl, B., Lee, H.-H. (eds.) Canadian AI 2005. LNCS (LNAI), vol. 3501, pp. 268–279. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  13. Robnik-Sikonja, M., Kononenko, I.: Theoretical and empirical analysis of ReliefF and RReliefF. Mach. Learn. J. 53, 23–69 (2003)

    Article  MATH  Google Scholar 

  14. Sharma, A., Paliwala, K., Onwubolu, G.: Class-dependent PCA, MDC and LDA: a combined classifier for pattern classification. Pattern Recogn. 39, 1215–1229 (2006)

    Article  MATH  Google Scholar 

  15. Strandjev, B., Agre, G.: On impact of PCA for solving classification tasks defined on facial images. Intern. J. Reason. Based Intell. Syst. 6(3/4), 85–92 (2014)

    Google Scholar 

  16. Sun, Y., Li, J.: Iterative RELIEF for feature weighting: algorithms, theories, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1035–1051 (2007)

    Article  MathSciNet  Google Scholar 

  17. Tsymbal, A., Puuronen, S., Pechenizkiy, M., Baumgarten, M., Patterson, D.W.: Eigenvector-based feature extraction for classification. In: Proceedings of FLAIRS Conference, pp. 354–358 (2002)

    Google Scholar 

  18. Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)

    Article  Google Scholar 

  19. Vergara, J., Estevez, P.: A review of feature selection methods based on mutual information. Neural Comput. Appl. 24, 175–186 (2014)

    Article  Google Scholar 

  20. Wettschereck, D., Aha, D.W., Mohri, T.: A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms. Artif. Intell. Rev. 11, 273–314 (1997)

    Article  Google Scholar 

  21. Yang, J., Li, Y.-P.: Orthogonal relief algorithm for feature selection. In: Huang, D.-S., Li, K., Irwin, G.W. (eds.) ICIC 2006. LNCS, vol. 4113, pp. 227–234. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  22. Zeng, X., Wang, Q., Zhang, C., Cai, H.: Feature selection based on ReliefF and PCA for underwater sound classification. In: Proceedings of the 3rd International Conference on Computer Science and Network Technology (ICCSNT), Dalian, pp. 442–445 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Information and Communication Technologies, Bulgarian Academy of Sciences, Sofia, Bulgaria

    Gennady Agre

  2. Sofia University “St. Kliment Ohrisdski”, Sofia, Bulgaria

    Anton Dzhondzhorov

Authors
  1. Gennady Agre
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anton Dzhondzhorov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gennady Agre .

Editor information

Editors and Affiliations

  1. Winston-Salem State University, Winston Salem, North Carolina, USA

    Christo Dichev

  2. Institute of Information and Communication Technologies, Bulgarian Academy of Sciences, Sofia, Bulgaria

    Gennady Agre

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Agre, G., Dzhondzhorov, A. (2016). A Weighted Feature Selection Method for Instance-Based Classification. In: Dichev, C., Agre, G. (eds) Artificial Intelligence: Methodology, Systems, and Applications. AIMSA 2016. Lecture Notes in Computer Science(), vol 9883. Springer, Cham. https://doi.org/10.1007/978-3-319-44748-3_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-44748-3_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-44747-6

  • Online ISBN: 978-3-319-44748-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation