Skip to main content
SpringerLink
Log in
Book cover

International Conference on Innovative Techniques and Applications of Artificial Intelligence

SGAI 2014: Research and Development in Intelligent Systems XXXI pp 35–49Cite as

Improved Stability of Feature Selection by Combining Instance and Feature Weighting

  • Conference paper
  • First Online:

  • 533 Accesses

Abstract

The current study presents a technique that aims at improving stability of feature subset selection by means of a combined instance and feature weighting process. Both types of weights are based on margin concepts and can therefore be naturally interlaced. We report experiments performed on both synthetic and real data (including microarray data) showing improvements in selection stability at a similar level of prediction performance.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Recall \(=\) True positives/(True Positives \(+\) False Negatives); Precision \(=\) True positives/(True Positives \(+\) False Positives). A True Positive is a selected and relevant feature, a False Negative is a discarded and relevant feature, etc.

References

  1. Bachrach, R.G., Navot, A., Tishby, N.: Margin based feature selection-theory and algorithms. In: Proceedings of International Conference on Machine Learning (ICML), pp. 43–50 (2004)

    Google Scholar 

  2. Crammer, K., Gilad-Bachrach, R., Navot, A., Tishby, N.: Margin analysis of the LVQ algorithm. Adv. NIPS, 462–469 (2002)

    Google Scholar 

  3. Devijver, P.A., Kittler, J.: Pattern Recognition: A Statistical Approach. Prentice Hall, Englewood Cliffs (2002)

    Google Scholar 

  4. Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 10, 1895–1923 (1998)

    Article  Google Scholar 

  5. Dudoit, S., Fridlyand, J., Speed, T.P.: Comparison of discrimination methods for the classification of tumors using gene expression data. J. Am. Stat. Assoc. 97(457), 77–87 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  6. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  7. Han, Y., Yu, L.: A variance reduction framework for stable feature selection. Stat. Anal. Data Min. 5, 428–445 (2012)

    Article  MathSciNet  Google Scholar 

  8. Kalousis, A., Prados, J., Hilario, M.: Stability of feature selection algorithms: a study on high-dimensional spaces. Knowl. Inf. Syst. 12(1), 95–116 (2006)

    Article  Google Scholar 

  9. Kira, K., Rendell, L.A.: The feature selection problem: traditional methods and a new algorithm, pp. 129–134. AAAI Press and MIT Press, Cambridge (1992)

    Google Scholar 

  10. Křížek, P., Kittler, J., Hlaváč, V.: Improving stability of feature selection methods. In: Kropatsch, W.G., Kampel, M., Hanbury, A. (eds.) CAIP. Lecture Notes in Computer Science, vol. 4673, pp. 929–936. Springer, Berlin (2007)

    Google Scholar 

  11. Kuncheva, L.I.: A stability index for feature selection. In: IASTED International Conference on Artificial Intelligence and Applications, Innsbruck, Austria. ACTA Press, Anaheim, pp. 390–395 (2007)

    Google Scholar 

  12. Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI repository of machine learning databases, vol. 55. Department of Information and Computer Science, University of California, Irvine (1998). http://www.ics.uci.edu/mlearn/MLRepository.html

  13. Raudys, A., Baumgartner, R., Somorjai, R: On understanding and assessing feature selection bias. LNCS, vol. 3581, pp. 468–472. Springer, Berlin (2005)

    Google Scholar 

  14. Saeys, Y., Abeel, T., Peer, Y.: Robust feature selection using ensemble feature selection techniques. ECML-PKDD, pp. 313–325. Springer, Berlin (2008)

    Google Scholar 

  15. Singhi, S.K., Liu, H.: Feature subset selection bias for classification learning. In: Cohen, W.W., Moore, A. (eds.) ICML, vol. 148, pp. 849–856 (2006)

    Google Scholar 

  16. Somol, P., Novovičová, J.: Evaluating stability and comparing output of feature selectors that optimize feature subset cardinality. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 1921–1939 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, Universitat Politècnica de Catalunya, Jordi Girona, 1-3, 08034, Barcelona, Catalonia, Spain

    Gabriel Prat & Lluís A. Belanche

Authors
  1. Gabriel Prat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lluís A. Belanche
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lluís A. Belanche .

Editor information

Editors and Affiliations

  1. University of Portsmouth, Portsmouth, United Kingdom

    Max Bramer

  2. School of Computing, Eng & Mathematics, University of Brighton, Brighton, West Sussex, United Kingdom

    Miltos Petridis

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Prat, G., Belanche, L.A. (2014). Improved Stability of Feature Selection by Combining Instance and Feature Weighting. In: Bramer, M., Petridis, M. (eds) Research and Development in Intelligent Systems XXXI. SGAI 2014. Springer, Cham. https://doi.org/10.1007/978-3-319-12069-0_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-12069-0_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-12068-3

  • Online ISBN: 978-3-319-12069-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation