Skip to main content
SpringerLink
Log in

Generational Feature Elimination and Some Other Ranking Feature Selection Methods

  • Chapter
  • First Online:
Advances in Feature Selection for Data and Pattern Recognition

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 138))

Abstract

Feature selection methods are effective in reducing dimensionality, removing irrelevant data, increasing learning accuracy, and improving result comprehensibility. However, the recent increase of dimensionality of data poses a severe challenge to many existing feature selection methods with respect to efficiency and effectiveness. In this chapter, both an overview of reasons for using ranking feature selection methods and the main general classes of this kind of algorithms are described. Moreover, some background of ranking method issues is defined. Next, we are focused on selected algorithms based on random forests and rough sets. Additionally, a newly implemented method, called Generational Feature Elimination (GFE), based on decision tree models, is introduced. This method is based on feature occurrences at given levels inside decision trees created in subsequent generations. Detailed information, about its particular properties and results of performance with comparison to other presented methods, is also included. Experiments are performed on real-life data sets as well as on an artificial benchmark data set.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bermingham, M., Pong-Wong, R., Spiliopoulou, A., Hayward, C., Rudan, I., Campbell, H., Wright, A., Wilson, J., Agakov, F., Navarro, P., Haley, C.: Application of high-dimensional feature selection: evaluation for genomic prediction in man. Sci. Rep. 5 (2015). https://doi.org/10.1038/srep10312

  2. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). https://doi.org/10.1023/A:1010933404324

  3. Cheng, X., Cai, H., Zhang, Y., Xu, B., Su, W.: Optimal combination of feature selection and classification via local hyperplane based learning strategy. BMC Bioinform. 16, 219 (2015). https://doi.org/10.1186/s12859-015-0629-6

  4. Cyran, K.A.: Modified indiscernibility relation in the theory of rough sets with real-valued attributes: application to recognition of Fraunhofer diffraction patterns. In: Peters, J.F., Skowron, A., Rybiński, H. (eds.) Transactions on Rough Sets IX, pp. 14–34. Springer, Berlin (2008). https://doi.org/10.1007/978-3-540-89876-4_2

  5. Dubois, D., Prade, H.: Rough fuzzy sets and fuzzy rough sets. Int. J. Gen. Syst. 17(2–3), 191–209 (1990). https://doi.org/10.1080/03081079008935107

  6. Greco, S., Matarazzo, B., Slowinski, R.: Rough sets theory for multicriteria decision analysis. Eur. J. Oper. Res. 129(1), 1–47 (2001)

    Article  MATH  Google Scholar 

  7. Guyon, I., Gunn, S., Hur, A.B., Dror, G.: Result analysis of the NIPS 2003 feature selection challenge. In: Proceedings of the 17th International Conference on Neural Information Processing Systems, pp. 545–552 (2004)

    Google Scholar 

  8. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46(1), 389–422 (2002). https://doi.org/10.1023/A:1012487302797

  9. Jensen, R., Shen, Q.: New approaches to fuzzy-rough feature selection. IEEE Trans. Fuzzy Syst. 17(4), 824–838 (2009). https://doi.org/10.1109/TFUZZ.2008.924209

  10. Johannes, M., Brase, J., Frohlich, H., Gade, S., Gehrmann, M., Falth, M., Sultmann, H., Beissbarth, T.: Integration of pathway knowledge into a reweighted recursive feature elimination approach for risk stratification of cancer patients. Bioinformatics 26(17), 2136–2144 (2010). https://doi.org/10.1093/bioinformatics/btq345

  11. Klement, E.P., Mesiar, R., Pap, E.: Triangular Norms. Kluwer Academic Publishers, Dordrecht (2000)

    Book  MATH  Google Scholar 

  12. Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. 97(1), 273–324 (1997). https://doi.org/10.1016/S0004-3702(97)00043-X

  13. Kuhn, M., Johnson, K.: Applied Predictive Modeling. Springer, New York, NY (2013)

    Book  MATH  Google Scholar 

  14. Kursa, M., Rudnicki, W.: Feature selection with the Boruta package. J. Stat. Softw. 36(1), 1–13 (2010)

    Google Scholar 

  15. Kursa, M.B., Jankowski, A., Rudnicki, W.R.: Boruta—a system for feature selection. Fundam. Inf. 101(4), 271–285 (2010). https://doi.org/10.3233/FI-2010-288

  16. Li, J., Cheng, K., Wang, S., Morstatter, F., Trevino, R.P., Tang, J., Liu, H.: Feature selection: A data perspective. CoRR. arXiv:1601.07996 (2016)

  17. Nilsson, R., Peña, J.M., Björkegren, J., Tegnér, J.: Detecting multivariate differentially expressed genes. BMC Bioinform. 8(1), 150 (2007). https://doi.org/10.1186/1471-2105-8-150

  18. Paja, W.: Feature selection methods based on decision rule and tree models. In: Czarnowski, I., Caballero, A.M., Howlett, R.J., Jain, L.C. (eds.) Intelligent Decision Technologies 2016: Proceedings of the 8th KES International Conference on Intelligent Decision Technologies (KES-IDT 2016)—Part II, pp. 63–70. Springer International Publishing, Cham (2016). https://doi.org/10.1007/978-3-319-39627-9_6

  19. Paja, W., Wrzesien, M., Niemiec, R., Rudnicki, W.R.: Application of all-relevant feature selection for the failure analysis of parameter-induced simulation crashes in climate models. Geosci. Model Dev. 9(3), 1065–1072 (2016). https://doi.org/10.5194/gmd-9-1065-2016

  20. Pancerz, K., Paja, W., Gomuła, J.: Random forest feature selection for data coming from evaluation sheets of subjects with ASDs. In: Ganzha, M., Maciaszek, L., Paprzycki, M. (eds.) Proceedings of the 2016 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 299–302. Gdańsk, Poland (2016)

    Google Scholar 

  21. Pawlak, Z.: Rough sets. Int. J. Comput. Inf. Sci. 11(5), 341–356 (1982)

    Article  MATH  Google Scholar 

  22. Pawlak, Z.: Rough Sets: Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht (1991)

    Book  MATH  Google Scholar 

  23. Phuong, T.M., Lin, Z., Altman, R.B.: Choosing SNPs using feature selection. In: Proceedings of the 2005 IEEE Computational Systems Bioinformatics Conference (CSB’05), pp. 301–309 (2005). https://doi.org/10.1109/CSB.2005.22

  24. Radzikowska, A.M., Kerre, E.E.: A comparative study of fuzzy rough sets. Fuzzy Sets Syst. 126(2), 137–155 (2002). https://doi.org/10.1016/S0165-0114(01)00032-X

  25. Rudnicki, W.R., Wrzesień, M., Paja, W.: All relevant feature selection methods and applications. In: Stańczyk, U., Jain, L.C. (eds.) Feature Selection for Data and Pattern Recognition, pp. 11–28. Springer, Berlin (2015). https://doi.org/10.1007/978-3-662-45620-0_2

  26. Shen, Q., Chouchoulas, A.: A modular approach to generating fuzzy rules with reduced attributes for the monitoring of complex systems. Eng. Appl. Artif. Intell. 13(3), 263–278 (2000). https://doi.org/10.1016/S0952-1976(00)00010-5

  27. Skowron, A., Rauszer, C.: The discernibility matrices and functions in information systems. In: Słowiński, R. (ed.) Intelligent Decision Support: Handbook of Applications and Advances of the Rough Sets Theory, pp. 331–362. Kluwer Academic Publishers, Dordrecht (1992). https://doi.org/10.1007/978-94-015-7975-9_21

  28. Stoean, C., Stoean, R., Lupsor, M., Stefanescu, H., Badea, R.: Feature selection for a cooperative coevolutionary classifier in liver fibrosis diagnosis. Comput. Biol. Med. 41(4), 238–246 (2011). https://doi.org/10.1016/j.compbiomed.2011.02.006

  29. Stoppiglia, H., Dreyfus, G., Dubois, R., Oussar, Y.: Ranking a random feature for variable and feature selection. J. Mach. Learn. Res. 3, 1399–1414 (2003)

    Google Scholar 

  30. Tuv, E., Borisov, A., Torkkola, K.: Feature selection using ensemble based ranking against artificial contrasts. In: Proceedings of the 2006 IEEE International Joint Conference on Neural Network, pp. 2181–2186 (2006). https://doi.org/10.1109/IJCNN.2006.246991

  31. Zhu, Z., Ong, Y.S., Dash, M.: Wrapper-filter feature selection algorithm using a memetic framework. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 37(1), 70–76 (2007). https://doi.org/10.1109/TSMCB.2006.883267

Download references

Acknowledgements

This work was supported by the Center for Innovation and Transfer of Natural Sciences and Engineering Knowledge at the University of Rzeszów.

Author information

Authors and Affiliations

  1. Faculty of Mathematics and Natural Sciences, Department of Computer Science, University of Rzeszów, Pigonia Str. 1, 35-310, Rzeszów, Poland

    Wiesław Paja, Krzysztof Pancerz & Piotr Grochowalski

Authors
  1. Wiesław Paja
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Krzysztof Pancerz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Piotr Grochowalski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wiesław Paja .

Editor information

Editors and Affiliations

  1. Silesian University of Technology , Gliwice, Poland

    Urszula Stańczyk

  2. University of Silesia in Katowice , Katowice, Poland

    Beata Zielosko

  3. University of Bournemouth , Poole, United Kingdom

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Paja, W., Pancerz, K., Grochowalski, P. (2018). Generational Feature Elimination and Some Other Ranking Feature Selection Methods. In: Stańczyk, U., Zielosko, B., Jain, L. (eds) Advances in Feature Selection for Data and Pattern Recognition. Intelligent Systems Reference Library, vol 138. Springer, Cham. https://doi.org/10.1007/978-3-319-67588-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67588-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67587-9

  • Online ISBN: 978-3-319-67588-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation