Skip to main content
SpringerLink
Log in
Book cover

Multi-Objective Memetic Algorithms pp 111–131Cite as

Feature Selection Using Single/Multi-Objective Memetic Frameworks

  • Chapter

Part of the book series: Studies in Computational Intelligence ((SCI,volume 171))

Abstract

Memetic frameworks for the hybridization of wrapper and filter feature selection methods have been proposed for classification problems. The frameworks incorporate filter methods in the traditional evolutionary algorithms to improve classification performance while accelerating the search in the identification of crucial feature subsets. Filter methods are introduced as local learning procedures in the evolutionary search to add or delete features from the chromosome which encodes the selected feature subset. Both single/multi-objective memetic frameworks are described in this chapter. Single objective memetic framework is shown to speedup the identification of optimal feature subset while at the same time maintaining good prediction accuracy. Subsequently, the multiobjective memetic framework extends the notion of optimal feature subset as the simultaneous identification of full class relevant (FCR) and partial class relevant (PCR) features in multiclass problems. Comparison study to existing state-of-the-art filter and wrapper methods, and the standard genetic algorithm highlights the efficacy of the memetic framework in facilitating a good compromise of the classification accuracy and selected feature size on binary and multi class problems.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Liu, H., Yu, L.: Toward integrating feature selection algorithms for classification and clustering. IEEE Transaction on Knowledge and Data Engineering 17(4), 491–501 (2005)

    Article  Google Scholar 

  2. Yu, L.: Redundancy-based feature selection for high-dimensional data and application in bioinformatics, Phd Thesis, Arizona State University (2005)

    Google Scholar 

  3. Golub, T.R., Slonim, D.K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J.P., Coller, H., Loh, M.L., Downing, J.R., Caligiuri, M.A., Bloomfield, C.D., Lander, E.S.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286(5439), 531–537 (1999)

    Article  Google Scholar 

  4. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Machine Learning 46(1-3), 389–422 (2002)

    Article  MATH  Google Scholar 

  5. Li, L., Weinberg, C.R., Darden, T.A., Pedersen, L.G.: Gene selection for sample classification based on gene expression data: study of sensitivity to choice of parameters of the ga/knn method. Bioinformatics 17(12), 1131–1142 (2001)

    Article  Google Scholar 

  6. Deutsch, J.M.: Evolutionary algorithms for finding optimal gene sets in microarray prediction. Bioinformatics 19(1), 45–52 (2003)

    Article  Google Scholar 

  7. Hart, G.L.W., Blum, V., Walorski, M.J., Zunger, A.: Evolutionary approach for determining first-principles hamiltonians. Nature Materials 4, 391–394 (2005)

    Article  Google Scholar 

  8. Blum, V., Hart, G.L.W., Walorski, M.J., Zunger, A.: Using genetic algorithm to map first-principle results to model hamiltonians: Application to the generalized ising model for alloys. Physical Review B 72(165113) (2005)

    Google Scholar 

  9. Kohavi, R., John, G.H.: Wrapper for feature subset selection. Artificial Intelligence 97(1-2), 273–324 (1997)

    Article  MATH  Google Scholar 

  10. Holland, J.H.: Adaptation in natural artificial systems, 2nd edn. MIT Press, Cambridge (1992)

    Google Scholar 

  11. Yang, J.H., Honavar, V.: Feature selection using a genetic algorithm. IEEE Transaction on Intelligent Systems 13(2), 44–49 (1998)

    Article  Google Scholar 

  12. Raymer, M.L., Punch, W.F., Goodman, E.D., Kuhn, L.A., Jain, A.K.: Dimensionality reduction using genetic algorithms. IEEE Transaction on Evolutionary Computation 4(2), 164–171 (2000)

    Article  Google Scholar 

  13. Zhu, Z., Ong, Y.S., Dash, M.: Wrapper-filter feature selection algorithm using a memetic framework. IEEE Transactions On Systems, Man and Cybernetics - Part B 37(1), 70–76 (2007)

    Article  Google Scholar 

  14. Zhu, Z., Ong, Y.S., Dash, M.: Markov blanket-embedded genetic algorithm for gene selection. Pattern Recognition 49(11), 3236–3248 (2007)

    Article  Google Scholar 

  15. Zhu, Z., Ong, Y.S., Zurada, M.: Simultaneous identification of full class relevant and partial class relevant genes. IEEE/ACM Transactions on Computational Biology and Bioinformatics (2008) (Under Review)

    Google Scholar 

  16. Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading (1989)

    MATH  Google Scholar 

  17. Dash, M., Liu, H.: Feature selection for classification. Intelligent Data Analysis: An International Journal 1(3), 131–156 (1997)

    Article  Google Scholar 

  18. Dash, M., Liu, H.: Consistency-based search in feature selection. Artificial Intelligence 151(1-2), 155–176 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  19. Liu, H., Motoda, H.: Feature Selection for Knowledge Discovery and Data Mining. Kluwer Academic, Boston (1998)

    MATH  Google Scholar 

  20. Yu, L., Liu, H.: Efficient feature selection via analysis of relevance and redundancy. Journal of Machine Learning Research 5, 1205–1224 (2004)

    MathSciNet  Google Scholar 

  21. Pearl, J.: Probabilitic Reasoning in Intelligent Systems. Morgan Kaufmann, San Mateo (1988)

    Google Scholar 

  22. Koller, D., Sahami, M.: Toward optimal feature selection. In: Proceedings of the 13th International Conference on Machine Learning, pp. 284–292 (1996)

    Google Scholar 

  23. Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C. Cambridge University Press, Cambridge (1998)

    Google Scholar 

  24. Krasnogor, N.: Studies on the Theory and Design Space of Memetic Algorithms, Ph.D. Thesis, Faculty of Computing, Mathematics and Engineering, University of the West of England, Bristol, U.K (2002)

    Google Scholar 

  25. Ong, Y.S., Keane, A.J.: Meta-lamarckian in memetic algorithm. IEEE Transaction on Evolutionary Computation 8(2), 99–110 (2004)

    Article  Google Scholar 

  26. Baker, J.E.: Adaptive selection methods for genetic algorithms. In: Proceedings of the 1st International Conference on Genetic Algorithms, pp. 101–111 (1985)

    Google Scholar 

  27. QuinLan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  28. Robnic-Sikonja, M., Kononenko, I.: Theoretical and empirical analysis of relieff and rrelieff. Machine Learning 53(1-2), 23–69 (2003)

    Article  Google Scholar 

  29. Liu, H., Setiono, R.: Chi2: Feature selection and discretization of numeric attributes. In: Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence (1995)

    Google Scholar 

  30. Statnikov, A., Aliferis, C.F., Tsamardinos, I., Hardin, D., Levy, S.: A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis. Bioinformatics 21(5), 631–643 (2005)

    Article  Google Scholar 

  31. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Transaction on Evolutionary Computation 6(2), 182–197 (2002)

    Article  Google Scholar 

  32. Ishibuchi, H., Murata, T.: A multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Trans. on Systems, Man, and Cybernetics -Part C: Applications and Reviews 28(3), 392–403 (1998)

    Article  Google Scholar 

  33. Yoshida, T., Ishibuchi, H., Murata, T.: Balance between genetic search and local search in memetic algorithm for multiobjective permutation flowshop scheduling. IEEE Transaction on Evolutionary Computation 7(2), 204–223 (2003)

    Article  Google Scholar 

  34. Oliveira, L.S., Morita, M., Sabourin, R.: Feature Selection for Ensembles Using the Multi-Objective Optimization Approach. In: Jin, Y. (ed.) Multi-Objective Machine Learning, ch. 3. Springer, Heidelberg (2006)

    Google Scholar 

  35. Tsymbal, A., Puuronen, S., Patterson, D.W.: Ensemble feature selection with the simple bayesian classification. Information Fusion 4(2), 87–100 (2003)

    Article  Google Scholar 

  36. Braga-Neto, U.M., Dougherty, E.R.: Is cross-validation valid for small-sample microarray classification? Bioinformatics 20(3), 374–380 (2004)

    Article  Google Scholar 

  37. Ruiz, R., Riquelme, J.C., Aguilar-Ruiz, J.S.: Incremental wrapper-based gene selection from microarray data for cancer classification. Pattern Recognition 39(12), 2383–2392 (2006)

    Article  Google Scholar 

  38. Li, L., Pedersen, L.G., Darden, T.A., Weinberg, C.R.: Computational analysis of leukemia microarray expression data using ga/knn method. In: Proceeding of the 1st Conference on Critical Assessment of Microarray Data Analysis, CAMDA 2000 (2000)

    Google Scholar 

  39. John, G.H., Kohavi, R., Pfleger, K.: Irrelevant features and the subset selection problem. In: Proceeding of the 11th International Conference on Machine Learning, pp. 121–129 (1994)

    Google Scholar 

  40. Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)

    MATH  Google Scholar 

  41. Diaz-Uriarte, R., de Andres, S.A.: Gene selection and classification of microarray data using random forest. BMC Bioinformatics 7(3) (2006)

    Google Scholar 

  42. Graham, A.: Kronecker Products and Matrix Calculus With Applications. Halsted Press/John Wiley and Sons, NY (1981)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Engineering, Nanyang Technological University, Singapore

    Zexuan Zhu & Yew-Soon Ong

  2. School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore

    Jer-Lai Kuo

Authors
  1. Zexuan Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yew-Soon Ong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jer-Lai Kuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, National University of Singapore, Singapore

    Chi-Keong Goh  & Kay Chen Tan  & 

  2. School of Computer Engineering, Nanyang Technological University, Singapore

    Yew-Soon Ong

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Zhu, Z., Ong, YS., Kuo, JL. (2009). Feature Selection Using Single/Multi-Objective Memetic Frameworks. In: Goh, CK., Ong, YS., Tan, K.C. (eds) Multi-Objective Memetic Algorithms. Studies in Computational Intelligence, vol 171. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88051-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-88051-6_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-88050-9

  • Online ISBN: 978-3-540-88051-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation