Multi-objective Genetic Algorithm Evaluation in Feature Selection

  • Newton Spolaôr
  • Ana Carolina Lorena
  • Huei Diana Lee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6576)


Feature Selection may be viewed as a search for optimal feature subsets considering one or more importance criteria. This search may be performed with Multi-objective Genetic Algorithms. In this work, we present an application of these algorithms for combining different filter approach criteria, which rely on general characteristics of the data, as feature-class correlation, to perform the search for subsets of features. We conducted experiments on public data sets and the results show the potential of this proposal when compared to mono-objective genetic algorithms and two popular filter algorithms.


filter feature selection feature importance measures multi-objective genetic algorithms 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arauzo-Azofra, A., Benitez, J.M., Castro, J.L.: Consistency measures for feature selection. Journal of Intelligent Information Systems 30(3), 273–292 (2008)CrossRefGoogle Scholar
  2. 2.
    Asuncion, A., Newman, D.: UCI machine learning repository (2007),
  3. 3.
    Banerjee, M., Mitra, S., Banka, H.: Evolutionary rough feature selection in gene expression data. IEEE Transactions on Systems Man and Cybernetics 37(4), 622–632 (2007)CrossRefGoogle Scholar
  4. 4.
    Bleuler, S., Laumanns, M., Thiele, L., Zitzler, E.: PISA — a platform and programming language independent interface for search algorithms. In: Evolutionary Multi-Criterion Optimization, pp. 494–508 (2003)Google Scholar
  5. 5.
    Bruzzone, L., Persello, C.: A novel approach to the selection of spatially invariant features for the classification of hyperspectral images with improved generalization capability. IEEE Transactions on Geoscience and Remote Sensing 47, 3180–3191 (2009)CrossRefGoogle Scholar
  6. 6.
    Bui, L.T., Alam, S.: An Introduction to Multiobjetive Optimization. Information Science Reference (2008)Google Scholar
  7. 7.
    Charikar, M., Guruswami, V., Kumar, R., Rajagopalan, S., Sahai, A.: Combinatorial feature selection problems. In: Annual Symposium on Foundations of Computer Science, pp. 631–640 (2000)Google Scholar
  8. 8.
    Chung, F.: Spectral Graph Theory. AMS, Providence (1997)zbMATHGoogle Scholar
  9. 9.
    Coello, C.A.C.: Evolutionary multi-objective optimization: a historical view of the field. Computational Intelligence Magazine, 28–36 (2006)Google Scholar
  10. 10.
    Cristianini, N., Shawe-Taylor, J.: Support Vector Machines and other Kernel-Based Learning Methods. Cambridge University Press, Cambridge (2000)CrossRefzbMATHGoogle Scholar
  11. 11.
    Danger, R., Segura-Bedmar, I., Martínez, P., Rosso, P.: A comparison of machine learning techniques for detection of drug target articles. Journal of Biomedical Informatics, 1–12 (2010)Google Scholar
  12. 12.
    Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: Nsga-ii. In: Schoenauer, M., Deb, K., Rudolph, G., Yao, X., Lutton, E., Merelo, J., Schwefel, H.P. (eds.) PPSN 2000. LNCS, vol. 1917, pp. 849–858. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  13. 13.
    Dessí N., Pes, B.: An evolutionary method for combining different feature selection criteria in microarray data classification. Journal of Artificial Evolution and Applications, 1–10 (2009)Google Scholar
  14. 14.
    Duangsoithong, R., Windeatt, T.: Correlation-based and causal feature selection analysis for ensemble classifiers. In: Artificial Neural Networks in Pattern Recognition, pp. 25–36 (2010)Google Scholar
  15. 15.
    Dy, J.G.: Unsupervised feature selection. In: Liu, H., Motoda, H. (eds.) Computational Methods of Feature Selection, pp. 19–39. Chapman & Hall/CRC (2008)Google Scholar
  16. 16.
    Hall, M.A.: Correlation-based Feature Selection for Machine Learning. Phd thesis, University of Waikato (1999)Google Scholar
  17. 17.
    Hall, M.A.: Correlation-based feature selection for discrete and numeric class machine learning. In: International Conference on Machine Learning, pp. 359–366 (2000)Google Scholar
  18. 18.
    Han, J., Kamber, M.: Data mining: concepts and techniques. Morgan Kaufmann, San Francisco (2006)zbMATHGoogle Scholar
  19. 19.
    Handl, J., Kell, D.B., Knowles, J.: Multiobjective optimization in bioinformatics and computational biology. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 279–292 (2007)Google Scholar
  20. 20.
    He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: Advances in Neural Information Processing Systems, pp. 507–514 (2005)Google Scholar
  21. 21.
    Jaimes, A.L., Coello, C.A., Barrientos, J.E.U.: Online objective reduction to deal with many-objective problems. In: International Conference on Evolutionary Multi-Criterion Optimization, pp. 423–437 (2009)Google Scholar
  22. 22.
    Kruskal, W., Wallis, W.A.: Use of ranks in one-criterion variance analysis. American Statistical Association 47, 583–621 (1952)CrossRefzbMATHGoogle Scholar
  23. 23.
    Lee, H.D., Monard, M.C., Wu, F.C.: A fractal dimension based filter algorithm to select features for supervised learning. In: Advances in Artificial Intelligence, pp. 278–288 (2006)Google Scholar
  24. 24.
    Liu, H., Setiono, R.: A probabilistic approach to feature selection - a filter solution. In: International Conference on Machine Learning, pp. 319–327 (1996)Google Scholar
  25. 25.
    Liu, H., Motoda, H.: Computational Methods of Feature Selection. Chapman & Hall/CRC (2008)Google Scholar
  26. 26.
    Lutu, P.E.N., Engelbrecht, A.P.: A decision rule-based method for feature selection in predictive data mining. Expert Systems with Applications 37(1), 602–609 (2010)CrossRefGoogle Scholar
  27. 27.
    Mitchell, M.: An introduction to genetic algorithms. MIT Press, Cambridge (1998)zbMATHGoogle Scholar
  28. 28.
    Mitra, P., Murthy, C.A., Pal, S.K.: Unsupervised feature selection using feature similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(3), 301–312 (2002)CrossRefGoogle Scholar
  29. 29.
    Neshatian, K., Zhang, M.: Pareto front feature selection: using genetic programming to explore feature space. In: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 1027–1034 (2009)Google Scholar
  30. 30.
    Nguyen, H., Franke, K., Petrovic, S.: Improving effectiveness of intrusion detection by correlation feature selection. In: International Conference on Availability, Reliability and Security, pp. 17–24 (2010)Google Scholar
  31. 31.
    QuinLan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  32. 32.
    Salzberg, S.L.: On comparing classifiers: Pitfalls to avoid and a recommended approach. Data Mining and Knowledge Discovery 1, 317–328 (1997)CrossRefGoogle Scholar
  33. 33.
    Santana, L.E.A., Silva, L., Canuto, A.M.P.: Feature selection in heterogeneous structure of ensembles: a genetic algorithm approach. In: International Joint Conference on Neural Networks, pp. 1491–1498 (2009)Google Scholar
  34. 34.
    Shon, T., Kovah, X., Moon, J.: Applying genetic algorithm for classifying anomalous tcp/ip packets. Neurocomputing 69, 2429–2433 (2006)CrossRefGoogle Scholar
  35. 35.
    Spolaôr, N., Lorena, A.C., Lee, H.D.: Seleção de atributos por meio de algoritmos genéticos multiobjetivo (in portuguese). In: Workshop on MSc Dissertation and PhD Thesis in Artificial Intelligence, pp. 1–10 (2010)Google Scholar
  36. 36.
    Spolaôr, N., Lorena, A.C., Lee, H.D.: Use of multiobjective genetic algorithms in feature selection. In: IEEE Brazilian Symposium on Artificial Neural Network, pp. 1–6 (2010)Google Scholar
  37. 37.
    Wang, C.M., Huang, Y.F.: Evolutionary-based feature selection approaches with new criteria for data mining: A case study of credit approval data. Expert Systems with Applications 36(3), 5900–5908 (2009)CrossRefGoogle Scholar
  38. 38.
    Wang, L., Fu, X.: Data Mining With Computational Intelligence. Springer, Heidelberg (2005)zbMATHGoogle Scholar
  39. 39.
    Wilson, D.R., Martinez, T.R.: Improved heterogeneous distance functions. Journal of Artificial Intelligence Research 6, 1–34 (1997)MathSciNetzbMATHGoogle Scholar
  40. 40.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar
  41. 41.
    Yan, W.: Fusion in multi-criterion feature ranking. In: International Conference on Information Fusion, pp. 01–06 (2007)Google Scholar
  42. 42.
    Zaharie, D., Holban, S., Lungeanu, D., Navolan, D.: A computational intelligence approach for ranking risk factors in preterm birth. In: International Symposium on Applied Computational Intelligence and Informatics, pp. 135–140 (2007)Google Scholar
  43. 43.
    Zeleny, M.: An introduction to multiobjetive optimization. In: Cochrane, J.L., Zeleny, M. (eds.) Multiple Criteria Decision Making, pp. 262–301. University of South Carolina Press (1973)Google Scholar
  44. 44.
    Zhu, Z., Ong, Y.S., Kuo, J.L.: Feature selection using single/multi-objective memetic frameworks. In: Goh, C.K., Ong, Y.S., Tan, K.C. (eds.) Multi-Objective Memetic Algorithms, pp. 111–131. Springer, Heidelberg (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Newton Spolaôr
    • 1
    • 2
  • Ana Carolina Lorena
    • 1
  • Huei Diana Lee
    • 2
  1. 1.Grupo Interdisciplinar em Mineração de Dados e AplicaçõesUniversidade Federal do ABCSanto AndréBrasil
  2. 2.Laboratório de BioinformáticaUniversidade Estadual do Oeste do ParanáFoz do IguaçuBrasil

Personalised recommendations