Facing the Feature Selection Problem with a Binary PSO-GSA Approach

  • Malek SarhaniEmail author
  • Abdellatif El Afia
  • Rdouan Faizi
Part of the Operations Research/Computer Science Interfaces Series book series (ORCS, volume 62)


Feature selection has become the focus of much research in many areas where we can face the problem of big data or complex relationship among features. Metaheuristics have gained much attention in solving many practical problems, including feature selection. Our contribution in this paper is to propose a binary hybrid metaheuristic to minimize a fitness function representing a trade-off between the classification error of selecting the feature subset and the corresponding number of features. This algorithm combines particle swarm optimization (PSO) and gravitational search algorithm (GSA). Also, a mutation operator is integrated to enhance population diversity. Experimental results on ten benchmark dataset show that our proposed hybrid method for feature selection can achieve high performance when comparing with other metaheuristic algorithms and well-known feature selection approaches.


Feature selection Particle swarm optimization Gravitational search algorithm Machine learning Metaheuristics 


  1. 1.
    Z. Beheshti, S.M. Shamsuddin, S. Hasan, Memetic binary particle swarm optimization for discrete optimization problems. Inf. Sci. 299, 58–84 (2015)CrossRefGoogle Scholar
  2. 2.
    A.R. Behjat, A. Mustapha, H. Nezamabadi, M.N. Sulaiman, N. Mustapha, Feature subset selection using binary gravitational search algorithm for intrusion detection system, in Intelligent Information and Database Systems. Lecture Notes in Computer Science, vol. 7803 (Springer, Berlin/Heidelberg, 2013), pp. 377–386Google Scholar
  3. 3.
    G. Brown, A. Pocock, M.-J. Zhao, M. Lujan, Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. J. Mach. Learn. Res. 13, 27–66 (2012)Google Scholar
  4. 4.
    T. Chakraborti, A. Chatterjee, A novel binary adaptive weight GSA based feature selection for face recognition using local gradient patterns, modified census transform and local binary patterns. Eng. Appl. Artif. Intel. 33, 80–90 (2014)CrossRefGoogle Scholar
  5. 5.
    R. Diao, Q. Shen, Nature inspired feature selection meta-heuristics. Eng. Appl. Artif. Intel. 44(3), 311–340 (2015)Google Scholar
  6. 6.
    T. Ganesan, I.M. Elamvazuthi, K.Z. Ku Shaari, P. Vasant, Swarm intelligence and gravitational search algorithm for multi-objective optimization of synthesis gas production. Appl. Energy 103, 368–374 (2013)CrossRefGoogle Scholar
  7. 7.
    J. Garca-Nieto, E. Alba, L. Jourdan, E. Talbi, Sensitivity and specificity based multiobjective approach for feature selection: application to cancer diagnosis. Inf. Process. Lett. 109, 887–896 (2009)CrossRefGoogle Scholar
  8. 8.
    I. Guyon, A. Elisseeff, An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)Google Scholar
  9. 9.
    X.H. Han, X.M. Chang, L. Quan, X.Y. Xiong, J.X. Li, Z.X. Zhang, Y. Liu, Feature subset selection by gravitational search algorithm optimization. Inf. Sci. 281, 128–146 (2014)CrossRefGoogle Scholar
  10. 10.
    Z. Hu, Y. Bao, T. Xiong, Comprehensive learning particle swarm optimization based memetic algorithm for model selection in short-term load forecasting using support vector regression. Appl. Soft Comput. 25, 15–25 (2014)CrossRefGoogle Scholar
  11. 11.
    S. Jiang, A novel hybrid particle swarm optimization and gravitational search algorithm for solving economic emission load dispatch problems with various practical constraints. Electr. Power Energy Syst. 55, 628–644 (2014)CrossRefGoogle Scholar
  12. 12.
    L. Jourdan, C. Dhaenens, E.-G. Talbi, Evolutionary feature selection for bioinformatics, in Computational Intelligence in Bioinformatics, Chap. 6 (IEEE Press, New York, 2007), pp. 117–139Google Scholar
  13. 13.
    J. Kennedy, R.C. Eberhart, A discrete binary version of the particle swarm optimization, in Proceedings on Conference on Systems, Man and Cybernetics, vol. 4 (1997), pp. 104–109Google Scholar
  14. 14.
    R. Kohavi, G.H. John, Wrappers for feature subset selection. Artif. Intell. 97(1–2), 273–324 (1997)CrossRefGoogle Scholar
  15. 15.
    J. Krause, J. Cordeiro, R.S. Parpinelli, H.S. Lopes, A survey of swarm algorithms applied to discrete optimization problems, in Swarm Intelligence and Bio-Inspired Computation: Theory and Applications (Elsevier Science & Technology Books, London, 2013), pp. 169–191CrossRefGoogle Scholar
  16. 16.
    J. Lee, D.-W. Kim, Memetic feature selection algorithm for multi-label classification. Inf. Sci. 293, 80–96 (2015)CrossRefGoogle Scholar
  17. 17.
    Y. Lu, I. Cohen, X.S. Zhou, Q. Tian, Feature selection using principal feature analysis, in Proceedings of the 15th International Conference on Multimedia, MULTIMEDIA ’07 (ACM, New York, 2007), pp. 301–304CrossRefGoogle Scholar
  18. 18.
    S. Mallick, S.P. Ghoshal, P. Acharjee, S.S. Thakur, Optimal static state estimation using improved particle swarm optimization and gravitational search algorithm. Electr. Power Energy Syst. 52, 254–265 (2013)CrossRefGoogle Scholar
  19. 19.
    S. Mirjalili, G.-G. Wang, L.D. Coelho, Binary optimization using hybrid particle swarm optimization and gravitational search algorithm. Neural Comput. Appl. 25(6), 1423–1435 (2014)CrossRefGoogle Scholar
  20. 20.
    M. Nikravesh, L.A. Zadeh, I. Guyon, S. Gunn, Feature Extraction, Foundations and Applications (Springer, Berlin, 2006)Google Scholar
  21. 21.
    S. Piramuthu, Evaluating feature selection methods for learning in data mining applications. Eur. J. Oper. Res. 156, 483–494 (2004)CrossRefGoogle Scholar
  22. 22.
    J. Pohjalainen, O. Rsnen, S. Kadioglu, Feature selection methods and their combinations in high-dimensional classification of speaker likability, intelligibility and personality traits. Comput. Speech Lang. 29, 145–171 (2015)CrossRefGoogle Scholar
  23. 23.
    E. Rashedi, H. Nezamabadi-pour, S. Saryazdi, BGSA: binary gravitational search algorithm. Nat. Comput. 9(3), 727–745 (2010)CrossRefGoogle Scholar
  24. 24.
    Y. Shi, R.C. Eberhart, A modified particle swarm optimizer, in Proceedings of the IEEE World Congress of Computational Intelligence (1997), pp. 69–73Google Scholar
  25. 25.
    E.-G. Talbi, L. Jourdan, J. Garcia-Nieto, E. Alba, Comparison of population based metaheuristics for feature selection: application to microarray data classification, in AICCSA’2008 IEEE/ACS International Conference on Computer Systems and Applications, Doha, Nov 2008, pp. 45–52Google Scholar
  26. 26.
    A. Unler, A. Murat, A discrete particle swarm optimization method for feature selection in binary classification problems. Eur. J. Oper. Res. 206, 528–539 (2010)CrossRefGoogle Scholar
  27. 27.
    J.R. Vergara, P.A. Estvez, A review of feature selection methods based on mutual information. Neural Comput. Appl. 24(1), 175–186 (2014)CrossRefGoogle Scholar
  28. 28.
    L. Vignolo, D. Milone, J. Scharcanski, Feature selection for face recognition based on multi-objective evolutionary wrapper. Expert Syst. Appl. 40, 5077–5084 (2013)CrossRefGoogle Scholar
  29. 29.
    B. Xue, M. Zhang, W. Browne, Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms. Appl. Soft Comput. 18, 261–176 (2014)CrossRefGoogle Scholar
  30. 30.
    Y. Zhang, S. Wang, P. Phillips, G. Ji, Binary PSO with mutation operator for feature selection using decision tree applied to spam detection. Knowl.-Based Syst. 64, 22–31 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Malek Sarhani
    • 1
    Email author
  • Abdellatif El Afia
    • 1
  • Rdouan Faizi
    • 2
  1. 1.Department of Informatics and Decision SupportMohammed V UniversityRabatMorocco
  2. 2.ENSIAS - Mohammed V UniversityRabatMorocco

Personalised recommendations