New Representations in PSO for Feature Construction in Classification

  • Yan Dai
  • Bing XueEmail author
  • Mengjie Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8602)


Feature construction can improve the classification performance by constructing high-level features using the original low-level features and function operators. Particle swarm optimisation (PSO) is an powerful global search technique, but it cannot be directly used for feature construction because of its representation scheme. This paper proposes two new representations, pair representation and array representation, which allow PSO to direct evolve function operators. Two PSO based feature construction algorithms (PSOFCPair and PSOFCArray) are then developed. The two new algorithms are examined and compared with the first PSO based feature construction algorithm (PSOFC), which employs an inner loop to select function operators. Experimental results show that both PSOFCPair and PSOFCArray can increase the classification performance by constructing a new high-level feature. PSOFCArray outperforms PSOFCPair and achieves similar results to PSOFC, but uses significantly shorter computational time. This paper represents the first work on using PSO to directly evolve function operators for feature construction.


Particle swarm optimisation Feature construction Classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Azevedo, G., Cavalcanti, G., Filho, E.: An approach to feature selection for keystroke dynamics systems based on PSO and feature weighting. In: IEEE Congress on Evolutionary Computation (CEC 2007), pp. 3577–3584 (2007)Google Scholar
  2. 2.
    Clerc, M., Kennedy, J.: The particle swarm- explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation 6(1), 58–73 (2002)CrossRefGoogle Scholar
  3. 3.
    Engelbrecht, A.P.: Computational intelligence: an introduction, 2. ed. Wiley (2007)Google Scholar
  4. 4.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010)Google Scholar
  5. 5.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. IEEE International Conference on Neural Networks. 4, 1942–1948 (1995)Google Scholar
  6. 6.
    Krawiec, K.: Genetic programming-based construction of features for machine learning and knowledge discovery tasks. Genetic Programming and Evolvable Machines 3(4), 329–343 (2002)CrossRefzbMATHGoogle Scholar
  7. 7.
    Liu, H., Motada, H. (eds.): Feature extraction, construction and selection: A data mining perspective. Kluwer Academic Publishers, Norwell (1998)zbMATHGoogle Scholar
  8. 8.
    Marinakis, Y., Marinaki, M., Dounias, G.: Particle swarm optimization for pap-smear diagnosis. Expert Systems with Applications 35(4), 1645–1656 (2008)CrossRefGoogle Scholar
  9. 9.
    Muharram, M.A., Smith, G.D.: Evolutionary Feature Construction Using Information Gain and Gini Index. In: Keijzer, M., O’Reilly, U.-M., Lucas, S., Costa, E., Soule, T. (eds.) EuroGP 2004. LNCS, vol. 3003, pp. 379–388. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Neshatian, K., Zhang, M., Andreae, P.: A filter approach to multiple feature construction for symbolic learning classifiers using genetic programming. IEEE Transactions on Evolutionary Computation 16(5), 645–661 (2012)CrossRefGoogle Scholar
  11. 11.
    Neshatian, K.: Feature Manipulation with Genetic Programming. PhD thesis, Victoria University of Wellington, Wellington, New Zealand (2010)Google Scholar
  12. 12.
    Neshatian, K., Zhang, M., Johnston, M.: Feature Construction and Dimension Reduction Using Genetic Programming. In: Orgun, M.A., Thornton, J. (eds.) AI 2007. LNCS (LNAI), vol. 4830, pp. 160–170. Springer, Heidelberg (2007)Google Scholar
  13. 13.
    Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: IEEE International Conference on Evolutionary Computation (CEC 1998), pp. 69–73 (1998)Google Scholar
  14. 14.
    Unler, A., Murat, A.: A discrete particle swarm optimization method for feature selection in binary classification problems. European Journal of Operational Research 206(3), 528–539 (2010)CrossRefzbMATHGoogle Scholar
  15. 15.
    Xue, B., Cervante, L., Shang, L., Browne, W.N., Zhang, M.: A multi-objective particle swarm optimisation for filter based feature selection in classification problems. Connection Science (2012)Google Scholar
  16. 16.
    Xue, B., Zhang, M., Browne, W.: Particle swarm optimization for feature selection in classification: A multi-objective approach. IEEE Transactions on Cybernetics 43(6), 1656–1671 (2013)CrossRefGoogle Scholar
  17. 17.
    Xue, B., Zhang, M., Dai, Y., Browne, W.N.: PSO for feature construction and binary classification. In: Proceeding of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference, GECCO 2013, 137–144 (2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.School of Engineering and Computer ScienceVictoria University of WellingtonWellingtonNew Zealand

Personalised recommendations