New Representations in PSO for Feature Construction in Classification
Feature construction can improve the classification performance by constructing high-level features using the original low-level features and function operators. Particle swarm optimisation (PSO) is an powerful global search technique, but it cannot be directly used for feature construction because of its representation scheme. This paper proposes two new representations, pair representation and array representation, which allow PSO to direct evolve function operators. Two PSO based feature construction algorithms (PSOFCPair and PSOFCArray) are then developed. The two new algorithms are examined and compared with the first PSO based feature construction algorithm (PSOFC), which employs an inner loop to select function operators. Experimental results show that both PSOFCPair and PSOFCArray can increase the classification performance by constructing a new high-level feature. PSOFCArray outperforms PSOFCPair and achieves similar results to PSOFC, but uses significantly shorter computational time. This paper represents the first work on using PSO to directly evolve function operators for feature construction.
KeywordsParticle swarm optimisation Feature construction Classification
Unable to display preview. Download preview PDF.
- 1.Azevedo, G., Cavalcanti, G., Filho, E.: An approach to feature selection for keystroke dynamics systems based on PSO and feature weighting. In: IEEE Congress on Evolutionary Computation (CEC 2007), pp. 3577–3584 (2007)Google Scholar
- 3.Engelbrecht, A.P.: Computational intelligence: an introduction, 2. ed. Wiley (2007)Google Scholar
- 4.Frank, A., Asuncion, A.: UCI machine learning repository (2010)Google Scholar
- 5.Kennedy, J., Eberhart, R.: Particle swarm optimization. IEEE International Conference on Neural Networks. 4, 1942–1948 (1995)Google Scholar
- 11.Neshatian, K.: Feature Manipulation with Genetic Programming. PhD thesis, Victoria University of Wellington, Wellington, New Zealand (2010)Google Scholar
- 12.Neshatian, K., Zhang, M., Johnston, M.: Feature Construction and Dimension Reduction Using Genetic Programming. In: Orgun, M.A., Thornton, J. (eds.) AI 2007. LNCS (LNAI), vol. 4830, pp. 160–170. Springer, Heidelberg (2007)Google Scholar
- 13.Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: IEEE International Conference on Evolutionary Computation (CEC 1998), pp. 69–73 (1998)Google Scholar
- 15.Xue, B., Cervante, L., Shang, L., Browne, W.N., Zhang, M.: A multi-objective particle swarm optimisation for filter based feature selection in classification problems. Connection Science (2012)Google Scholar
- 17.Xue, B., Zhang, M., Dai, Y., Browne, W.N.: PSO for feature construction and binary classification. In: Proceeding of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference, GECCO 2013, 137–144 (2013)Google Scholar