Data Mining and Knowledge Discovery

, Volume 19, Issue 2, pp 277–292 | Cite as

A fast ensemble pruning algorithm based on pattern mining process

  • Qiang-Li Zhao
  • Yan-Huang Jiang
  • Ming Xu


Ensemble pruning deals with the reduction of base classifiers prior to combination in order to improve generalization and prediction efficiency. Existing ensemble pruning algorithms require much pruning time. This paper presents a fast pruning approach: pattern mining based ensemble pruning (PMEP). In this algorithm, the prediction results of all base classifiers are organized as a transaction database, and FP-Tree structure is used to compact the prediction results. Then a greedy pattern mining method is explored to find the ensemble of size k. After obtaining the ensembles of all possible sizes, the one with the best accuracy is outputted. Compared with Bagging, GASEN, and Forward Selection, experimental results show that PMEP achieves the best prediction accuracy and keeps the size of the final ensemble small, more importantly, its pruning time is much less than other ensemble pruning algorithms.


Pattern mining based ensemble pruning FP-Tree Bagging Back-propagation neural network 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Ali KM, Pazzani MJ (1996) Error reduction through learning multiple descriptions. Mach Learn 24(3): 173–202Google Scholar
  2. Allwein EL, Schapire RE, Singer Y (2000) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1: 113–141CrossRefMathSciNetGoogle Scholar
  3. Asuncion DNA (2007) UCI machine learning repository.
  4. Breiman L (1996) Bagging predictors. Mach Learn 24(2): 123–140MATHMathSciNetGoogle Scholar
  5. Caruana R, Niculescu-Mizil A, Crew G, Ksikes A (2004) Ensemble selection from libraries of models. In: Proceedings of the 21st international conference on machine learning (ICML2004), Banff, AlbertaGoogle Scholar
  6. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7: 1–30MathSciNetGoogle Scholar
  7. Han J, Pei J (2000) Mining frequent patterns by pattern growth: methodology and implications. SIGKDD Explor 2(2): 14–20CrossRefGoogle Scholar
  8. Jain AK, Duin RPW, Mao JC (2000) Statistical pattern recognition: a review. IEEE Trans Patt Anal Mach Intell 22(1): 4–37CrossRefGoogle Scholar
  9. Martínez-Muñoz G, Suarez A (2007) Using boosting to prune bagging ensembles. Patt Recogn Lett 28(1): 156–165CrossRefGoogle Scholar
  10. Opitz D, Maclin R (1999) Popular ensemble methods: an empirical study. J Artif Intell Res 11: 169–198MATHGoogle Scholar
  11. Parmanto B, Munro PW, Doyle HR (1996) Improving committee diagnosis with resampling techniques. In: Touretzky DS, Mozer MC, Hesselmo ME (eds) Advances in neural information processing systems. MIT Press, Cambridge, pp 882–888Google Scholar
  12. Partalas I, Tsoumakas G, Vlahavas I (2009) Pruning an ensemble of classifiers via reinforcement learning. Neurocomputing (in press)Google Scholar
  13. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL (eds) Parallel distributed processing: explorations in the microstructure of cognition. MIT Press, Cambridge, pp 318–362Google Scholar
  14. Ruta D, Gabrys B (2005) Classifier selection for majority voting. Inf Fusion 6(1): 63–81CrossRefGoogle Scholar
  15. Schapire RE (1999) A brief introduction to boosting. Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. Morgan Kaufmann 1401–1406Google Scholar
  16. Tsoumakas G, Angelis L, Vlahavas I (2005) Selective fusion of heterogeneous classifiers. Intell Data Anal 9(6): 511–525Google Scholar
  17. Wolpert DH (1992) Stacked generalization. Neural Netw 5(2): 241–259CrossRefGoogle Scholar
  18. Xu L, Krzyzak A, Suen CY (1992) Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans Syst Man Cybern 22(3): 418–435CrossRefGoogle Scholar
  19. Zhang Y, Burer S, Street WN (2006) Ensemble pruning via semi-definite programming. J Mach Learn Res 7: 1315–1338MathSciNetGoogle Scholar
  20. Zhou ZH, Wu JX, Tang W (2002) Ensembling neural networks: many could be better than all. Artif Intell 137(1–2): 239–263MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.School of Computer ScienceNational University of Defense TechnologyChangshaPeople’s Republic of China

Personalised recommendations