Classifier Selection by Clustering

  • Hamid Parvin
  • Behrouz Minaei-Bidgoli
  • Hamideh Shahpar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6718)


This paper proposes an innovative combinational algorithm for improving the performance of classifier ensembles both in stabilities of their results and in their accuracies. The proposed method uses bagging and boosting as the generators of base classifiers. Base classifiers are kept fixed as decision trees during the creation of the ensemble. Then we partition the classifiers using a clustering algorithm. After that by selecting one classifier per each cluster, we produce the final ensemble. The weighted majority vote is taken as consensus function of the ensemble. We evaluate our framework on some real datasets of UCI repository and the results show effectiveness of the algorithm comparing with the original bagging and boosting algorithms.


Decision Tree Classifier Ensembles Bagging AdaBoosting 


  1. 1.
    Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases (1998),
  2. 2.
    Breiman, L.: Bagging Predictors. Journal of Machine Learning 24(2), 123–140 (1996)zbMATHGoogle Scholar
  3. 3.
    Breiman, L.: Random Forests. Machine Learning 45(1), 5–32 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Freund, Y., Schapire, R.E.: A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Gunter, S., Bunke, H.: Creation of classifier ensembles for handwritten word recognition using feature selection algorithms. IWFHR (2002)Google Scholar
  6. 6.
    Kuncheva, L.I.: Combining Pattern Classifiers, Methods and Algorithms. Wiley, New York (2005)zbMATHGoogle Scholar
  7. 7.
    Minaei-Bidgoli, B., Topchy, A.P., Punch, W.F.: Ensembles of Partitions via Data Resampling. In: ITCC, pp. 188–192 (2004)Google Scholar
  8. 8.
    Yang, T.: Computational Verb Decision Trees. International Journal of Computational Cognition, 34–46 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Hamid Parvin
    • 1
  • Behrouz Minaei-Bidgoli
    • 1
  • Hamideh Shahpar
    • 1
  1. 1.School of Computer EngineeringIran University of Science and Technology (IUST)TehranIran

Personalised recommendations