Effect of Feature Selection on Bagging Classifiers Based on Kernel Density Estimators

  • Edgar Acuña
  • Alex Rojas
  • Frida Coaquira
Conference paper
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


A combination of classification rules (classifiers) is known as an Ensemble, and in general it is more accurate than the individual classifiers used to build it. One method to construct an Ensemble is Bagging introduced by Breiman, (1996). This method relies on resampling techniques to obtain different training sets for each of the classifiers. Previous work has shown that Bagging is very effective for unstable classifiers. In this paper we present some results in application of Bagging to classifiers where the class conditional density is estimated using kernel density estimators. The effect of feature selection in bagging is also considered.


Feature Selection Linear Discriminant Analysis Kernel Density Misclassification Error Kernel Density Estimator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. BAUER, E. and KOKAVI, R. (1999): An empirical comparison of voting classification algorithms: Bagging, Boosting and variants. Machine Learning, 36, 105–139.CrossRefGoogle Scholar
  2. BLAKE, C. and MERZ, C. (1998): UCI repository of machine learning databases. Department of Computer Science and Information, University of California, Irvine, USA.Google Scholar
  3. BREIMAN, L. (1996): Bagging Predictors. Machine Learning, 26, 123–140.Google Scholar
  4. BREIMAN, L. (1998): Arcing Classfiers. Annals of Statistic, 26, 801–849.MathSciNetzbMATHCrossRefGoogle Scholar
  5. DIETTERICH, T.G (2000): An Experimental comparison of three methods for constructing Ensembles of decision trees: Bagging, Boosting, and randomization. Machine Learning, 26, 801–849.Google Scholar
  6. FREUND, Y. and SCHAPIRE, R. (1996): Experiments with a new boosting algorithm. In Machine Learning, Proceedings of the Thirteenth International Conference, San Francisco, Morgan Kaufman, 148–156.Google Scholar
  7. KOHAVI, R. and JOHN, G.H. (1997): Wrappers for feature subset selection. Artificial Intelligence, 97, 273–324.zbMATHCrossRefGoogle Scholar
  8. MACLIN, R. and OPTIZ, D. (1997): An empirical evaluation of Bagging and Bosting. Proceedings of the Fourteenth National Conference on Artificial Intelligence, AAAI/MIT Press.Google Scholar
  9. MICHIE, D., SPIGELHALTER, D.J. and TAYLOR, C.C. (1994): Machine Learning, Neural and Statistical Classification. London: Ellis Horwood.zbMATHGoogle Scholar
  10. QUINLAN, J.R. (1996): Bagging, Boosting and C4.5. Proceedings of the Thirteenth National Conference on Artificial Intelligence, AAAI/MIT Press, 725–730.Google Scholar
  11. SILVERMAN, B.W. (1986): Density Estimation for Statistics and Data Analysis. Chapman and Hall. London.zbMATHGoogle Scholar
  12. TITTERINGTON, D.M. (1980): A comparative study of kernel-based density estimates for categorical data. Technometrics, 22, 259–268.MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Edgar Acuña
    • 1
  • Alex Rojas
    • 2
  • Frida Coaquira
    • 1
  1. 1.Department of MathematicsUniversity of Puerto Rico at MayaguezMayaguezPuerto Rico
  2. 2.Department of StatisticsCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations