Bagging Combined Classifiers

  • Torsten Hothorn
  • Berthold Lausen
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


Aggregated classifiers have proven to be successful in reducing misclas­sification error in a wide range of classification problems. One of the most popular is bagging. But often simple procedures perform comparably in specific applications. For example, linear discriminant analysis (LDA) provides efficient classifiers if the underlying class structure is linear regarding the predictors.

We suggest bagging for a combination of tree classifiers and LDA. The out-of-bag sample is used as an independent learning sample for the computation of linear discriminant functions. The corresponding discriminant variables of the bootstrap sample are used as additional predictors for a classification tree. We illustrate the proposal by a glaucoma classification with laser scanning image data. Moreover, we analyse the properties with a simulation study and benchmark data sets. In summary, our proposal has misclassification error comparable to LDA when LDA performs best and comparable to bagged trees when bagged trees perform best.


Linear Discriminant Analysis Bootstrap Sample Classification Tree Optic Nerve Head Misclassification Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. BREIMAN, L. (1996): Bagging predictors, Machine Learning, Vol. 24, 2, 123–140.MathSciNetzbMATHGoogle Scholar
  2. BREIMAN, L. (1998): Arcing classifiers, The Annals of Statistics, Vol. 26, 3, 801–824MathSciNetzbMATHCrossRefGoogle Scholar
  3. Breiman, L. (1996): Out-of-bag estimation, Tech. rep., Statistics Department, University of California Berkeley, Berkeley CA 94708.Google Scholar
  4. CIAMPI, A. (1991): Generalized regression trees, Computational Statistics and Data Analysis, Vol. 12, 57–78.MathSciNetzbMATHCrossRefGoogle Scholar
  5. RIPLEY, B. D. (1996): Pattern Recognition and Neural Networks, Cambridge University Press, Cambridge, UK.zbMATHGoogle Scholar
  6. FRIEDMAN, J. H. (1997): On bias, variance, 0/1-loss, and the curse-of-dimensionality, Data Mining and Knowledge Discovery, Vol. 1, 1, 55–77.CrossRefGoogle Scholar
  7. LEBLANC, M. and TIBSHIRANI, R. (1996): Combing estimates in regression and classification, Journal of the American Statistical Association, Vol. 91, 436, 1641–1650.MathSciNetzbMATHGoogle Scholar
  8. EFRON, B. and TIBSHIRANI, R. (1997): Improvements on cross-validation: The.632+ bootstrap method, Journal of the American Statistical Association, Vol. 92, 438, 548–560.MathSciNetzbMATHGoogle Scholar
  9. LAUSEN, B. (1997): Generalized regression trees applied to longitudinal nutritional survey data, in Klar, R. and Opitz, O. (Eds.): Classification and Knowledge Organization, Springer, Heidelberg, 467–474.CrossRefGoogle Scholar
  10. BREIMAN, L., FRIEDMAN, J. H., OLSHEN, R. A., and STONE, C. J. (1984): Classification and regression trees, Wadsworth, California.zbMATHGoogle Scholar
  11. LAUSEN, B., SAUERBREI, W., and SCHUMACHER, M. (1994): Classification and regression trees (CART) used for the exploration of prognostic factors measured on different scales, in Dirschedl, P. and Ostermann, R. (Eds.): Computational Statistics, Physica-Verlag, Heidelberg, 483–496.CrossRefGoogle Scholar
  12. Heidelberg Engineering (1997): Heidelberg Retina Tomograph: Bedie-nungsanleitung Software version 2.01., Heidelberg Engineering GmbH, Heidelberg.Google Scholar
  13. HOTHORN, T., PAL, I., GEFELLER, O., LAUSEN, B., MICHELSON, G., and PAULUS, D. (2002): Automated classification of optic nerve head topography images for glaucoma screening, in Studies in Classification, Data Analysis, and Knowledge Organization (to appear), Springer, Heidelberg.Google Scholar
  14. SWINDALE, N. V., STJEPANOVIC, G., CHIN, A., and MIKELBERG, F. S. (2000): Automated analysis of normal and glaucomatous optic nerve head to-pography images., Investigative Ophthalmology and Visual Science, Vol. 41, 7, 1730–42.Google Scholar
  15. CIAMPI, A. and LECHEVALLIER, Y. (2000): Constructing artificial neural net-works for censored survival data from statistical models, in Kiers, H., Rasson, J.-P., Groenen, P., and Schader, M. (Eds.): Data Analysis, Classification, and Related Methods, Springer, Heidelberg, 223–228.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Torsten Hothorn
    • 1
  • Berthold Lausen
    • 1
  1. 1.Department of Medical Informatics, Biometry and EpidemiologyUniversity Erlangen-NurembergErlangenGermany

Personalised recommendations