Abstract
Aggregated classifiers have proven to be successful in reducing misclasÂsification error in a wide range of classification problems. One of the most popular is bagging. But often simple procedures perform comparably in specific applications. For example, linear discriminant analysis (LDA) provides efficient classifiers if the underlying class structure is linear regarding the predictors.
We suggest bagging for a combination of tree classifiers and LDA. The out-of-bag sample is used as an independent learning sample for the computation of linear discriminant functions. The corresponding discriminant variables of the bootstrap sample are used as additional predictors for a classification tree. We illustrate the proposal by a glaucoma classification with laser scanning image data. Moreover, we analyse the properties with a simulation study and benchmark data sets. In summary, our proposal has misclassification error comparable to LDA when LDA performs best and comparable to bagged trees when bagged trees perform best.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
BREIMAN, L. (1996): Bagging predictors, Machine Learning, Vol. 24, 2, 123–140.
BREIMAN, L. (1998): Arcing classifiers, The Annals of Statistics, Vol. 26, 3, 801–824
Breiman, L. (1996): Out-of-bag estimation, Tech. rep., Statistics Department, University of California Berkeley, Berkeley CA 94708.
CIAMPI, A. (1991): Generalized regression trees, Computational Statistics and Data Analysis, Vol. 12, 57–78.
RIPLEY, B. D. (1996): Pattern Recognition and Neural Networks, Cambridge University Press, Cambridge, UK.
FRIEDMAN, J. H. (1997): On bias, variance, 0/1-loss, and the curse-of-dimensionality, Data Mining and Knowledge Discovery, Vol. 1, 1, 55–77.
LEBLANC, M. and TIBSHIRANI, R. (1996): Combing estimates in regression and classification, Journal of the American Statistical Association, Vol. 91, 436, 1641–1650.
EFRON, B. and TIBSHIRANI, R. (1997): Improvements on cross-validation: The.632+ bootstrap method, Journal of the American Statistical Association, Vol. 92, 438, 548–560.
LAUSEN, B. (1997): Generalized regression trees applied to longitudinal nutritional survey data, in Klar, R. and Opitz, O. (Eds.): Classification and Knowledge Organization, Springer, Heidelberg, 467–474.
BREIMAN, L., FRIEDMAN, J. H., OLSHEN, R. A., and STONE, C. J. (1984): Classification and regression trees, Wadsworth, California.
LAUSEN, B., SAUERBREI, W., and SCHUMACHER, M. (1994): Classification and regression trees (CART) used for the exploration of prognostic factors measured on different scales, in Dirschedl, P. and Ostermann, R. (Eds.): Computational Statistics, Physica-Verlag, Heidelberg, 483–496.
Heidelberg Engineering (1997): Heidelberg Retina Tomograph: Bedie-nungsanleitung Software version 2.01., Heidelberg Engineering GmbH, Heidelberg.
HOTHORN, T., PAL, I., GEFELLER, O., LAUSEN, B., MICHELSON, G., and PAULUS, D. (2002): Automated classification of optic nerve head topography images for glaucoma screening, in Studies in Classification, Data Analysis, and Knowledge Organization (to appear), Springer, Heidelberg.
SWINDALE, N. V., STJEPANOVIC, G., CHIN, A., and MIKELBERG, F. S. (2000): Automated analysis of normal and glaucomatous optic nerve head to-pography images., Investigative Ophthalmology and Visual Science, Vol. 41, 7, 1730–42.
CIAMPI, A. and LECHEVALLIER, Y. (2000): Constructing artificial neural net-works for censored survival data from statistical models, in Kiers, H., Rasson, J.-P., Groenen, P., and Schader, M. (Eds.): Data Analysis, Classification, and Related Methods, Springer, Heidelberg, 223–228.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hothorn, T., Lausen, B. (2002). Bagging Combined Classifiers. In: Jajuga, K., Sokołowski, A., Bock, HH. (eds) Classification, Clustering, and Data Analysis. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-56181-8_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-56181-8_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43691-1
Online ISBN: 978-3-642-56181-8
eBook Packages: Springer Book Archive