Abstract
AdaBoost boosts the performance of a weak learner by training a committee of weak learners which learn different features of the training sample space with different emphasis and jointly perform classification or regression of each new data sample by a weighted cumulative vote.We use RBF kernel classifiers to demonstrate that boosting a Strong Learner generally contributes to performance degradation, and identify three patterns of performance degradation due to three different strength levels of the underlying learner. We demonstrate that boosting productivity increases, peaks and then falls as the strength of the underlying learner increases. We highlight patterns of behaviour in the distribution and argue that AdaBoost’s characteristic of forcing the strong learner to concentrate on the very hard samples or outliers with too much emphasis is the cause of performance degradation in Strong Learner boosting. However, by boosting an underlying classifier of appropriately low strength, we are able to boost the performance of the committee to achieve or surpass the performance levels achievable by strengthening the individual classifier with parameter or model selection in many instances. We conclude that, if the strength of the underlying learner approaches the identified strength levels, it is possible to avoid performance degradation and achieve high productivity in boosting by weakening the learner prior to boosting . . .
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Freund, Yoav & Schapire, Robert E.: A Brief Introduction to Boosting. Proceedings of the Sixteenth International Joint Conference on Artificial intelligence.
Freund, Yoav & Schapire, Robert E.: Discussions of the paper ”Arching Classifiers” by Leo Brieman. The Annals of Statistics, 26(3):824–832.
Wickramaratna, Holden & Buxton: Efects of the Strength of the Weak Learner in Boosting Kernel Machines. Submitted to the Journal of Machine Learning Research.
Onoda, T., Ratsch, G. & Muller, K.: An Asymptotic Analysis of AdaBoost in the Binary Classification Case. GMD FIRST Rudower Chaussee, Germany.
Quinlan, J.: Bagging, Boosting and C4.5. National Conference on Artificial Intelligence.
Schapire, Robert E.: Theoretical Views of Boosting and Applications. Proceedings of Algorithmic Learning Theory.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wickramaratna, J., Holden, S., Buxton, B. (2001). Performance Degradation in Boosting. In: Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2001. Lecture Notes in Computer Science, vol 2096. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48219-9_2
Download citation
DOI: https://doi.org/10.1007/3-540-48219-9_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42284-6
Online ISBN: 978-3-540-48219-2
eBook Packages: Springer Book Archive