Abstract
This paper investigates enhancements of decision tree bagging which mainly aim at improving computation times, but also accuracy. The three questions which are reconsidered are: discretization of continuous attributes, tree pruning, and sampling schemes. A very simple discretization procedure is proposed, resulting in a dramatic speedup without significant decrease in accuracy. Then a new method is proposed to prune an ensemble of trees in a combined fashion, which is significantly more effective than individual pruning. Finally, different resampling schemes are considered leading to different CPU time/accuracy tradeoffs. Combining all these enhancements makes it possible to apply tree bagging to very large datasets, with computational performances similar to single tree induction. Simulations are carried out on two synthetic databases and four real-life datasets.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
E. Bauer and R. Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning, 36:105–139, 1999.
L. Breiman. Bagging predictors. Technical report, University of California, Department of Statistics, September 1994.
L. Breiman. Pasting small votes for classification in large databases and on-line. Machine Learning, 36:85–103, 1999.
L. Breiman. Using adaptive bagging to debias regressions. Technical report, Statistics Department, University of California, Berkeley, February 1999.
L. Breiman, J.H. Friedman, R.A. Olsen, and C.J. Stone. Classification and Regression Trees. Wadsworth International (California), 1984.
J. H. Friedman and P. Hall. On bagging and nonlinear estimation. Technical report, Statistics Department, Standford University, January 2000.
J.H. Friedman. On bias, variance, 0/1-loss, and the curse-of-dimensionality. Data Mining and Knowledge Discovery, 1:55–77, 1997.
P. Geurts and L. Wehenkel. Investigation and reduction of discretization variance in decision tree induction. In Proc. of the 11th European Conference on Machine Learning (ECML-2000), Barcelona, pages 162–170, May 2000.
T.O. Kvålseth. Entropy and correlation: Some comments. IEEE Trans. on Systems, Man and Cybernetics, SMC-17(3):517–519, 1987.
Dragos D. Margineantu and Thomas G. Dietterich. Pruning adaptive boosting. In Morgan Kaufmann, editor, Proc. of Fourteenth International Conference on Machine Learning (ICML-97), 19
D. Michie and D.J. Spiegelhalter, editors. Machine learning, neural and statistical classifcation. Ellis Horwood, 1994.
Peter Sollich and Anders Krogh. Learning with ensembles: How over-fiing can be useful In D.S. Touretzky, M.C. Mozer, and M.E. Hasselmo, editors, Advances in Neural Information Processing Systems, volume 8, pages 190–196. MIT Press, 1996.
L. Wehenkel. Automatic learning techniques in power systems. Kluwer Academic, Boston, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Geurts, P. (2000). Some Enhancements of Decision Tree Bagging. In: Zighed, D.A., Komorowski, J., Żytkow, J. (eds) Principles of Data Mining and Knowledge Discovery. PKDD 2000. Lecture Notes in Computer Science(), vol 1910. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45372-5_14
Download citation
DOI: https://doi.org/10.1007/3-540-45372-5_14
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41066-9
Online ISBN: 978-3-540-45372-7
eBook Packages: Springer Book Archive