Abstract
Recently, we have proposed a new decision tree family called soft decision trees where a node chooses both its left and right children with different probabilities as given by a gating function, different from a hard decision node which chooses one of the two. In this paper, we extend the original algorithm by introducing local dimension reduction via \(L_1\) and \(L_2\) regularization for feature selection and smoother fitting. We compare our novel approach with the standard decision tree algorithms over 27 classification data sets. We see that both regularized versions have similar generalization ability with less complexity in terms of number of nodes, where \(L_2\) seems to work slightly better than \(L_1\).
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann, San Meteo, CA
Murthy SK, Kasif S, Salzberg S (1994) A system for induction of oblique decision trees. J Artif Intell Res 2:1–32
Guo H, Gelfand SB (1992) Classification trees with neural network feature extraction. IEEE Trans Neural Netw 3:923–933
Yıldız OT, Alpaydın E (2001) Omnivariate decision trees. IEEE Trans Neural Netw 12(6):1539–1546
Irsoy O, Yildiz OT, Alpaydin E (2012) Soft decision trees. In: Proceedings of the international conference on pattern recognition, Tsukuba, Japan, pp 1819–1822
Blake C, Merz C (2000) UCI repository of machine learning databases
Yıldız OT, Alpaydın E (2005) Linear discriminant trees. Int J Pattern Recogn Artif Intell 19(3):323–353
Alpaydın E (1999) Combined \(5\times 2\) cv F test for comparing supervised classification learning classifiers. Neural Comput 11:1975–1982
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer International Publishing Switzerland
About this paper
Cite this paper
Yıldız, O.T., Alpaydın, E. (2013). Regularizing Soft Decision Trees. In: Gelenbe, E., Lent, R. (eds) Information Sciences and Systems 2013. Lecture Notes in Electrical Engineering, vol 264. Springer, Cham. https://doi.org/10.1007/978-3-319-01604-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-01604-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-01603-0
Online ISBN: 978-3-319-01604-7
eBook Packages: Computer ScienceComputer Science (R0)