Abstract
This paper presents a new algorithm that finds the generative model of a decision tree from data. We show that for infinite data and finite number of attributes the algorithm always finds the generative model (i.e. the model of the decision tree, from which the data were generated) except measure zero set of distributions. The algorithm returns reasonable results even when the above-mentioned assumptions are not satisfied. The algorithm is polynomial in the number of leaves of the generative model compared to the exponential complexity of the trivial exhaustive search algorithm. Similar result was recently obtained for learning Bayesian networks from data ([1],[2]). Experimental comparison of the new algorithm with the CART standard on both simulated and real data is shown. The new algorithm shows significant improvements over the CART algorithm in both cases. The whole paper is for simplicity restricted to binary variables but can be easily generalized.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
1. Chickering, M.: Learning Equivalence Classes of Bayesian-Network Structures, Journal of Machine Learning Research 2 (2002), pp. 445–498.
2. Chickering, M., Meek, Ch.: Finding Optimal Bayesian Networks, In Proceedings of Eighteenth Conference on Uncertainty in Artificial Intelligence, Edmonton, AB (2002), pp. 94–102.
3. Breiman L. et al.: Classification and Regression Trees, Woodsworth International Group (1984).
4. P. Utgo., N. C. Berkman, and J. A. Clouse: Decision tree induction based on efficient tree restructuring, Machine Learning (1997), pp. 5–44.
5. Utgo., P.E.: Decision Tree Induction Based on Efficient Tree Restructuring, Technical Report 95–18, University of Massachusetts, Department of Computer Science, Amherst, MA (1996).
6. Quinlan, J. R.: Simplifying decision trees. International Journal of Man- Machine Studies, 27 (1987), pp. 221–234.
7. Wikipedia contributions. Occam's Razor. Retrieved from http://en.wikipedia.org/wiki/Occam's Razor on January 8, 2006.
8. Pfahringer, B.: Inducing Small and Accurate Decision Trees, Technical Report, Oesterreichisches Forschungsinstitut fuer Artificial Intelligence, Wien, 1998.
9. Esposito, F.,Malerba, D., Semerado, G.: A Comparative Analysi of Methods for Pruning Decision Trees, IEEE Transactions on Pattern Analysis and Machine Intelligence, 5 (1997), pp. 476–491.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer
About this paper
Cite this paper
Máša, P., Kočka, T. (2006). Finding Optimal Decision Trees. In: Kłopotek, M.A., Wierzchoń, S.T., Trojanowski, K. (eds) Intelligent Information Processing and Web Mining. Advances in Soft Computing, vol 35. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-33521-8_17
Download citation
DOI: https://doi.org/10.1007/3-540-33521-8_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33520-7
Online ISBN: 978-3-540-33521-4
eBook Packages: EngineeringEngineering (R0)