Abstract
This chapter studies the effects on decision tree learning of constructing four types of new feature (conjunctive, disjunctive, M-of-N, and X-of-N representations). To reduce effects of other factors such as tree learning methods, new feature search strategies, search starting points, evaluation functions, and stopping criteria, a single tree learning algorithm is developed. With different option settings, it can construct four different types of new feature, but all other factors are fixed. The study reveals that conjunctive and disjunctive representations have very similar performance in terms of prediction accuracy and theory complexity on a variety of concepts, even on DNF and CNF concepts that are usually thought to be suited only to one of the two kinds of representation. In addition, the study demonstrates that the stronger representation power of M-of-N than conjunction and disjunction and the stronger representation power of X-of-N than these three types of new feature can be reflected in the performance of decision tree learning in terms of higher prediction accuracy and lower theory complexity.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bloedorn, E., Michalski, R.S., and Wnek, J. (1993). Multistrategy constructive induction: AQ17-MCI. Proceedings of the Second International Workshop on Multistrategy Learning, pages 188–203.
Breiman, L., Friedman, J H, Olshen, R.A., and Stone, R.A. (1984). Classification And Regression Trees. Wadsworth, Belmont, CA.
Catlett, J. (1991). On changing continuous attributes into ordered discrete attributes. Proceedings of the Fifth European Working Session on Learning, pages 164–178. Springer-Verlag, Berlin.
Fayyad, U.M. and Irani, K.B. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pages 1022–1027. Morgan Kaufmann, San Mateo, CA.
Matheus, C.J. (1989). Feature Construction: An Analytic Framework and an Application to Decision Trees, Ph.D. Thesis, Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL.
Merz, C.J. and Murphy, P.M. (1997). UCI Repository of Machine Learning Databases http://www.ics.uci.edu/~mlearn/MLRepository.html. Department of Information and Computer Science, University of California, Irvine, CA.
Michalski, R.S. (1980). Pattern recognition as rule-guided inductive inference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2:349–361.
Murphy, P.M. and Pazzani, M.J. (1991). ID2-of-3: Constructive induction of Mof-N concepts for discriminators in decision trees. Proceedings of the Eighth International Workshop on Machine Learning, pages 183–187. Morgan Kaufmann, San Mateo, CA.
Pagallo, G. (1990). Adaptive Decision Tree Algorithms for Learning from Examples, Ph.D. Thesis, University of California at Santa Cruz, Santa Cruz, CA.
Quinlan, J.R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA.
Ragavan, H. and Rendell, L. (1993). Lookahead feature construction for learning hard concepts. Proceedings of the Tenth International Conference on Machine Learning, pages 252–259. Morgan Kaufmann, San Mateo, CA.
Spackman, K.A. (1988). Learning categorical decision criteria in biomedical domains. Proceedings of the Fifth International Conference on Machine Learning, pages 36–46. Morgan Kaufmann, San Mateo, CA.
Van de Merckt, T. (1993). Decision trees in numerical attribute spaces. Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pages 1016–1021. Morgan Kaufmann, San Mateo, CA.
Vapnik, V.N. and Chervonenkis, A.Y. (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theor. Probab. Appl.,16:264–280.
Wnek, J. and Michalski, R.S. (1994). Hypothesis-driven constructive induction in AQ17-HcI: A method and experiments. Machine Learning,14:139–168.
Yang, D., Rendell, L., and Blix, G. (1991). A scheme for feature construction and a comparison of empirical methods. Proceedings of the Twelfth International Joint Conference on Artificial Intelligence, pages 699–704. Morgan Kaufmann, San Mateo, CA.
Zheng, Z. (1995a). Constructing nominal X-of-N attributes. Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pages 1064–1070. Morgan Kaufmann, San Mateo, CA.
Zheng, Z. (1995b). Continuous-valued X-of-N attributes versus nominal X-of-N attributes for constructive induction: A case study. Proceedings of the Fourth International Conference for Young Computer Scientists,pages 566–573. Peking University Press, Beijing.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer Science+Business Media New York
About this chapter
Cite this chapter
Zheng, Z. (1998). A Comparison of Constructing Different Types of New Feature For Decision Tree Learning. In: Liu, H., Motoda, H. (eds) Feature Extraction, Construction and Selection. The Springer International Series in Engineering and Computer Science, vol 453. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-5725-8_15
Download citation
DOI: https://doi.org/10.1007/978-1-4615-5725-8_15
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4613-7622-4
Online ISBN: 978-1-4615-5725-8
eBook Packages: Springer Book Archive