Advertisement

Pre-pruning Classification Trees to Reduce Overfitting in Noisy Domains

  • Max Bramer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2412)

Abstract

The automatic induction of classification rules from examples in the form of a classification tree is an important technique used in data mining. One of the problems encountered is the overfitting of rules to training data. In some cases this can lead to an excessively large number of rules, many of which have very little predictive value for unseen data. This paper describes a means of reducing overfitting known as J-pruning, based on the J-measure, an information theoretic means of quantifying the information content of a rule. It is demonstrated that using J-pruning generally leads to a substantial reduction in the number of rules generated and an increase in predictive accuracy. The advantage gained becomes more pronounced as the proportion of noise increases.

Keywords

Predictive Accuracy Classification Tree Classification Rule Categorical Attribute Unseen Data 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Hunt, E.B., Marin J. and Stone, P.J. (1966). Experiments in Induction. Academic PressGoogle Scholar
  2. 2.
    Quinlan, J.R. (1993). C4.5: Programs for Machine Learning. Morgan KaufmannGoogle Scholar
  3. 3.
    Bramer, M.A. (2002). An Information-Theoretic Approach to the Pre-pruning of Classification Rules. Proceedings of the IFIP World Computer Congress, Montreal 2002.Google Scholar
  4. 4.
    Blake, C.L. and Merz, C.J. (1998). UCI Repository of Machine Learning Databases [http://www.ics.uci.edu/~mlearn/MLRepository.html]. Irvine, CA: University of California, Department of Information and Computer ScienceGoogle Scholar
  5. 5.
    Mingers, J. (1989). An Empirical Comparison of Pruning Methods for Decision Tree Induction. Machine Learning, 4, pp. 227–243CrossRefGoogle Scholar
  6. 6.
    Bramer, M.A. (2002). Using J-Pruning to Reduce Overfitting in Classification Trees. In: Research and Development in Intelligent Systems XVIII. Springer-Verlag, pp. 25–38.Google Scholar
  7. 7.
    Smyth, P. and Goodman, R.M. (1991). Rule Induction Using Information Theory. In: Piatetsky-Shapiro, G. and Frawley, W.J. (eds.), Knowledge Discovery in Databases. AAAI Press, pp. 159–176Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Max Bramer
    • 1
  1. 1.Faculty of TechnologyUniversity of PortsmouthUK

Personalised recommendations