Tight Combinatorial Generalization Bounds for Threshold Conjunction Rules
We propose a combinatorial technique for obtaining tight data dependent generalization bounds based on a splitting and connectivity graph (SC-graph) of the set of classifiers. We apply this approach to a parametric set of conjunctive rules and propose an algorithm for effective SC-bound computation. Experiments on 6 data sets from the UCI ML Repository show that SC-bound helps to learn more reliable rule-based classifiers as compositions of less overfitted rules.
Keywordscomputational learning theory generalization bounds permutational probability splitting-connectivity bounds rule induction
- 1.Boucheron, S., Bousquet, O., Lugosi, G.: Theory of classification: A survey of some recent advances. ESAIM: Probability and Statistics (9), 323–375 (2005)Google Scholar
- 2.Cohen, W.W.: Fast effective rule induction. In: Proc. of the 12th International Conference on Machine Learning, Tahoe City, CA, pp. 115–123. Morgan Kaufmann, San Francisco (1995)Google Scholar
- 3.Cohen, W.W., Singer, Y.: A simple, fast and effective rule learner. In: Proc. of the 16 National Conference on Artificial Intelligence, pp. 335–342 (1999)Google Scholar
- 6.Quinlan, J.R.: C4.5: Programs for machine learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
- 7.Quinlan, J.R.: Bagging, boosting, and C4.5. AAAI/IAAI 1, 725–730 (1996)Google Scholar