Abstract
We propose a combinatorial technique for obtaining tight data dependent generalization bounds based on a splitting and connectivity graph (SC-graph) of the set of classifiers. We apply this approach to a parametric set of conjunctive rules and propose an algorithm for effective SC-bound computation. Experiments on 6 data sets from the UCI ML Repository show that SC-bound helps to learn more reliable rule-based classifiers as compositions of less overfitted rules.
Supported by Russian Foundation for Basic Research grant 11-07-00480 and the program “Algebraical and combinatorial methods of cybernetics and new generation information systems” of Russian Academy of Sciences, Branch of Mathematics.
Chapter PDF
Similar content being viewed by others
Keywords
References
Boucheron, S., Bousquet, O., Lugosi, G.: Theory of classification: A survey of some recent advances. ESAIM: Probability and Statistics (9), 323–375 (2005)
Cohen, W.W.: Fast effective rule induction. In: Proc. of the 12th International Conference on Machine Learning, Tahoe City, CA, pp. 115–123. Morgan Kaufmann, San Francisco (1995)
Cohen, W.W., Singer, Y.: A simple, fast and effective rule learner. In: Proc. of the 16 National Conference on Artificial Intelligence, pp. 335–342 (1999)
Fürnkranz, J., Flach, P.A.: Roc ‘n’ rule learning-towards a better understanding of covering algorithms. Machine Learning 58(1), 39–77 (2005)
Martin, J.K.: An exact probability metric for decision tree splitting and stopping. Machine Learning 28(2-3), 257–291 (1997)
Quinlan, J.R.: C4.5: Programs for machine learning. Morgan Kaufmann, San Francisco (1993)
Quinlan, J.R.: Bagging, boosting, and C4.5. AAAI/IAAI 1, 725–730 (1996)
Vorontsov, K.V.: Combinatorial probability and the tightness of generalization bounds. Pattern Recognition and Image Analysis 18(2), 243–259 (2008)
Vorontsov, K.V.: Splitting and similarity phenomena in the sets of classifiers and their effect on the probability of overfitting. Pattern Recognition and Image Analysis 19(3), 412–420 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Vorontsov, K., Ivahnenko, A. (2011). Tight Combinatorial Generalization Bounds for Threshold Conjunction Rules. In: Kuznetsov, S.O., Mandal, D.P., Kundu, M.K., Pal, S.K. (eds) Pattern Recognition and Machine Intelligence. PReMI 2011. Lecture Notes in Computer Science, vol 6744. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21786-9_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-21786-9_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21785-2
Online ISBN: 978-3-642-21786-9
eBook Packages: Computer ScienceComputer Science (R0)