Abstract
In machine-learning, maximizing the sample margin can reduce the learning generalization-error. Thus samples on which the target function has a large margin (γ) convey more information so we expect fewer such samples. In this paper, we estimate the complexity of a class of sets of large-margin samples for a general learning problem over a finite domain. We obtain an explicit dependence of this complexity on γ and the sample size.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anthony, M., Bartlett, P.L.: Neural Network Learning: Theoretical Foundations. Cambridge University Press, UK (1999)
Antos, A., Kgl, B., Linder, T., Lugosi, G.: Data-dependent margin-based generalization bounds for classification. Journal of Machine Learning Research 3, 73–98 (2002)
Bartlett, P.L., Boucheron, S., Lugosi, G.: Model selection and error estimation. Machine Learning 48, 85–113 (2002)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and other Kernel-based learning methods. Cambridge University Press, UK (2000)
Haussler, D.: Decision theoretic generalizations of the PAC model for neural net and other learning applications. Information and Computation 100(1), 78–150 (1992)
Sauer, N.: On the density of families of sets. J. Combinatorial Theory (A) 13, 145–147 (1972)
Ratsaby, J.: A constrained version of Sauer’s Lemma. In: Proc. of Third Colloquium on Mathematics and Computer Science Algorithms, Trees, Combinatorics and Probabilities (MathInfo 2004), Vienna, Austria, September 2004. Birkhäuser, Basel (2004)
Ratsaby, J.: A Sharp Threshold Result for Finite-VC Classes of Large- Margin Functions, Department of Computer Science Technical Report RN/04/06, University College London (2004)
Shawe-Taylor, J., Bartlett, P.L., Williamson, R.C., Anthony, M.: Structural risk minimization over data-dependent hierarchies. IEEE Trans. Inf. Theory 44, 1926–1940 (1998)
Vapnik, V.N.: Statistical Learning Theory. Wiley, Chichester (1998)
Vapnik, V.N., Chervonenkis, A.: On the uniform convergence of relative frequencies of events to their probabilities. Theoret. Probl. Appl. 16, 264–280 (1971)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ratsaby, J. (2004). On the Complexity of Samples for Learning. In: Chwa, KY., Munro, J.I.J. (eds) Computing and Combinatorics. COCOON 2004. Lecture Notes in Computer Science, vol 3106. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-27798-9_23
Download citation
DOI: https://doi.org/10.1007/978-3-540-27798-9_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22856-1
Online ISBN: 978-3-540-27798-9
eBook Packages: Springer Book Archive