Abstract
We investigate measures of complexity of function classes based on continuity moduli of Gaussian and Rademacher processes. For Gaussian processes, we obtain bounds on the continuity modulus on the convex hull of a function class in terms of the same quantity for the class itself. We also obtain new bounds on generalization error in terms of localized Rademacher complexities. This allows us to prove new results about generalization performance for convex hulls in terms of characteristics of the base class. As a byproduct, we obtain a simple proof of some of the known bounds on the entropy of convex hulls.
Partially supported by NSA Grant MDA904-99-1-0031
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
K. Ball and A. Pajor. The entropy of convex bodies with “few” extreme points. London MAth. Soc. Lectrure Note Ser. 158, pages 25–32, 1990.
P. Bartlett, O. Bousquet and S. Mendelson. Localized Rademacher Complexity. Preprint, 2002.
P. Bartlett, S. Mendelson. Rademacher and Gaussian complexities: risk bounds and structural results. In Proceeding of the 14th Annual Conference on Computational Learning Theory, Srpinger, 2001.
S. Boucheron, G. Lugosi and P. Massart. Concentration inequalities using the entropy method. Preprint, 2002.
O. Bousquet. A Bennett concentration inequality and its application to empirical processes. C. R. Acad. Sci. Paris, Ser. I 334, pages 495–500, 2002.
B. Carl. Metric entropy of convex hulls in Hilbert spaces. Bulletin of the London Mathematical Society, 29, pages 452–458, 1997.
B. Carl, I. Kyrezi and A. Pajor. Metric entropy of convex hulls in Banach spaces. Journal of the London Mathematical Society, 2001.
J. Creutzig and I. Steinwart. Metric entropy of convex hulls in type p spaces — the critical case. 2001.
R. Dudley. Universal Donsker classes and metric entropy. Annals of Probability, 15, pages 1306–1326, 1987.
R. Dudley. Uniform central limit theorems. Cambridge University Press, 2000.
F. Gao. Metric entropy of convex hulls. Israel Journal of Mathematics, 123, pages 359–364, 2001.
E. Giné and J. Zinn. Gaussian characterization of uniform Donsker classes of functions. Annals of Probability, 19, pages 758–782, 1991.
V. I. Koltchinskii and D. Panchenko. Rademacher processes and bounding the risk of function learning. In High Dimensional Probability II, Eds. E. Gine, D. Mason and J. Wellner, pp. 443–459, 2000.
V. I. Koltchinskii and D. Panchenko. Empirical margin distributions and bounding the generalization error of combined classifiers. Annals of Statistics, 30(1), 2002.
M. Ledoux and M. Talagrand Probability in Banach spaces. Springer-Verlag, 1991.
W. Li and W. Linde. Metric entropy of convex hulls in Hilbert spaces. Preprint, 2001.
M. Lifshits. Gaussian random functions. Kluwer, 1995.
P. Massart. Some applications of concentration inequalities to statistics. Annales de la Faculté des Sciences de Toulouse, IX, pages 245–303, 2000.
S. Mendelson. On the size of convex hulls of small sets. Preprint, 2001.
S. Mendelson. Improving the sample complexity using global data. Preprint, 2001.
A. van der Vaart and J. Wellner. Weak convergence and empirical processes with applications to statistics. John Wiley & Sons, New York, 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bousquet, O., Koltchinskii, V., Panchenko, D. (2002). Some Local Measures of Complexity of Convex Hulls and Generalization Bounds. In: Kivinen, J., Sloan, R.H. (eds) Computational Learning Theory. COLT 2002. Lecture Notes in Computer Science(), vol 2375. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45435-7_5
Download citation
DOI: https://doi.org/10.1007/3-540-45435-7_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43836-6
Online ISBN: 978-3-540-45435-9
eBook Packages: Springer Book Archive