Advertisement

Some Local Measures of Complexity of Convex Hulls and Generalization Bounds

  • Olivier Bousquet
  • Vladimir Koltchinskii
  • Dmitriy Panchenko
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2375)

Abstract

We investigate measures of complexity of function classes based on continuity moduli of Gaussian and Rademacher processes. For Gaussian processes, we obtain bounds on the continuity modulus on the convex hull of a function class in terms of the same quantity for the class itself. We also obtain new bounds on generalization error in terms of localized Rademacher complexities. This allows us to prove new results about generalization performance for convex hulls in terms of characteristics of the base class. As a byproduct, we obtain a simple proof of some of the known bounds on the entropy of convex hulls.

Keywords

Convex Hull Function Class Local Measure Base Class Generalization Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    K. Ball and A. Pajor. The entropy of convex bodies with “few” extreme points. London MAth. Soc. Lectrure Note Ser. 158, pages 25–32, 1990.Google Scholar
  2. 2.
    P. Bartlett, O. Bousquet and S. Mendelson. Localized Rademacher Complexity. Preprint, 2002.Google Scholar
  3. 3.
    P. Bartlett, S. Mendelson. Rademacher and Gaussian complexities: risk bounds and structural results. In Proceeding of the 14th Annual Conference on Computational Learning Theory, Srpinger, 2001.Google Scholar
  4. 4.
    S. Boucheron, G. Lugosi and P. Massart. Concentration inequalities using the entropy method. Preprint, 2002.Google Scholar
  5. 5.
    O. Bousquet. A Bennett concentration inequality and its application to empirical processes. C. R. Acad. Sci. Paris, Ser. I 334, pages 495–500, 2002.zbMATHMathSciNetGoogle Scholar
  6. 6.
    B. Carl. Metric entropy of convex hulls in Hilbert spaces. Bulletin of the London Mathematical Society, 29, pages 452–458, 1997.zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    B. Carl, I. Kyrezi and A. Pajor. Metric entropy of convex hulls in Banach spaces. Journal of the London Mathematical Society, 2001.Google Scholar
  8. 8.
    J. Creutzig and I. Steinwart. Metric entropy of convex hulls in type p spaces — the critical case. 2001.Google Scholar
  9. 9.
    R. Dudley. Universal Donsker classes and metric entropy. Annals of Probability, 15, pages 1306–1326, 1987.zbMATHMathSciNetCrossRefGoogle Scholar
  10. 10.
    R. Dudley. Uniform central limit theorems. Cambridge University Press, 2000.Google Scholar
  11. 11.
    F. Gao. Metric entropy of convex hulls. Israel Journal of Mathematics, 123, pages 359–364, 2001.zbMATHMathSciNetCrossRefGoogle Scholar
  12. 12.
    E. Giné and J. Zinn. Gaussian characterization of uniform Donsker classes of functions. Annals of Probability, 19, pages 758–782, 1991.zbMATHMathSciNetCrossRefGoogle Scholar
  13. 13.
    V. I. Koltchinskii and D. Panchenko. Rademacher processes and bounding the risk of function learning. In High Dimensional Probability II, Eds. E. Gine, D. Mason and J. Wellner, pp. 443–459, 2000.Google Scholar
  14. 14.
    V. I. Koltchinskii and D. Panchenko. Empirical margin distributions and bounding the generalization error of combined classifiers. Annals of Statistics, 30(1), 2002.Google Scholar
  15. 15.
    M. Ledoux and M. Talagrand Probability in Banach spaces. Springer-Verlag, 1991.Google Scholar
  16. 16.
    W. Li and W. Linde. Metric entropy of convex hulls in Hilbert spaces. Preprint, 2001.Google Scholar
  17. 17.
    M. Lifshits. Gaussian random functions. Kluwer, 1995.Google Scholar
  18. 18.
    P. Massart. Some applications of concentration inequalities to statistics. Annales de la Faculté des Sciences de Toulouse, IX, pages 245–303, 2000.MathSciNetGoogle Scholar
  19. 19.
    S. Mendelson. On the size of convex hulls of small sets. Preprint, 2001.Google Scholar
  20. 20.
    S. Mendelson. Improving the sample complexity using global data. Preprint, 2001.Google Scholar
  21. 21.
    A. van der Vaart and J. Wellner. Weak convergence and empirical processes with applications to statistics. John Wiley & Sons, New York, 1996.zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Olivier Bousquet
    • 1
  • Vladimir Koltchinskii
    • 2
  • Dmitriy Panchenko
    • 2
  1. 1.Centre de Mathématiques AppliquéesEcole PolytechniquePalaiseauFrance
  2. 2.Department of Mathematics and StatisticsThe University of New MexicoAlbuquerqueUSA

Personalised recommendations