Advertisement

A Metric Entropy Bound is Not Sufficient for Learnability

  • R. M. Dudley
  • S. R. Kulkarni
  • T. Richardson
  • O. Zeitouni
Chapter
Part of the Selected Works in Probability and Statistics book series (SWPS)

Abstract

We prove by means of a counterexample that it is not sufficient, for probably approximately correct (PAC) learning under a class of distributions, to have a uniform bound on the metric entropy of the class of concepts to be learned. This settles a conjecture of Benedek and Itai.

Index Terms

Learning estimation PAC metric entropy class of distributions 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    L. G. Valiant, “A theory of the learnable,” Commun. ACM, vol. 27, no. 11, pp. 1134–1142, 1984.MATHCrossRefGoogle Scholar
  2. 2.
    A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Learnability and the Vapnik–Chervonenkis dimension,” J. ACM, vol. 36, no. 4, pp. 929–965, 1989.MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    D. Haussler, “Decision theoretic generalizations of the PAC model for neural net and other learning applications,” Inf. Comput., vol. 20, pp. 78–150, 1992.CrossRefMathSciNetGoogle Scholar
  4. 4.
    G. M. Benedek and A. Itai, “Learnability with respect to a fixed distribution,” Theor. Comput. Sci., vol. 86, pp. 377–389, 1991.MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    V. N. Vapnik, Estimation of Dependences Based on Empirical Data. New York: Springer-Verlag, 1982.MATHGoogle Scholar
  6. 6.
    R. M. Dudley, “A course on empirical processes,” Lecture Notes in Math. Vol. 1097. New York: Springer, 1984, pp. 1–142.Google Scholar
  7. 7.
    V. N. Vapnik and A. Ya. Chervonenkis, “On the uniform convergence of relative frequencies of events to their probabilities,” Theory Probab. Its Appl., vol. 16, no. 2, pp. 264–280, 1971.MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    V. N. Vapnik and A. Ya. Chervonenkis, “Necessary and sufficient conditions for the uniform convergence of means to their expectations,” Theory Probab. Its Appl., vol. 26, no. 3, pp. 532–553, 1981.CrossRefMathSciNetGoogle Scholar
  9. 9.
    S. R. Kulkarni, “Problems of computational and information complexity in machine vision and learning,” Ph.D. thesis, Dep. Elec. Eng. Comput. Sci., M.I.T., June 1991.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • R. M. Dudley
    • 1
  • S. R. Kulkarni
    • 2
  • T. Richardson
    • 3
  • O. Zeitouni
    • 4
  1. 1.Dept. of MathematicsM.I.TCambridgeUSA
  2. 2.Dept. of Electrical EngineeringPrinceton UniversityPrincetonUSA
  3. 3.AT & T Bell LabsMurray HillUSA
  4. 4.Dept. of Electrical EngineeringTechnionHaifaIsrael

Personalised recommendations