A Metric Entropy Bound is Not Sufficient for Learnability

  • R. M. Dudley
  • S. R. Kulkarni
  • T. Richardson
  • O. Zeitouni
Part of the Selected Works in Probability and Statistics book series (SWPS)


We prove by means of a counterexample that it is not sufficient, for probably approximately correct (PAC) learning under a class of distributions, to have a uniform bound on the metric entropy of the class of concepts to be learned. This settles a conjecture of Benedek and Itai.

Index Terms

Learning estimation PAC metric entropy class of distributions 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    L. G. Valiant, “A theory of the learnable,” Commun. ACM, vol. 27, no. 11, pp. 1134–1142, 1984.zbMATHCrossRefGoogle Scholar
  2. 2.
    A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Learnability and the Vapnik–Chervonenkis dimension,” J. ACM, vol. 36, no. 4, pp. 929–965, 1989.zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    D. Haussler, “Decision theoretic generalizations of the PAC model for neural net and other learning applications,” Inf. Comput., vol. 20, pp. 78–150, 1992.CrossRefMathSciNetGoogle Scholar
  4. 4.
    G. M. Benedek and A. Itai, “Learnability with respect to a fixed distribution,” Theor. Comput. Sci., vol. 86, pp. 377–389, 1991.zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    V. N. Vapnik, Estimation of Dependences Based on Empirical Data. New York: Springer-Verlag, 1982.zbMATHGoogle Scholar
  6. 6.
    R. M. Dudley, “A course on empirical processes,” Lecture Notes in Math. Vol. 1097. New York: Springer, 1984, pp. 1–142.Google Scholar
  7. 7.
    V. N. Vapnik and A. Ya. Chervonenkis, “On the uniform convergence of relative frequencies of events to their probabilities,” Theory Probab. Its Appl., vol. 16, no. 2, pp. 264–280, 1971.zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    V. N. Vapnik and A. Ya. Chervonenkis, “Necessary and sufficient conditions for the uniform convergence of means to their expectations,” Theory Probab. Its Appl., vol. 26, no. 3, pp. 532–553, 1981.CrossRefMathSciNetGoogle Scholar
  9. 9.
    S. R. Kulkarni, “Problems of computational and information complexity in machine vision and learning,” Ph.D. thesis, Dep. Elec. Eng. Comput. Sci., M.I.T., June 1991.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • R. M. Dudley
    • 1
  • S. R. Kulkarni
    • 2
  • T. Richardson
    • 3
  • O. Zeitouni
    • 4
  1. 1.Dept. of MathematicsM.I.TCambridgeUSA
  2. 2.Dept. of Electrical EngineeringPrinceton UniversityPrincetonUSA
  3. 3.AT & T Bell LabsMurray HillUSA
  4. 4.Dept. of Electrical EngineeringTechnionHaifaIsrael

Personalised recommendations