Skip to main content
SpringerLink
Log in

A Metric Entropy Bound is Not Sufficient for Learnability

  • Chapter
  • First Online:
Selected Works of R.M. Dudley

Part of the book series: Selected Works in Probability and Statistics ((SWPS))

Abstract

We prove by means of a counterexample that it is not sufficient, for probably approximately correct (PAC) learning under a class of distributions, to have a uniform bound on the metric entropy of the class of concepts to be learned. This settles a conjecture of Benedek and Itai.

Manuscript received No. 9, 1992; revised Sept. 22, 1993. The research of R. M. Dudley was partially supported by National Sciences Foundation grants. The work of S. Kulkarni was supported in part by the Army Research Office under Grant DAAL03-91-G-0320 and by the National Science Foundation under Grant IRI-92-09577. The work of O. Zeitouni was done while visiting the Center for Intelligent Control Systems at M.I.T. under support of the U.S. Army Research Office Grant DAAL03-92-G-0115.

IEEE Log Number 9401814.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. L. G. Valiant, “A theory of the learnable,” Commun. ACM, vol. 27, no. 11, pp. 1134–1142, 1984.

    Article  MATH  Google Scholar 

  2. A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Learnability and the Vapnik–Chervonenkis dimension,” J. ACM, vol. 36, no. 4, pp. 929–965, 1989.

    Article  MATH  MathSciNet  Google Scholar 

  3. D. Haussler, “Decision theoretic generalizations of the PAC model for neural net and other learning applications,” Inf. Comput., vol. 20, pp. 78–150, 1992.

    Article  MathSciNet  Google Scholar 

  4. G. M. Benedek and A. Itai, “Learnability with respect to a fixed distribution,” Theor. Comput. Sci., vol. 86, pp. 377–389, 1991.

    Article  MATH  MathSciNet  Google Scholar 

  5. V. N. Vapnik, Estimation of Dependences Based on Empirical Data. New York: Springer-Verlag, 1982.

    MATH  Google Scholar 

  6. R. M. Dudley, “A course on empirical processes,” Lecture Notes in Math. Vol. 1097. New York: Springer, 1984, pp. 1–142.

    Google Scholar 

  7. V. N. Vapnik and A. Ya. Chervonenkis, “On the uniform convergence of relative frequencies of events to their probabilities,” Theory Probab. Its Appl., vol. 16, no. 2, pp. 264–280, 1971.

    Article  MATH  MathSciNet  Google Scholar 

  8. V. N. Vapnik and A. Ya. Chervonenkis, “Necessary and sufficient conditions for the uniform convergence of means to their expectations,” Theory Probab. Its Appl., vol. 26, no. 3, pp. 532–553, 1981.

    Article  MathSciNet  Google Scholar 

  9. S. R. Kulkarni, “Problems of computational and information complexity in machine vision and learning,” Ph.D. thesis, Dep. Elec. Eng. Comput. Sci., M.I.T., June 1991.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Mathematics, M.I.T, Cambridge, MA, 02139, USA

    R. M. Dudley

  2. Dept. of Electrical Engineering, Princeton University, Princeton, NJ, 08544, USA

    S. R. Kulkarni

  3. AT & T Bell Labs, 600 Mountain Ave, Murray Hill, NJ, 07974, USA

    T. Richardson

  4. Dept. of Electrical Engineering, Technion, Haifa, 32000, Israel

    O. Zeitouni

Authors
  1. R. M. Dudley
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. R. Kulkarni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. T. Richardson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. O. Zeitouni
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. , Mathematics, University of Connecticut, Auditorium Road 196, Storrs, 06269-3009, USA

    Evarist Giné

  2. , Mathematics, Georgia Institute of Technology, Atlanta, 30332-0160, USA

    Vladimir Koltchinskii

  3. Inst. Mathematics & Informatics, Akademijos 4, Vilnius, LT-08663, Lithuania

    Rimas Norvaisa

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Dudley, R.M., Kulkarni, S.R., Richardson, T., Zeitouni, O. (2010). A Metric Entropy Bound is Not Sufficient for Learnability. In: Giné, E., Koltchinskii, V., Norvaisa, R. (eds) Selected Works of R.M. Dudley. Selected Works in Probability and Statistics. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-5821-1_28

Download citation

Publish with us

Policies and ethics

Search

Navigation