Skip to main content
SpringerLink
Log in

A View of Computational Learning Theory

  • Chapter
Foundations of Knowledge Acquisition

Part of the book series: The Springer International Series in Engineering and Computer Science ((SECS,volume 195))

Abstract

The distribution-free or “pac” approach to machine learning is described. The motivations, basic definitions and some of the more important results in this theory are summarized.

Research at Harvard was supported in part by the National Science Foundation NSF-CCR-89-02500, the Office for Naval Research ONR-N0014-85-K-0445, the Center for Intelligent Control ARO DAAL 03-86-K-0171 and by DARPA AFOSR 89-0506. This article appeared also in “Computation and Cognition”, C.W. Gear (ed.), SIAM, Philadelphia (1991) 32–53.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Abe, N. (1989). Polynomial learnability of semilinear sets. In Proceedings of the 2nd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 25–40.

    Google Scholar 

  • Angluin, D. (1987a). Learning regular sets from queries and counter examples. Information and Computation, 75:87–106.

    Article  MATH  MathSciNet  Google Scholar 

  • Angluin, D. (1987b). Queries and concept learning. Machine Learning, 2:319–342.

    Google Scholar 

  • Angluin, D., Hellerstein, L., & Karpinski, M. (1989). Learning read-once formulas with queries (Technical Report Rept. No. UCB/CSD 89/528). Computer Science Division and University of California and Berkeley.

    Google Scholar 

  • Angluin, D. & Laird, P. (1987). Learning from noisy examples. Machine Learning, 2:343–370.

    Google Scholar 

  • Baum, E. (1990a). The perceptron algorithm is fast for non-malicious distributions. Neural Computation, 2:249–261.

    Google Scholar 

  • Baum, E. (1990b). A polynomial time algorithm that learns two hidden unit nets. In Proceedings of the 3rd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA.

    Google Scholar 

  • Baum, E. (1990c). When are k-nearest neighbor and back propagation accurate for feasible sized sets of examples? Lecture Notes in Computer Science, 412:2–25.

    MathSciNet  Google Scholar 

  • Baum, E. & Haussler, D. (1989). What size net gives valid generalization. Neural Computation, 1(1):151–160.

    Article  Google Scholar 

  • Ben-David, S., Benedek, G., & Mansour, Y. (1989). A parametrization scheme for classifying models of learnability. In Proceedings of the 2nd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 285–302.

    Google Scholar 

  • Benedek, G. & Itai, A. (1987). Nonuniform learnability, (Technical Report TR 474). Computer Science Department, Technion, Haifa, Israel.

    Google Scholar 

  • Benedek, G. M. & Itai, A. (1988). Learnability by fixed distributions. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 80–90.

    Google Scholar 

  • Berman, P. & Roos, R. (1987). Learning one-counter languages in polynomial time. In Proceedings of the 28th IEEE Symposium on Foundations of Computer Science, IEEE Computer Society Press, Washington, D.C., 61–67.

    Google Scholar 

  • Blum, A. (1990a). Learning boolean functions in an infinite attribute space. In Proceedings of the 22nd ACM Symposium on Theory of Computing, The Association for Computing Machinery, New York, NY.

    Google Scholar 

  • Blum, A. (1990b). Separating distribution-free and mistake-bound learning models over the boolean domain. In Proceedings of the 31st IEEE Symposium on Foundation of Computer Science, IEEE Computer Society Press, Washington, D.C., 211–218.

    Google Scholar 

  • Blum, A. & Rivest, R. (1988). Training a 3-node neural network is NP-complete. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 9–18.

    Google Scholar 

  • Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. (1987). Occam’s razor. Information Proc. Letters, 25:377–380.

    Article  MathSciNet  Google Scholar 

  • Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. (1989). Learnability and the Vapnik-Chervonenkis dimension. J. ACM, 36(2):929–965.

    Article  MATH  MathSciNet  Google Scholar 

  • Board, R. & Pitt, L. (1990). On the necessity of Occam algorithms. In Proceedings of the 22nd ACM Symposium on Theory of Computing, The Association for Computing Machinery, New York, NY.

    Google Scholar 

  • Boucheron, S. & Sallantin, J. (1988). Some remarks about space-complexity of learning, and circuit complexity of recognizing. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 125–138.

    Google Scholar 

  • Dietterich, T. (1990). Machine learning. Ann. Rev. of Comp. Sci., 4.

    Google Scholar 

  • Ehrenfeucht, A. & Haussler, D. (1989). Learning decision trees from random examples. Inf. and Computation, 231–247.

    Google Scholar 

  • Ehrenfeucht, A., Haussler, D., Kearns, M., & Valiant, L. (1989). A general lower bound on the number of examples needed for learning. Inf. and Computation, 247–261.

    Google Scholar 

  • Floyd, S. (1989). Space-bounded learning and the Vapnik-Chervonenkis dimension. Proceedings of the 2nd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 349–364.

    Google Scholar 

  • Freund, Y. (1990). Boosting a weak learning algorithm by majority. Proceedings of the 3rd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA.

    Google Scholar 

  • Geréb-Graus, M. (1989). Lower Bounds on Parallel, Distributed and Automata Computations. (PhD thesis, Harvard University).

    Google Scholar 

  • Goldman, S., Rivest, R., & Schapire, R. (1989). Learning binary relations and total orders. In Proceedings of the 30th IEEE Symposium on Foundations of Computer Science, IEEE Computer Society Press, Washington, D.C., 46–53.

    Chapter  Google Scholar 

  • Goldreich, O., Goldwasser, S., & Micali, S. (1986). How to construct random functions. J. ACM, 33(4):792–807.

    Article  MathSciNet  Google Scholar 

  • Gu, Q. & Maruoka, A. (1988). Learning monotone boolean functions by uniform distributed examples. Manuscript.

    Google Scholar 

  • Hancock, T. (1990). Identifying μ-formula decision trees with queries. In Proceedings of the 2nd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA.

    Google Scholar 

  • Haussler, D. (1987). Bias, version spaces and Valiant’s learning frame-work. In Proc. 4th Intl. Workshop on Machine Learning, Morgan Kaufmann, 324–336

    Google Scholar 

  • Haussler, D. (1988). Quantifying inductive bias: AI learning algorithms and Valiant’s learning framework. Artificial Intelligence, 36(2):177–222.

    Article  MATH  MathSciNet  Google Scholar 

  • Haussler, D. (1990). Learning conjunctive concepts in structural domains. Machine Learning, 4.

    Google Scholar 

  • Haussler, D., Kearns, M., Littlestone, N., & Warmuth, M. (1988a). Equivalence of models of polynomial learnability. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 42–55.

    Google Scholar 

  • Haussler, D., Littlestone, N., & Warmuth, M. (1988b). Predicting 0,1-functions on randomly drawn points. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 280–296.

    Google Scholar 

  • Helmbold, D., Sloan, R., & Warmuth, M. (1989). Learning nested differences of intersection-closed concept classes. In Proceedings of the 2nd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 41–56.

    Google Scholar 

  • Helmbold, D., Sloan, R., & Warmuth, M. (1990). Learning integer lattices. In Proceedings of the 3rd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA.

    Google Scholar 

  • Judd, J. (1988). Learning in neural nets. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 2–8.

    Google Scholar 

  • Kearns, M. (1990). The Computational Complexity of Machine Learning. MIT Press.

    Google Scholar 

  • Kearns, M. & Li, M. (1988). Learning in the presence of malicious errors. In Proceedings of the 20th ACM Symposium on Theory of Computing, The Association for Computing Machinery, New York, NY, 267–279.

    Google Scholar 

  • Kearns, M., Li, M., Pitt, L., & Valiant, L. (1987a). On the learnability of Boolean formulae. In Proceedings of the 19th ACM Symposium on Theory of Computing, The Association for Computing Machinery, New York, NY, 285–295.

    Google Scholar 

  • Kearns, M., Li, M., Pitt, L., & Valiant, L. (1987b). Recent results on Boolean concept learning. In Proc. 4th Int. Workshop on Machine Learning, Los Altos, CA. Morgan Kaufmann, 337–352.

    Google Scholar 

  • Kearns, M., Li, M., & Valiant, L. (1989). Learning boolean formulae. Submitted for publication.

    Google Scholar 

  • Kearns, M. & Pitt, L. (1989). A polynomial-time algorithm for learning k-variable pattern languages from examples. In Proceedings of the 2nd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 57–71.

    Google Scholar 

  • Kearns, M. & Schapire, R. (1990). Efficient distribution-free learning of probabilistic concepts. In Proceedings of the 3rd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA.

    Google Scholar 

  • Kearns, M. & Valiant, L. (1989). Cryptographic limitations on learning boolean formulae and finite automata. In Proceedings of the 21st ACM Symposium on Theory of Computing, The Association for Computing Machinery, New York, NY, 433–444.

    Google Scholar 

  • Kivinen, J. (1989). Reliable and useful learning. In Proceedings of the 2nd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 365–380.

    Google Scholar 

  • Kucera, L., Marchetti-Spaccamela, A., & Protasi, M. (1988). On the learnability of dnf formulae. In ICALP, 347–361.

    Google Scholar 

  • Laird, P. (1989). A survey of computational learning theory (Technical Report RIA-89-01-07-0), NASA, Ames Research Center.

    Google Scholar 

  • Li, M. & Vazirani, U. (1988). On the learnability of finite automata. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 359–370.

    Google Scholar 

  • Li, M. & Vitanyi, P. (1989). A theory of learning simple concepts under simple distributions and average case complexity for the universal distribution. In Proceedings of the 30th IEEE Symposium on Foundations of Computer Science, IEEE Computer Society Press, Washington, D.C., 34–39.

    Chapter  Google Scholar 

  • Lin, J.-H. & Vitter, S. (1989). Complexity issues in learning by neural nets. In Proceedings of the 2nd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 118–133.

    Google Scholar 

  • Linial, N., Mansour, Y., & Nisan, N. (1989). Constant depth circuits, Fourier transforms and learnability. In Proceedings of the 30th IEEE Symposium on Foundations of Computer Science, IEEE Computer Society Press, Washington, D.C., 574–579.

    Chapter  Google Scholar 

  • Linial, N., Mansour, Y., & Rivest, R. (1988). Results on learnability and the Vapnik-Chervonenkis dimension. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 56–68.

    Google Scholar 

  • Littlestone, N. (1988). Learning quickly when irrelevant attributes abound: a new linear threshold algorithm. Machine Learning, 2(4):245–318.

    Google Scholar 

  • Littlestone, N. (1989). From on-line to batch learning. In Proceedings of the 2nd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 269–284.

    Google Scholar 

  • Megiddo, N. (1986). On the complexity of polyhedral separability, (Technical Report RJ 5252), IBM Almaden Research Center.

    Google Scholar 

  • Minsky, M. & Papert, S. (1988). Perceptrons: an introduction to computational geometry. MIT Press.

    Google Scholar 

  • Natarajan, B. (1987). On learning boolean functions. In Proceedings of the 19th ACM Symposium on Theory of Computing, The Association for Computing Machinery, New York, NY, 296–304.

    Google Scholar 

  • Natarajan, B. (1990). Probably approximate learning over classes of distributions. Manuscript.

    Google Scholar 

  • Ohguro, T. & Maruoka, A. (1989). A learning algorithm for monotone k-term dnf. In Fujitsu HAS-SIS Workshop on Computational Learning Theory.

    Google Scholar 

  • Paturi, R., Rajasekaran, S., & Reif, J. (1989). The light bulb problem. In Proceedings of the 2nd Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 261–268.

    Google Scholar 

  • Pitt, L. (1989). Inductive inference, dfas and computational complexity. In Jantke, K., (editor), Analogical and Indictive Inference. Lecture Notes in Computer Science, Vol. 397, pp.(18–44) Spring-Verlag.

    Google Scholar 

  • Pitt, L. & Valiant, L. (1988). Computational limitations on learning from examples. J. ACM, 35(4):965–984.

    Article  MATH  MathSciNet  Google Scholar 

  • Pitt, L. & Warmuth, M. (1988). Reductions among prediction problems: on the difficulty of predicting automata. In Proc. 3rd IEEE Conf. on Structure in Complexity Theory, 60–69.

    Google Scholar 

  • Pitt, L. & Warmuth, M. (1989). The minimal consistent dfa problem cannot be approximated within any polynomial. In Proceedings of the 21st ACM Symposium on Theory of Computing, The Association for Computing Machinery, New York, NY, 421–432.

    Google Scholar 

  • Rivest, R. (1987). Learning decision lists. Machine Learning, 2(3):229–246.

    Google Scholar 

  • Rivest, R. & Sloan, R. (1988). Learning complicated concepts reliably and usefully. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 69–79.

    Google Scholar 

  • Rivest, R. L. & Schapire, R. (1987). Diversity-based inference of finite automata. In Proceedings of the 28th IEEE Symposium on Foundations of Computer Science, IEEE Computer Society Press, Washington, D.C., 78–88.

    Google Scholar 

  • Rivest, R. L. & Schapire, R. (1989). Inference of finite automata using homing sequences. In Proceedings of the 21st ACM Symposium on Theory of Computing, The Association for Computing Machinery, New York, NY, 411–420.

    Google Scholar 

  • Rosenblatt, F. (1961). Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books, Washington, D.C.

    Google Scholar 

  • Sakakibara, Y. (1988). Learning context-free grammars from structural data in polynomial time. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 330–344.

    Google Scholar 

  • Schapire, R. (1989). On the strength of weak learnability. In Proceedings of the 30th IEEE Symposium on Foundations of Computer Science, IEEE Computer Society Press, Washington, D.C., 28–33.

    Chapter  Google Scholar 

  • Shackelford, G. & Volper, D. (1988). Learning k-dnf with noise in the attributes. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 97–105.

    Google Scholar 

  • Shvaytser, H. (1990). A necessary condition for learning from positive examples. Machine Learning, 5:101–113.

    Google Scholar 

  • Sloan, R. (1988). Types of noise for concept learning. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 91–96.

    Google Scholar 

  • Valiant, L. (1984). A theory of the learnable. Comm. ACM, 27(11):1134–1142.

    Article  MATH  Google Scholar 

  • Valiant, L. (1985). Learning disjunctions of conjunctions. In Proc. 9th Int. Joint Conf on Artificial Intelligence, 560–566, Los Altos, CA. Morgan Kaufmann.

    Google Scholar 

  • Valiant, L. (1988). Functionality in neural nets. In Proc. Amer. Assoc. for Artificial Intelligence, 629–634, San Mateo, CA. Morgan Kaufmann.

    Google Scholar 

  • Vapnik, V. (1982). Estimation of dependencies based on Empirical Data. Springer-Verlag.

    Google Scholar 

  • Vapnik, V. & Chervonenkis, A. Y. (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theor. Probability and Appl, 16(2):264–280.

    Article  MATH  MathSciNet  Google Scholar 

  • Vitter, J. & Lin, J.-H. (1988). Learning in parallel. In Proceedings of Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 106–124.

    Google Scholar 

  • Warmuth, M. (1989). Toward representation independence in pac learning. In Jantke, K., (editor), Analogical and Inductive Inference, vol 397, Lecture Notes in Computer Science, 78–103. Springer-Verlag.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Harvard University and NEC Research Institute, USA

    Leslie G. Valiant

Authors
  1. Leslie G. Valiant
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Naval Research Laboratory, USA

    Alan L. Meyrowitz

  2. Office of Naval Research, USA

    Susan Chipman

Rights and permissions

Reprints and permissions

Copyright information

© 1993 Kluwer Academic Publishers

About this chapter

Cite this chapter

Valiant, L.G. (1993). A View of Computational Learning Theory. In: Meyrowitz, A.L., Chipman, S. (eds) Foundations of Knowledge Acquisition. The Springer International Series in Engineering and Computer Science, vol 195. Springer, Boston, MA. https://doi.org/10.1007/978-0-585-27366-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-0-585-27366-2_8

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-7923-9278-1

  • Online ISBN: 978-0-585-27366-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation