On PAC Learning Algorithms for Rich Boolean Function Classes

  • Rocco A. Servedio
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3959)


We survey the fastest known algorithms for learning various expressive classes of Boolean functions in the Probably Approximately Correct (PAC) learning model.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Angluin, D.: Learning Regular Sets from Queries and Counterexamples. Information and Computation 75(2), 87–106 (1987)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Aspnes, J., Beigel, R., Furst, M., Rudich, S.: The expressive power of voting polynomials. Combinatorica 14(2), 1–14 (1994)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Beigel, R., Reingold, N., Spielman, D.: PP is closed under intersection. Journal of Computer and System Sciences 50(2), 191–202 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Beimel, A., Bergadano, F., Bshouty, N., Kushilevitz, E., Varricchio, S.: Learning functions represented as multiplicity automata. J. ACM 47(3), 506–530 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Ben-David, S., Dichterman, E.: Learning with restricted focus of attention. Journal of Computer and System Sciences 56(3), 277–298 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Blum, A.: Rank-r decision trees are a subclass of r-decision lists. Information Processing Letters 42(4), 183–185 (1992)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Blum, A., Chalasani, P., Jackson, J.: On learning embedded symmetric concepts. In: Proceedings of the Sixth Annual Conference on Computational Learning Theory, pp. 337–346 (1993)Google Scholar
  8. 8.
    Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.: Occam’s razor. Information Processing Letters 24, 377–380 (1987)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.: Learnability and the Vapnik-Chervonenkis dimension. Journal of the ACM 36(4), 929–965 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Bshouty, N.: Exact learning via the monotone theory. Information and Computation 123(1), 146–153 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Bshouty, N.: A subexponential exact learning algorithm for DNF using equivalence queries. Information Processing Letters 59, 37–39 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Bshouty, N., Tamon, C.: On the Fourier spectrum of monotone functions. Journal of the ACM 43(4), 747–770 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Bshouty, N., Tamon, C., Wilson, D.: On learning width two branching programs. Information Processing Letters 65, 217–222 (1998)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Ehrenfeucht, A., Haussler, D.: Learning decision trees from random examples. Information and Computation 82(3), 231–246 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Fischer, P., Simon, H.U.: On learning ring-sum expansions. SIAM Journal on Computing 21(1), 181–192 (1992)zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Freund, Y.: Boosting a weak learning algorithm by majority. Information and Computation 121(2), 256–285 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55(1), 119–139 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Helmbold, D., Sloan, R., Warmuth, M.: Learning integer lattices. SIAM Journal on Computing 21(2), 240–266 (1992)zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Jackson, J.: An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. Journal of Computer and System Sciences 55, 414–440 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Jackson, J., Klivans, A., Servedio, R.: Learnability beyond AC0. In: Proceedings of the 34th ACM Symposium on Theory of Computing, pp. 776–784 (2002)Google Scholar
  21. 21.
    Kearns, M.: Efficient noise-tolerant learning from statistical queries. Journal of the ACM 45(6), 983–1006 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Kearns, M., Vazirani, U.: An introduction to computational learning theory. MIT Press, Cambridge (1994)Google Scholar
  23. 23.
    Khachiyan, L.: A polynomial algorithm in linear programming. Soviet Math. Dokl 20, 1093–1096 (1979)MathSciNetGoogle Scholar
  24. 24.
    Kharitonov, M.: Cryptographic hardness of distribution-specific learning. In: Proceedings of the Twenty-Fifth Annual Symposium on Theory of Computing, pp. 372–381 (1993)Google Scholar
  25. 25.
    Klivans, A., O’Donnell, R., Servedio, R.: Learning intersections and thresholds of halfspaces. Journal of Computer & System Sciences 68(4), 808–840 (2004) Preliminary version in Proc. of FOCS 2002Google Scholar
  26. 26.
    Klivans, A., Servedio, R.: Learning DNF in time \(2^{O(n^{1/3})}\). Journal of Computer & System Sciences 68(2), 303–318 (2004) Preliminary version in. Proc. STOC 2001Google Scholar
  27. 27.
    Krause, M., Pudlak, P.: Computing boolean functions by polynomials and threshold circuits. Computational Complexity 7(4), 346–370 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    Kushilevitz, E., Mansour, Y.: Learning decision trees using the Fourier spectrum. SIAM J. on Computing 22(6), 1331–1348 (1993)zbMATHCrossRefMathSciNetGoogle Scholar
  29. 29.
    Linial, N., Mansour, Y., Nisan, N.: Constant depth circuits, Fourier transform and learnability. Journal of the ACM 40(3), 607–620 (1993)zbMATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    Linial, N., Nisan, N.: Approximate inclusion-exclusion. Combinatorica 10(4), 349–365 (1990)zbMATHCrossRefMathSciNetGoogle Scholar
  31. 31.
    Minsky, M., Papert, S.: Perceptrons: an introduction to computational geometry. MIT Press, Cambridge (1968)Google Scholar
  32. 32.
    Muroga, S., Toda, I., Takasu, S.: Theory of majority switching elements. J. Franklin Institute 271, 376–418 (1961)zbMATHCrossRefMathSciNetGoogle Scholar
  33. 33.
    O’Donnell, R., Servedio, R.: New degree bounds for polynomial threshold functions. In: Proceedings of the 35th ACM Symposium on Theory of Computing, pp. 325–334 (2003)Google Scholar
  34. 34.
    O’Donnell, R., Servedio, R.: Learning monotone decision trees in polynomial time. Submitted for publication (2005)Google Scholar
  35. 35.
    Rivest, R.: Learning decision lists. Machine Learning 2(3), 229–246 (1987)Google Scholar
  36. 36.
    Shawe-Taylor, J., Cristianini, N.: An introduction to support vector machines. Cambridge University Press, Cambridge (2000)Google Scholar
  37. 37.
    Håstad, J.: On the size of weights for threshold gates. SIAM Journal on Discrete Mathematics 7(3), 484–492 (1994)zbMATHCrossRefMathSciNetGoogle Scholar
  38. 38.
    Tarui, J., Tsukiji, T.: Learning DNF by approximating inclusion-exclusion formulae. In: Proceedings of the Fourteenth Conference on Computational Complexity, pp. 215–220 (1999)Google Scholar
  39. 39.
    Valiant, L.: A theory of the learnable. Communications of the ACM 27(11), 1134–1142 (1984)zbMATHCrossRefGoogle Scholar
  40. 40.
    Verbeurgt, K.: Learning DNF under the uniform distribution in quasi-polynomial time. In: Proceedings of the Third Annual Workshop on Computational Learning Theory, pp. 314–326 (1990)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Rocco A. Servedio
    • 1
  1. 1.Department of Computer ScienceColumbia UniversityNew YorkUSA

Personalised recommendations