On the Proper Learning of Axis Parallel Concepts
C is α-properly exactly learnable (with hypotheses of size at most α times the target size) from membership and equivalence queries.
C is α-properly PAC learnable (without membership queries) under any product distribution.
There is an α-approximation algorithm for the MinEqui C problem. (given a g ∈ C find a minimal size f ∈ C that is equivalent to g).
In particular, C is α-properly learnable in poly time from membership and equivalence queries if and only if C is α-properly PAC learnable in poly time under the product distribution if and only if MinEqui C has a poly time α-approximation algorithm. Using this result we give the first proper learning algorithm of decision trees over the constant dimensional domain and the first negative results in proper learning from membership and equivalence queries for many classes.
For the non-constant dimensional axis parallel concepts we show that with the equivalence oracle (1) ⇒ (3). We use this to show that (binary) decision trees are not properly learnable in polynomial time (assuming P≠NP) and DNF is not sε-properly learnable (∈ < 1) in polynomial time even with an NP-oracle (assuming Σ 2 p ≠ P NP ).
KeywordsPolynomial Time Boolean Function Multivariate Polynomial Membership Query Equivalence Query
Unable to display preview. Download preview PDF.
- [A88]D. Angluin. Queries and concept learning. Machine Learning, 319–342, 1988.Google Scholar
- [BBK97]S. Ben-David, N. H. Bshouty, E. Kushilevitz. A composition theorem for learning algorithms with applications to geometric concept classes. 29th STOC, 1997.Google Scholar
- [BCV96]F. Bergadano, D. Catalano, S. Varricchio. Learning sat-k-DNF formulas from membership queries. In 28th STOC, pp 126–130, 1996.Google Scholar
- [BD92]P. Berman, B. DasGupta. Approximating the rectilinear polygon cover problems. In Proceedings of the 4th Canadian Conference on Computational Geometry, pages 229–235, 1992.Google Scholar
- [B95a]N. H. Bshouty. Simple learning algorithms using divide and conquer. In Proceedings of the Annual ACM Workshop on Computational Learning Theory, 1995.Google Scholar
- [B98]N. H. Bshouty. A new composition theorem for learning algorithms. Proceedings of the 30th annual ACM Symposium on Theory of Computing (STOC), May 1998.Google Scholar
- [DK91]V. J. Dielissen, A. Kaldewaij. Rectangular partition is polynomial in two dimensions but NP-complete in three. In IPL, pages 1–6, 1991.Google Scholar
- [HR02]L. Hellerstein and V. Raghavan. Exact learning of DNF formulas using DNF hypotheses. In 34rd Annual Symposium on Theory of Computing (STOC), 2002.Google Scholar
- [KS01]A. Klivans and R. Servedio. Learning DNF in Time 2O(n1/3). In 33rd Annual Symposium on Theory of Computing (STOC), 2001, pp. 258–265.Google Scholar
- [PV88]L. Pitt and L. G. Valiant. Computational Limitations on Learning from Examples. Journal of the ACM, 35(4):965–984, October 1988.Google Scholar
- [SS93]R. E. Schapire, L. M. Sellie. Learning sparse multivariate polynomial over a field with queries and counterexamples. In Proceedings of the Sixth Annual ACM Workshop on Computational Learning Theory. July, 1993.Google Scholar
- [U99]C. Umans. Hardness of approximating Σ2 p minimization problems. In Proceedings of the 40th Symposium on Foundations of Computer Science, 1999.Google Scholar
- [Val84]L. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134–1142, November 1984.Google Scholar