Bounds for the Minimum Disagreement Problem with Applications to Learning Theory
Many studies have been done in the literature on minimum disagreement problems and their connection to Agnostic learning and learning with Malicious errors. We further study these problems and some extensions of them. The classes that are studied in the literature are monomials, monotone monomials, antimonotone monomials, decision lists, halfspaces, neural networks and balls. For some of these classes we improve on the best previously known factors for approximating the minimum disagreement. We also find new bounds for exclusive-or, k-term DNF, k-DNF and multivariate polynomials (Xor of monomials).
We then apply the above and some other results from the literature to Agnostic learning and give negative and positive results for Agnostic learning and PAC learning with malicious errors of the above classes.
KeywordsBoolean Function Decision List Consistent Hypothesis Minimum Disagreement Problem Constant Approximation Algorithm
Unable to display preview. Download preview PDF.
- [AL88]D. Angluin and P. D. Laird, Learning from noisy examples, Machine Learning, 2:343–370, 1988.Google Scholar
- [BB99]P. L. Barlett, S. Ben-David. Hardness results for neural network approximation problems. 4th Euro-COLT, 50–62, 1999.Google Scholar
- [BEL00]S. Ben-David, N. Eiron, P. M. Long. On the difficulty of approximately maximizing agreement. 13th COLT, 266–274, 2000.Google Scholar
- [BK01]P. Berman and M. Karpinski. Approximating minimal unsatisfiability of linear equationsx, Manuscript. From ECCC reports 2001, TR01–025.Google Scholar
- [BB02]N. H. Bshouty and L. Burroughs. Maximizing Agreements, CoAgnostic Learning and Weak Agnostic Learning. Manuscript, 2002.Google Scholar
- [F96]U. Feige, A threshold of log n for approximating set cover, 28th STOC, 314–318, 1996.Google Scholar
- [GJ79]M. R. Garey, D.S. Johnson, Computers and Intractability: A Guide to the theory of NP-Completeness. W. H. Freeman, 1979.Google Scholar
- [H97]J. Håstad, Some optimal inapproximability results, 29th STOC, 1–10, 1997.Google Scholar
- [Ha92]D. Haussler, Decision theoretic generalization of the PAC model for neural net and other learning applications. Inform. Comput., 100(1):78–150, Sept. 1992.Google Scholar
- [KMS94]D. Karger, R. Motwani, and M. Sudan, Approximate graph coloring by semidefinite programming. 35th FOCS, 2–13, 1994.Google Scholar
- [KLPV87]M. Kearns, M. Li, L. Pitt and L. Valiant. On the learnability of Boolean formulae, 19th STOC, 285–295, 1987.Google Scholar
- [KSS92]M. Kearns, R. E. Schapire and L. M. Sellie. Toward efficient agnostic learning. 5th COLT, 341–352, 1992.Google Scholar
- [KLS93]S. Khanna, N. Linial and S. Safra. On the hardness of approximating the chromatic number, Proc. 2nd Israel Symp. on Theory of Computing and Systems, 250–260, 1993.Google Scholar
- [K00]C. Kuhlmann. Harness Results for General Two-Layer Neural Networks. 13th COLT, 275–285, 2000.Google Scholar
- [M94]Y. Mansour, Learning Boolean Functions via the Fourier Transform. In Theoretical Advances in Neural Computation and Learning, (V. P. Roy chodhury, K-Y. Siu and A. Orlitsky, ed), 391–424 (1994).Google Scholar
- [Raz95]R. Raz, A parallel repetition theorem. 27th STOC, 447–456, 1995.Google Scholar
- [S01]H. U. Simon, private communication, 2001.Google Scholar