Bounds for the Minimum Disagreement Problem with Applications to Learning Theory

  • Nader H. Bshouty
  • Lynn Burroughs
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2375)


Many studies have been done in the literature on minimum disagreement problems and their connection to Agnostic learning and learning with Malicious errors. We further study these problems and some extensions of them. The classes that are studied in the literature are monomials, monotone monomials, antimonotone monomials, decision lists, halfspaces, neural networks and balls. For some of these classes we improve on the best previously known factors for approximating the minimum disagreement. We also find new bounds for exclusive-or, k-term DNF, k-DNF and multivariate polynomials (Xor of monomials).

We then apply the above and some other results from the literature to Agnostic learning and give negative and positive results for Agnostic learning and PAC learning with malicious errors of the above classes.


Boolean Function Decision List Consistent Hypothesis Minimum Disagreement Problem Constant Approximation Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [AL88]
    D. Angluin and P. D. Laird, Learning from noisy examples, Machine Learning, 2:343–370, 1988.Google Scholar
  2. [ABSS97]
    S. Arora, L. Babai, J. Stern and Z. Sweedyk, Hardness of approximate optima in lattices, codes, and systems of linear equations, JCSS 54(2): 317–331, 1997.zbMATHMathSciNetGoogle Scholar
  3. [BB99]
    P. L. Barlett, S. Ben-David. Hardness results for neural network approximation problems. 4th Euro-COLT, 50–62, 1999.Google Scholar
  4. [BEL00]
    S. Ben-David, N. Eiron, P. M. Long. On the difficulty of approximately maximizing agreement. 13th COLT, 266–274, 2000.Google Scholar
  5. [BK01]
    P. Berman and M. Karpinski. Approximating minimal unsatisfiability of linear equationsx, Manuscript. From ECCC reports 2001, TR01–025.Google Scholar
  6. [BB02]
    N. H. Bshouty and L. Burroughs. Maximizing Agreements, CoAgnostic Learning and Weak Agnostic Learning. Manuscript, 2002.Google Scholar
  7. [F96]
    U. Feige, A threshold of log n for approximating set cover, 28th STOC, 314–318, 1996.Google Scholar
  8. [GJ79]
    M. R. Garey, D.S. Johnson, Computers and Intractability: A Guide to the theory of NP-Completeness. W. H. Freeman, 1979.Google Scholar
  9. [H97]
    J. Håstad, Some optimal inapproximability results, 29th STOC, 1–10, 1997.Google Scholar
  10. [Ha92]
    D. Haussler, Decision theoretic generalization of the PAC model for neural net and other learning applications. Inform. Comput., 100(1):78–150, Sept. 1992.Google Scholar
  11. [HSV95]
    Klaus-U Höffgen, Hans-U. Simon and Kevin S. Van Horn, Robust train-ability of single neurons, J. of Comput. Syst. Sci., 50(1): 114–125, 1995.zbMATHCrossRefGoogle Scholar
  12. [KMS94]
    D. Karger, R. Motwani, and M. Sudan, Approximate graph coloring by semidefinite programming. 35th FOCS, 2–13, 1994.Google Scholar
  13. [KL93]
    M. Kearns and M. Li, Learning in the presence of malicious errors, SIAM Journal on Computing, 22(4): 807–837, 1993.zbMATHCrossRefMathSciNetGoogle Scholar
  14. [KLPV87]
    M. Kearns, M. Li, L. Pitt and L. Valiant. On the learnability of Boolean formulae, 19th STOC, 285–295, 1987.Google Scholar
  15. [KSS92]
    M. Kearns, R. E. Schapire and L. M. Sellie. Toward efficient agnostic learning. 5th COLT, 341–352, 1992.Google Scholar
  16. [KLS93]
    S. Khanna, N. Linial and S. Safra. On the hardness of approximating the chromatic number, Proc. 2nd Israel Symp. on Theory of Computing and Systems, 250–260, 1993.Google Scholar
  17. [K00]
    C. Kuhlmann. Harness Results for General Two-Layer Neural Networks. 13th COLT, 275–285, 2000.Google Scholar
  18. [M94]
    Y. Mansour, Learning Boolean Functions via the Fourier Transform. In Theoretical Advances in Neural Computation and Learning, (V. P. Roy chodhury, K-Y. Siu and A. Orlitsky, ed), 391–424 (1994).Google Scholar
  19. [PV88]
    L. Pitt and L. G. Valiant, Computational Limitation on Learning from Examples. JACM, 35,(4): 965–984, 1988.zbMATHCrossRefMathSciNetGoogle Scholar
  20. [Raz95]
    R. Raz, A parallel repetition theorem. 27th STOC, 447–456, 1995.Google Scholar
  21. [S01]
    H. U. Simon, private communication, 2001.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Nader H. Bshouty
    • 1
  • Lynn Burroughs
    • 2
  1. 1.Department of Computer ScienceTechnionHaifaIsrael
  2. 2.Department of Computer ScienceUniversity of CalgaryCalgaryCanada

Personalised recommendations