Alternative Measures of Computational Complexity with Applications to Agnostic Learning

  • Shai Ben-David
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3959)


We address a fundamental problem of complexity theory – the inadequacy of worst-case complexity for the task of evaluating the computational resources required for real life problems. While being the best known measure and enjoying the support of a rich and elegant theory, worst-case complexity seems gives rise to over-pessimistic complexity values. Many standard task, that are being carried out routinely in machine learning applications, are NP-hard, that is, infeasible from the worst-case-complexity perspective. In this work we offer an alternative measure of complexity for approximations-optimization tasks. Our approach is to define a hierarchy on the set of inputs to a learning task, so that natural (’real data’) inputs occupy only bounded levels of this hierarchy and that there are algorithms that handle in polynomial time each such bounded level.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Levin, L.A.: Robust measures of information. Comput. J. 42, 284–286 (1999)zbMATHCrossRefGoogle Scholar
  2. Ben-David, S., Chor, B., Goldreich, O., Luby, M.: On the theory of average case complexity. J. Comput. Syst. Sci. 44, 193–219 (1992)zbMATHCrossRefMathSciNetGoogle Scholar
  3. Fellows, M.R.: Parameterized complexity: The main ideas and connections to practical computing. Electr. Notes Theor. Comput. Sci. 61 (2002)Google Scholar
  4. Spielman, D.A., Teng, S.H.: Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time. J. ACM 51, 385–463 (2004)CrossRefMathSciNetzbMATHGoogle Scholar
  5. Ben-David, S., Eiron, N., Long, P.M.: On the difficulty of approximately maximizing agreements. J. Comput. Syst. Sci. 66, 496–514 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  6. Ben-David, S., Eiron, N., Simon, H.U.: The computational complexity of densest region detection. J. Comput. Syst. Sci. 64, 22–47 (2002)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Shai Ben-David
    • 1
  1. 1.School of Computer ScienceUniversity of WaterlooWaterloo, OntarioCanada

Personalised recommendations