Complexity Regularization

Abstract

This chapter offers key theoretical results that confirm the existence of certain “good” rules. Although the proofs are constructive—we do tell you how you may design such rules—the computational requirements are often prohibitive. Many of these rules are thus not likely to filter down to the software packages and pattern recognition implementations. An attempt at reducing the computational complexity somewhat is described in the section entitled “Simple empirical covering.” Nevertheless, we feel that much more serious work on discovering practical algorithms for empirical risk minimization is sorely needed.