Order Compression Schemes

Abstract

Sample compression schemes are schemes for “encoding” a set of examples in a small subset of examples. The long-standing open sample compression conjecture states that, for any concept class $\mathcal{C}$ of VC-dimension d, there is a sample compression scheme in which samples for concepts in $\mathcal{C}$ are compressed to samples of size at most d.

We show that every order over $\mathcal{C}$ induces a special type of sample compression scheme for $\mathcal{C}$, which we call order compression scheme. It turns out that order compression schemes can compress to samples of size at most d if $\mathcal{C}$ is maximum, intersection-closed, a Dudley class, or of VC-dimension 1–and thus in most cases for which the sample compression conjecture is known to be true.

Since order compression schemes are much simpler than sample compression schemes in general, their study seems to be a promising step towards resolving the sample compression conjecture. We reveal a number of fundamental properties of order compression schemes, which are helpful in such a study. In particular, order compression schemes exhibit interesting graph-theoretic properties as well as connections to the theory of learning from teachers.

This work was partly supported by the Natural Sciences and Engineering Research Council of Canada (NSERC).