Learning Under a Fixed Probability Measure

Abstract

In this chapter, we study the problems of concept and function learning in the case where the samples are drawn in accordance with a known fixed distribution. Various necessary and/or sufficient conditions are presented for a concept class or a function class to be learnable. The principal results of the chapter can be summarized as follows: Suppose the input sequence to the learning algorithm is i.i.d. Then we have the following:

• 1. If a function class F (or a concept class C) has the property of uniform convergence of empirical means, then it is also ASEC learnable. However, the converse is not true in general — there exist function classes that are ASEC learnable even though they do not possess the UCEM property.

• 2. A function class is PUAC learnable if it possesses a property known as the “shrinking width” property. The shrinking width property is also a necessary condition in order for every consistent algorithm to be PU AC.

• 3. Similarly, there is a necessary and sufficient condition for a function family to be consistently PAC learnable.

• 4. It can be shown that PUAC learnability is equivalent to consistent PUAC learnabilit. In contrast, PAC learnability is not equivalent in general to consistent PAC learnability.

• 5. A function class (or a concept class) is learnable if it satisfies a property known as “finite metric entropy.”

• 6. In order for a concept class to be learnable, the finite metric entropy condition is necessary as well as sufficient; however, for a function class to be learnable, the finite metric entropy condition is sufficient but is not necessary in general.