Slow Rates of Convergence

Abstract

In this chapter we consider the general pattern recognition problem: Given the observation X and the training data D _n = ((X ₁, Y ₁),..., (X _n , Y _n )) of independent identically distributed random variable pairs, we estimate the label Y by the decision

$$$$

. The error probability is

$$$$

. Obviously, the average error probability E L _n = P{Y ≠ g _n (X)} is completely determined by the distribution of the pair (X, Y), and the classifier g _n . We have seen in Chapter 6 that there exist classification rules such as the cubic histogram rule with properly chosen cube sizes such that lim _n→∞ E L _n = L* for all possible distributions. The next question is whether there are classification rules with E L _n tending to the Bayes risk at a specified rate for all distributions. Disappointingly, such rules do not exist.