EE-Inspired Risks

Abstract

In the previous chapters the behavior of classifiers trained to minimize error-entropy risks, for both discrete and continuous errors, was analyzed. The rationale behind the use of these risks is the fact that entropy is a PDF concentration measure — higher concentration implies lower entropy —, and in addition (recalling what was said in Sect. 2.3.1) minimum entropy is attained for Dirac-δ combs (including a single one). Ideally, in supervised classification, one would like to drive the learning process such that the final distribution of the error variable is a Dirac-δ centered at the origin. In rigor, this would only happen for completely separable classes when dealing with the discrete error case or with infinitely distant classes when dealing with the continuous error case with the whole real line as support.