Subclass Maximum Margin Tree Error Correcting Output Codes

Abstract

Error Correcting Output Codes (ECOC) is an effective method to handle multi-class classification problems, whose performance is heavily affected by encoding strategies. However, traditional encoding strategies are usually data-independent which more likely leads to a less representation coding matrix. Therefore, recent researches emphasize more on the data-dependent construction of coding matrix. However, these methods usually can not guarantee that the binary problems in coding matrix are linearly separable. When the problems are linearly non-separable, the difficulty of designing base classifiers will increase. Even the non-linear base classifiers can not ensure that they can handle each binary problem well, which will decrease the performance of ECOC. In this paper, we propose subclass maximum margin tree error correcting output codes (SM$^2$ECOC) which aims to make each binary problem simple and linearly separable. Concretely, SM$^2$ECOC firstly uses hierarchical clustering techniques to split original classes into a series of linearly separable subclasses. Then it takes the margin as a criterion to evaluate the separability among subclasses and guide the construction of coding matrix. As a result, the binary problems in coding matrix more likely tend to be linearly separable which can reduce the difficulty in base classifiers effectively. Experimental results show the superiority of SM$^2$ECOC.