Hierarchical Classification for Solving Multi-class Problems: A New Approach Using Naive Bayesian Classification

Abstract

A hierarchical classification ensemble methodology is proposed as a solution to the multi-class classification problem where the output from a collection of classifiers, arranged in a hierarchical manner, are combined to produce a better composite global classification (better than when the classifiers making up the ensemble operate in isolation). A novel topology for arranging the classifiers in the hierarchy is proposed such that the leaf classifiers act as binary classifiers and the remaining classifiers (those at the root and intermediate nodes) address groupings of classes. The main challenge is how to address the general drawback of the hierarchical model, that is if a record is miss-classified early on in the classification process (near the root of the hierarchy) it will continue to be miss-classified at deeper levels too. Three different approaches, founded on Naive Bayes classification, are proposed whereby Bayesian probability values are used to indicate whether single or multiple paths should be followed within the hierarchy. Reported experimental results demonstrate that the proposed mechanism can improve classification performance, in terms of average AUC, in the context of selected data sets.