Canaries in the Data Mine: Improving Trained Classifiers

Abstract

Supervised classification uses a training set to construct a classifier such as a decision tree. Normally, the training set is discarded once the training process is complete. By imprinting information about the training population onto the classifier, we can make use of the extrema at each node as “canaries”, warning us that we have left the well explored area of parameter space and have crossed into a domain where the classifier is unreliable. This technique can identify training set deficiencies; provide reliability estimates for decision tree classifiers; improve the results of multi-tree voting; and provide helpful visualization tools. See http://www-gsss.stsci.edu/PublishedPapers/Canaries_SCMA.htm for the poster version of this paper.