Comparative Study of Accuracies on the Family of the Recursive-Rule Extraction Algorithm

Abstract

In this paper, we first compare the accuracies of the Recursive-Rule Extraction algorithm family, i.e., the Re-RX algorithm, its variant and the “Three-MLP Ensemble by the Re-RX algorithm” (shortened to “Three-MLP Ensemble”) using the Re-RX algorithm as a core part for six kinds of two-class mixed (i.e., discrete and continuous attributes) datasets. Two-class mixed datasets are commonly used for credit scoring and generally in financial domains. In this paper, we compare the accuracy by only the Re-RX algorithm family because of recent comparison reviews and benchmarking study results, obtained by complicated statistics, support vector machines, neuro-fuzzy hybrid classifications, and similar techniques. The Three-MLP Ensemble algorithm cascades standard backpropagation (BP) to train a three neural-network ensemble, where each neural network is a Multi-Layer Perceptron (MLP). Thus, strictly speaking, three neural networks do not need to be trained simultaneously. In addition, the Three-MLP Ensemble is a simple and new concept of rule extraction from neural network ensembles and can avoid previous complicated neural network ensemble structures and the difficulties of rule extraction algorithms. The extremely high accuracy of the Three-MLP Ensemble algorithm generally outperformed the Re-RX algorithm and the variant. The results confirm that the output from the network ensemble can be expressed in the form of rules, and thus opens the “black box” of trained neural network ensembles.