Recursive Pattern based Hybrid Supervised Training

Summary

We propose, theorize and implement the Recursive Pattern-based Hybrid Supervised (RPHS) learning algorithm. The algorithm makes use of the concept of pseudo global optimal solutions to evolve a set of neural networks, each of which can solve correctly a subset of patterns. The pattern-based algorithm uses the topology of training and validation data patterns to find a set of pseudo-optima, each learning a subset of patterns. It is therefore well adapted to the pattern set provided. We begin by showing that finding a set of local optimal solutions is theoretically equivalent, and more efficient, to finding a single global optimum in terms of generalization accuracy and training time. We also highlight that, as each local optimum is found by using a decreasing number of samples, the efficiency of the training algorithm is increased. We then compare our algorithm, both theoretically and empirically, with different recursive and subset based algorithms. On average, the RPHS algorithm shows better generalization accuracy, with improvement of up to 60% when compared to traditional methods. Moreover, certain versions of the RPHS algorithm also exhibit shorter training time when compared to other recent algorithms in the same domain. In order to increase the relevance of this paper to practitioners, we have added pseudo code, remarks, parameter and algorithmic considerations where appropriate.