A Regularized Line Search Tunneling for Efficient Neural Network Learning

Abstract

A novel two phases training algorithm for a multilayer perceptron with regularization is proposed to solve a local minima problem for training networks and to enhance the generalization property of networks trained. The first phase is a trust region-based local search for fast training of networks. The second phase is an regularized line search tunneling for escaping local minima and moving toward a weight vector of next descent. These two phases are repeated alternatively in the weight space to achieve a goal training error. Benchmark results demonstrate a significant performance improvement of the proposed algorithm compared to other existing training algorithms.