Wilcoxon-Norm-Based Robust Extreme Learning Machine
It is known in statistics that the linear estimators using the rank-based Wilcoxon approach in linear regression problems are usually insensitive to outliers. Outliers are the data points that differ greatly from the pattern set by the bulk of the data. Inspired by this, Hsieh et al introduced the Wilcoxon approach into the area of machine learning. They investigated four new learning machines, such as Wilcoxon neural network (WNN) etc., and developed four descent gradient based backpropagation algorithms to train these learning machines. The performances of these machines are better than the ordinary nonrobust neural networks. However, it is hard to balance the learning speed and the stability of these algorithms which is inherently the drawback of gradient descent based algorithms. In this paper, a new algorithm is used to train the output weights of single-layer feedforward neural networks (SLFN) with its input weights and biases being randomly chosen. This algorithm is called Wilcoxon-norm based robust extreme learning machine or WRELM for short.
KeywordsExtreme learning machine Wilcoxon neural network Wilcoxon-norm based robust extreme learning machine
Unable to display preview. Download preview PDF.
- 1.Kohonen, T.: Self-organizing formation of topologically correct feature maps. Biological Cybernetics 43 (1982)Google Scholar
- 2.Powell, M.J.D.: Radial basis functions for multivariable interpolation: a review. In: Mason, J.C., Cox, M.G. (eds.) Algorithms for Approximation, pp. 143–167. Clarendon Press, Oxford (1987)Google Scholar
- 3.Cortes, C., Vapnik, V.: Support vector networks. Machine Learning 20, 273–297 (1995)Google Scholar
- 4.Werbos, P.: Beyond regression: new tools for prediction and analysis in the behavioral sciences. Ph.D. dissertation, Harvard Univ., Cambridge, MA (1974)Google Scholar
- 5.Gibb, J.: Back propagation family album. Technical Report C/TR96-05, Macquarie University (August 1996)Google Scholar
- 6.Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In: Proc. of the IEEE Int. Conf. on Neural Netw., San Francisco, CA (April 1993)Google Scholar
- 8.Huang, G.B.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proc. Int. Joint Conf. Neural Netw. (IJCNN 2004), Budapest, Hungary, July 25–29, vol. 2, pp. 985–990 (2004)Google Scholar
- 10.Liang, N.Y., Huang, G.B., Saratchandran, P., Sundararajan, N.: A fast and accurate online sequential learning algorithm for feedforward network. IEEE Trans. on Neural Netw., 17(6) (November 2006)Google Scholar
- 15.Hettmansperger, T.P., McKean, J.W.: Robust non-parametric statistics. Wiley, New York (1998)Google Scholar
- 19.Asuncion, A., Newman, D.J.: UCI Machine Learning Repository (2007). http://www.ics.uci.edu/~mlearn/MLRepository.html