Wilcoxon-Norm-Based Robust Extreme Learning Machine

  • Xiao-Liang XieEmail author
  • Gui-Bin Bian
  • Zeng-Guang Hou
  • Zhen-Qiu Feng
  • Jian-Long Hao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8866)


It is known in statistics that the linear estimators using the rank-based Wilcoxon approach in linear regression problems are usually insensitive to outliers. Outliers are the data points that differ greatly from the pattern set by the bulk of the data. Inspired by this, Hsieh et al introduced the Wilcoxon approach into the area of machine learning. They investigated four new learning machines, such as Wilcoxon neural network (WNN) etc., and developed four descent gradient based backpropagation algorithms to train these learning machines. The performances of these machines are better than the ordinary nonrobust neural networks. However, it is hard to balance the learning speed and the stability of these algorithms which is inherently the drawback of gradient descent based algorithms. In this paper, a new algorithm is used to train the output weights of single-layer feedforward neural networks (SLFN) with its input weights and biases being randomly chosen. This algorithm is called Wilcoxon-norm based robust extreme learning machine or WRELM for short.


Extreme learning machine Wilcoxon neural network Wilcoxon-norm based robust extreme learning machine 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kohonen, T.: Self-organizing formation of topologically correct feature maps. Biological Cybernetics 43 (1982)Google Scholar
  2. 2.
    Powell, M.J.D.: Radial basis functions for multivariable interpolation: a review. In: Mason, J.C., Cox, M.G. (eds.) Algorithms for Approximation, pp. 143–167. Clarendon Press, Oxford (1987)Google Scholar
  3. 3.
    Cortes, C., Vapnik, V.: Support vector networks. Machine Learning 20, 273–297 (1995)Google Scholar
  4. 4.
    Werbos, P.: Beyond regression: new tools for prediction and analysis in the behavioral sciences. Ph.D. dissertation, Harvard Univ., Cambridge, MA (1974)Google Scholar
  5. 5.
    Gibb, J.: Back propagation family album. Technical Report C/TR96-05, Macquarie University (August 1996)Google Scholar
  6. 6.
    Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In: Proc. of the IEEE Int. Conf. on Neural Netw., San Francisco, CA (April 1993)Google Scholar
  7. 7.
    Leshno, M., Lin, V.Y., Pinkus, A., Schocken, S.: Multilayer feedfor-ward networks with a nonpolynomial activation function can approximate any function. Neural Netw. 6, 861–867 (1993)CrossRefGoogle Scholar
  8. 8.
    Huang, G.B.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proc. Int. Joint Conf. Neural Netw. (IJCNN 2004), Budapest, Hungary, July 25–29, vol. 2, pp. 985–990 (2004)Google Scholar
  9. 9.
    Huang, G.B., Zhou, H.M., Ding, X.J., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 42(2), 513–529 (2012)CrossRefGoogle Scholar
  10. 10.
    Liang, N.Y., Huang, G.B., Saratchandran, P., Sundararajan, N.: A fast and accurate online sequential learning algorithm for feedforward network. IEEE Trans. on Neural Netw., 17(6) (November 2006)Google Scholar
  11. 11.
    Hawkins, D.M.: Identification of Outliers. Chapman & Hall, London (1980)CrossRefzbMATHGoogle Scholar
  12. 12.
    Jureckova, J.: Asymptotic linearity of a rank statistic in regression parameter. Ann. Math. Statist. 40, 1889–1900 (1969)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Jaeckel, L.A.: Estimating regression coefficients by minimizing the dispersion of the residuals. Ann. Math. Statist. 43, 1449–1458 (1972)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Hettmansperger, T.P.: Statistical inference based on ranks. Wiley, New York (1984)zbMATHGoogle Scholar
  15. 15.
    Hettmansperger, T.P., McKean, J.W.: Robust non-parametric statistics. Wiley, New York (1998)Google Scholar
  16. 16.
    Hettmansperger, T.P., McKean, J.W.: Robust nonparametric statistical methods. Arnold, London (1998)zbMATHGoogle Scholar
  17. 17.
    Scuster, E.: On the rate of convergence of an estimate of a functional of a probability density. Scandinavian Actuarial Journal 1, 103–107 (1974)CrossRefGoogle Scholar
  18. 18.
    Choi, Y.H., Ozturk, O.: A new class of score generating functions for regression models. Statistics & Probability Letters 57, 205–214 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Asuncion, A., Newman, D.J.: UCI Machine Learning Repository (2007).
  20. 20.
    Hsieh, J.G., Lin, Y.L., Jeng, J.H.: Preliminary study on Wilcoxon learning machines. IEEE Trans. on Neural Netw. 19(2), 201–211 (2008)CrossRefGoogle Scholar
  21. 21.
    Rusiecki, A.: Robust LTS backpropagation learning algorithm. In: Sandoval, F., Prieto, A.G., Cabestany, J., Graña, M. (eds.) IWANN 2007. LNCS, vol. 4507, pp. 102–109. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  22. 22.
    Qing, C.Y., Annpey, P., Biao, X.: Rank regression in stability analysis. Journal of Biophamaceutical Statistics 13, 463–479 (2003)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Xiao-Liang Xie
    • 1
    Email author
  • Gui-Bin Bian
    • 1
  • Zeng-Guang Hou
    • 1
  • Zhen-Qiu Feng
    • 1
  • Jian-Long Hao
    • 1
  1. 1.State Key Laboratory of Management and Control for Complex Systems, Institute of AutomationChinese Academy of SciencesBeijingChina

Personalised recommendations