An Efficient Extreme Learning Machine for Robust Regression

Abstract

In this paper, we intend to build a robust extreme learning machine (RELM) with the advantage of both Bayesian framework and Huber loss function. The new method inherits the basic idea of training ELM in a Bayesian framework and replacing the original quadratic loss function by Huber loss function when estimating output weights, in order to enhance the robustness of model. However, the introduction of Huber loss function also yields the prior distribution of model output no longer Gaussian, which makes it difficult to estimate model parameters by using Bayesian method directly. To solve this problem, the iteratively re-weighted least squares (IRWLS) is employed and the Huber cost function can be equivalently transformed into the form of quadratic loss function, which results in an efficient Bayesian method for parameter estimation and remains robust to outliers. We demonstrate with experimental results that the proposed method can effectively increase the robustness of model.