Abstract
We consider the Extreme Learning Machine model for accurate regression estimation and the related problem of selecting the appropriate number of neurons for the model. Selection strategies that choose “the best” model from a set of candidate network structures neglect the issues of model selection uncertainty. To alleviate the problem, we propose to remove this selection phase with a combination layer that takes into account all considered models. The proposed method in this paper is the Extreme Learning Machine(Jackknife Model Averaging), where Jackknife Model Averaging is a combination method based on leave-one-out residuals of linear models. The combination approach is shown to have better predictive performance on several real-world data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: Theory and applications. Neurocomputing 70(1-3), 489–501 (2006)
Huang, G.B., Chen, L., Siew, C.K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Transactions on Neural Networks 17(4), 879–892 (2006)
Feng, G., Huang, G.B., Lin, Q., Gay, R.: Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Transactions on Neural Networks 20(8), 1352–1357 (2009)
Lan, Y., Soh, Y.C., Huang, G.B.: Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73(16-18), 3191–3199 (2010)
Rong, H.J., Ong, Y.S., Tan, A.H., Zhu, Z.: A fast pruned-extreme learning machine for classification problem. Neurocomputing 72(1-3), 359–366 (2008)
Miche, Y., van Heeswijk, M., Bas, P., Simula, O., Lendasse, A.: TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing 74(16), 2413–2421 (2011)
van Heeswijk, M., Miche, Y., Lindh-Knuutila, T., Hilbers, P., Honkela, T., Oja, E., Lendasse, A.: Adaptive ensemble models of extreme learning machines for time series prediction. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009, Part II. LNCS, vol. 5769, pp. 305–314. Springer, Heidelberg (2009)
Breiman, L.: Stacked regressions. Machine Learning 24(1), 49–64 (1996)
Draper, D.: Assessment and propagation of model uncertainty (with discussion). Journal of the Royal Statistical Society: Series B 57(1), 45–97 (1995)
Hoeting, J.A., Madigan, D., Raftery, A.E., Volinsky, C.T.: Bayesian model averaging: A tutorial. Statistical Science 14(4), 382–417 (1999)
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Allen, D.M.: The relationship between variable selection and data augmentation and a method for prediction. Techometrics 16(1), 125–127 (1974)
Hansen, B.E., Racine, J.S.: Jackknife model averaging. Journal of Econometrics 167(1), 38–46 (2012)
Frank, A., Asuncion, A.: UCI machine learning repository (2010), http://archive.ics.uci.edu/ml
Torgo, L.: LIACC regression data sets, http://www.dcc.fc.up.pt/~ltorgo/Regression/DataSets.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sovilj, D., Lendasse, A., Simula, O. (2013). Extending Extreme Learning Machine with Combination Layer. In: Rojas, I., Joya, G., Gabestany, J. (eds) Advances in Computational Intelligence. IWANN 2013. Lecture Notes in Computer Science, vol 7902. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38679-4_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-38679-4_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38678-7
Online ISBN: 978-3-642-38679-4
eBook Packages: Computer ScienceComputer Science (R0)