Abstract
Extreme learning machine (ELM), which proposed for generalized single-hidden layer feedforward neural networks, has become a popular research topic due to its fast learning speed, good generalization ability, and ease of implementation. However, ELM faces redundancy and randomness in the hidden layer which caused by random mapping of features. In ELM, although evolutionary algorithms have archived impressive improvement, they have not considered the sparsity of the hidden layers. In this paper, a hybrid learning algorithm is proposed, termed EMO-ELM, which adopts evolutionary multi-objective algorithm to optimise two conflict objectives simultaneously. Furthermore, the proposed method can be used for supervised classification and unsupervised sparse feature extraction tasks. Simulations on many UCI datasets have demonstrated that EMO-ELM generally outperforms the original ELM algorithm as well as several ELM variants in classification tasks, moreover, EMO-ELM achieves a competitive performance to PCA in sparse feature extraction tasks.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Tang, J., Deng, C., Huang, G.B.: Extreme learning machine for multilayer perceptron. IEEE Trans. Neural Netw. Learn. Syst. 27(4), 809–821 (2016)
Huang, G.B., Chen, L., Siew, C.K., et al.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Networks 17(4), 879–892 (2006)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)
Minhas, R., Baradarani, A., Seifzadeh, S., Wu, Q.J.: Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing 73(10), 1906–1917 (2010)
Zong, W., Huang, G.B.: Face recognition based on extreme learning machine. Neurocomputing 74(16), 2541–2551 (2011)
Zeng, Y., Xu, X., Shen, D., Fang, Y., Xiao, Z.: Traffic sign recognition using kernel extreme learning machines with deep perceptual features. IEEE Trans. Intell. Transp. Syst. (2016)
Chen, C., Li, W., Su, H., Liu, K.: Spectral-spatial classification of hyperspectral image based on kernel extreme learning machine. Remote Sens. 6(6), 5795–5814 (2014)
Pal, M., Maxwell, A.E., Warner, T.A.: Kernel-based extreme learning machine for remote-sensing image classification. Remote Sens. Lett. 4(9), 853–862 (2013)
Lv, Q., Niu, X., Dou, Y., Xu, J., Lei, Y.: Classification of hyperspectral remote sensing image using hierarchical local-receptive-field-based extreme learning machine. IEEE Geosci. Remote Sens. Lett. 13(3), 434–438 (2016)
Savojardo, C., Fariselli, P., Casadio, R.: Betaware: a machine-learning tool to detect and predict transmembrane beta barrel proteins in prokaryotes. Bioinformatics, bts728 (2013)
Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(2), 513–529 (2012)
Huang, G.B.: An insight into extreme learning machines: random neurons, random features and kernels. Cogn. Comput. 6(3), 376–390 (2014)
Cao, J., Lin, Z., Huang, G.B., Liu, N.: Voting based extreme learning machine. Inf. Sci. 185(1), 66–77 (2012)
Liu, N., Wang, H.: Ensemble based extreme learning machine. IEEE Signal Process. Lett. 17(8), 754–757 (2010)
Zhu, Q.Y., Qin, A.K., Suganthan, P.N., Huang, G.B.: Evolutionary extreme learning machine. Pattern Recogn. 38(10), 1759–1763 (2005)
Cao, J., Lin, Z., Huang, G.B.: Self-adaptive evolutionary extreme learning machine. Neural Process. Lett. 36, 1–21 (2012)
Li, K.E., Wang, R.A.N., Kwong, S.A.M., Cao, J.: Evolving extreme learning machine paradigm with adaptive operator selection and parameter control. Int. J. Uncert. Fuzz. Knowl.-Based Syst. 21(Suppl. 02), 143–154 (2013)
Zhang, Y., Wu, J., Cai, Z., Zhang, P., Chen, L.: Memetic extreme learning machine. Pattern Recogn. 58, 135–148 (2016)
Mukhopadhyay, A., Maulik, U., Bandyopadhyay, S., Coello, C.A.C.: A survey of multiobjective evolutionary algorithms for data mining: Part I. IEEE Trans. Evol. Comput. 18(1), 4–19 (2014)
Pan, L., He, C., Tian, Y., Su, Y., Zhang, X.: A region division based diversity maintaining approach for many-objective optimization. Integr. Comput.-Aided Eng. (Preprint), 1–18 (2017)
Gu, S., Cheng, R., Jin, Y.: Multi-objective ensemble generation. Wiley Interdisc. Rev. Data Mining Knowl. Discov. 5(5), 234–245 (2015)
Liu, J., Gong, M., Miao, Q., Wang, X., Li, H.: Structure learning for deep neural networks based on multiobjective optimization. IEEE Trans. Neural Netw. Learn. Syst. (2017)
Gong, M., Liu, J., Li, H., Cai, Q., Su, L.: A multiobjective sparse feature learning model for deep neural networks. IEEE Trans. Neural Netw. Learn. Syst. 26(12), 3263–77 (2015)
Mao, W., Tian, M., Cao, X., Xu, J.: Model selection of extreme learning machine based on multi-objective optimization. Neural Comput. Appl. 22(3–4), 521–529 (2013)
Huang, G.B.: What are extreme learning machines? filling the gap between frank Rosenblatt’s dream and John von Neumann’s puzzle. Cogn. Comput. 7(3), 263–278 (2015)
Kasun, L.L.C., Yang, Y., Huang, G.B., Zhang, Z.: Dimension reduction with extreme learning machine. IEEE Trans. Image Process. 25(8), 3906–3918 (2016)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Chao, T., Hongbo, P., Yansheng, L., Zhengrou, Z.: Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification. IEEE Geosci. Remote Sens. Lett. 12(12), 2438–2442 (2015)
Rachmawati, L., Srinivasan, D.: Multiobjective evolutionary algorithm with controllable focus on the knees of the pareto front. IEEE Trans. Evol. Comput. 13(4), 810–824 (2009)
Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)
Acknowledgment
We thank the reviewers for their valuable comments and suggestions. This work was partially supported by the National Natural Science Foundation of China under grant No. 61603355 and No. 61403351, and the Fundamental Research Funds for National University, China University of Geosciences (Wuhan) under grant No. G1323541717.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Cai, Y. et al. (2017). Extreme Learning Machine Based on Evolutionary Multi-objective Optimization. In: He, C., Mo, H., Pan, L., Zhao, Y. (eds) Bio-inspired Computing: Theories and Applications. BIC-TA 2017. Communications in Computer and Information Science, vol 791. Springer, Singapore. https://doi.org/10.1007/978-981-10-7179-9_32
Download citation
DOI: https://doi.org/10.1007/978-981-10-7179-9_32
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-7178-2
Online ISBN: 978-981-10-7179-9
eBook Packages: Computer ScienceComputer Science (R0)