Abstract
Extreme learning machine, as a generalized single-hidden-layer feedforward networks has achieved much attention for its extremely fast learning speed and good generalization performance. However, big data often makes a challenge in large scale learning of ELM due to the limitation of memory of single machine as well as the distributed manner of large scale data storage and collection in many applications. For the purpose of relieving the limitation of memory with big data, in this paper, we exploit a novel distributed extreme learning machine to implement the extreme learning machine algorithm in parallel for large-scale data set. A corresponding distributed algorithm is also developed on the basis of alternating direction method of multipliers which shows effectiveness in distributed convex optimization. Finally, some numerical experiments on well-known benchmark data sets are carried out to illustrate the effectiveness of the proposed DELM method and provide an analysis on the performance of speedup, scaleup and sizeup.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Huang, G.-B., Chen, L., Siew, C.K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17(4), 879–892 (2006)
Huang, G.-B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2010)
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. In: Sixth Symposium on Operating System Design and Implementation (2004)
Wang, X., Chen, Y., Zhao, H., Lu, B.: Parallel extreme learning machine for regression based on mapreduce, Neurocomputing 102
He, Q., Shang, T., Zhuang, F., Shi, Z.: Parallelized extreme learning machine ensemble based on min-max modular network, Neurocomputing 128
Zhang, C., Lee, H., Shin, K.G.: Efficient distributed linear classification algorithms via the alternating direction method of multipliers. In: The 15th International Conference on Aritificial Intellienge and Statistics (AISTATS)
Huang, G.-B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B Cybern. 42(2), 513–529 (2012)
Eckstein, J.: Parallel altrenating direction multiplier decomposition of convex programs. J. Optim. Theory Appl. 80(1), 39–62 (1994)
Acknowledgments
This work was supported in part by National Science Foundation of China (Grant No. 91118005, 91218301, 61502377, 61221063, 61428206), Ministry of Education Innovation Research Team (IRT13035), Key Projects in the National Science and Technology Pillar Program of China (2013BAK09B01) and the National Science Foundation (NSF) under grant CCF-1500365.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Luo, M., Zheng, Q., Liu, J. (2016). Distributed Extreme Learning Machine with Alternating Direction Method of Multiplier. In: Cao, J., Mao, K., Wu, J., Lendasse, A. (eds) Proceedings of ELM-2015 Volume 2. Proceedings in Adaptation, Learning and Optimization, vol 7. Springer, Cham. https://doi.org/10.1007/978-3-319-28373-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-28373-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-28372-2
Online ISBN: 978-3-319-28373-9
eBook Packages: EngineeringEngineering (R0)