An Improved Double Hidden-Layer Variable Length Incremental Extreme Learning Machine Based on Particle Swarm Optimization
Extreme learning machine (ELM) has been widely used in diverse domains. With the development of deep learning, integrating ELM with some deep learning method has become a new perspective method for extracting and classifications. However, it may require a large number of hidden nodes and lead to the ill-condition problem for its random generation. In this paper, an effective hybrid approach based on Variable-length Incremental ELM and Particle Swarm Optimization (PSO) algorithm (PSO-VIELM) is proposed which can be used to regulate weights and extract features. In the new method, we build two hidden layers to establish a structure which is compact with a better generalization performance. In the first hidden layer named extraction layer, we make the feature learning to the raw data, and make dynamic updates for hidden layer nodes, and use the fitting error as the fitness function to update the weights corresponding to the hidden nodes with the method of PSO. In the second hidden layer named classification layer, we make a classification for the processed data from extraction layer and use cross-entropy as the fitness function to update the weights in the net. In order to find the appropriate number of hidden layer nodes, all hidden nodes will no longer grow in the case of a rebound in the fitness function on the validation set. The result in some datasets shows that PSO-VIELM has a better generalization performance than other constructive ELMs.
KeywordsExtreme Learning Machine Particle Swarm Optimization Feature extraction Auto-encoder
This work was supported by the National Natural Science Foundation of China [Nos. 61572241 and 61271385], the National Key R&D Program of China [No. 2017YFC0806600], the Foundation of the Peak of Six Talents of Jiangsu Province [No. 2015-DZXX-024], the Fifth 333 High Level Talented Person Cultivating Project of Jiangsu Province [No. (2016) III-0845], and the Research Innovation Program for College Graduates of Jiangsu Province .
- 2.Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: IEEE International Joint Conference on Neural Networks, vol. 2, pp. 985–990 (2004)Google Scholar
- 4.Huang, G.B., Siew, C.K.: Extreme learning machine: RBF network case. In: Control, Automation, Robotics and Vision Conference, pp. 1029–1036 (2004)Google Scholar
- 6.Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (2002)Google Scholar
- 8.Zhao, G., Shen, Z., Miao, C., Gay, R.: Enhanced extreme learning machine with stacked generalization. In: IEEE International Joint Conference on Neural Networks, pp. 1191–1198 (2008)Google Scholar
- 11.Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bouchard, N., Wardefarley, D., Bengio, Y.: Theano: new features and speed improvements. Computer Science (2012)Google Scholar