Inverse partitioned matrix-based semi-random incremental ELM for regression
Incremental extreme learning machine has been verified that it has the universal approximation capability. However, there are two major issues lowering its efficiency: one is that some “random” hidden nodes are inefficient which decrease the convergence rate and increase the structural complexity, the other is that the final output weight vector is not the minimum norm least-squares solution which decreases the generalization capability. To settle these issues, this paper proposes a simple and efficient algorithm in which the parameters of even hidden nodes are calculated by fitting the residual error vector in the previous phase, and then, all existing output weights are recursively updated based on inverse partitioned matrix. The algorithm can reduce the inefficient hidden nodes and obtain a preferable output weight vector which is always the minimum norm least-squares solution. Theoretical analyses and experimental results show that the proposed algorithm has better performance on convergence rate, generalization capability and structural complexity than other incremental extreme learning machine algorithms.
KeywordsExtreme learning machine The minimum norm least-squares solution Inefficient nodes Convergence rate Inverse partitioned matrix
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
- 7.Huang G-B, Babri H (1997) General approximation theorem on feedforward networks. In: International conference on information, vol 2, pp 698–702Google Scholar
- 8.Huang G-B, Zhu Q-Y, Siew C-K (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: IEEE international joint conference, vol 2. IEEE, pp 985–990. https://doi.org/10.1109/IJCNN.2004.1380068
- 9.Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: Theory and applications. In: Brazilian symposium on neural networks, vol 70, pp 489–501. https://doi.org/10.1016/j.neucom.2005.12.126
- 13.An L, Bhanu B (2012) Image super-resolution by extreme learning machine. In: 2012 19th IEEE international conference on image processing, IEEE Signal Process. IEEE, pp 2209–2212. https://doi.org/10.1109/ICIP.2012.6467333
- 17.Rong H-J, Huang G-B, Ong Y-S (2008) Extreme learning machine for multi-categories classification applications. In: IJCNN. https://doi.org/10.1109/IJCNN.2008.4634028
- 22.Miche Y, Sorjamaa A, Lendasse A (2008) OP-ELM: theory, experiments and a toolbox. In: ICANN 2008. Springer, pp 145–154Google Scholar
- 31.Meyer C-D (2001) Matrix analysis and applied linear algebra. SIAM, PhiladelphiaGoogle Scholar