Abstract
The initialization of input weights and hidden biases plays an important role in random vector functional link networks (RVFL). Although some optimization algorithms for initialization have been proposed in recent years, the initialization strategies of these algorithms are under the premise of the uniform distribution. In this paper, ten benchmark datasets are used to study the impact of different probability distributions (e.g., Uniform, Gaussian, and Gamma distributions) initialization on the performance of RVFL. The experimental results present some interesting observations and valuable instructions: (1) No matter whether we use Uniform, Gaussian, or Gamma distributions, RVFL initialized by the distribution with smaller variances always get lower training and testing RMSE; (2) Compared with the Uniform distribution, the Gaussian and Gamma distributions with smaller variances usually give the RVFL model better performance; (3) Regardless of the distribution, RVFL with the direct link from the input layer to the output layer has better performance than those without the link; (4) RVFL initialized by the distribution with larger variances generally needs more hidden nodes to achieve equivalent accuracy with ones having the smaller variances; (5) With the increase of distribution variances, the performance of RVFL decreases first and then remains stable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Azad, N.L., Mozaffari, A., Fathi, A.: An optimal learning-based controller derived from Hamiltonian function combined with a cellular searching strategy for automotive coldstart emissions. Int. J. Mach. Learn. Cybern. 8(3), 955–979 (2017)
Ding, S., Zhang, N., Zhang, J., Xu, X., Shi, Z.: Unsupervised extreme learning machine with representational features. Int. J. Mach. Learn. Cybern. 8(2), 587–595 (2017)
Liu, P., Huang, Y., Meng, L., Gong, S., Zhang, G.: Two-stage extreme learning machine for high-dimensional data. Int. J. Mach. Learn. Cybern. 7(5), 765–772 (2016)
Zhang, J., Ding, S., Zhang, N., Shi, Z.: Incremental extreme learning machine based on deep feature embedded. Int. J. Mach. Learn. Cybern. 7(1), 111–120 (2016)
Zhang, L., Suganthan, P.N.: A survey of randomized algorithms for training neural networks. Inf. Sci. 364, 146–155 (2016)
Cao, W.P., Wang, X.Z., Ming, Z., Gao, J.Z.: A review on neural networks with random weights. Neurocomputing 275, 278–287 (2018). https://doi.org/10.1016/j.neucom.2017.08.040
He, Y.L., Wang, X.Z., Huang, J.Z.: Fuzzy nonlinear regression analysis using a random weight network. Inf. Sci. 364, 222–240 (2016)
Ren, Y., Suganthan, P.N., Srikanth, N., Amaratunga, G.: Random vector functional link network for short-term electricity load demand forecasting. Inf. Sci. 367, 1078–1093 (2016)
Pao, Y.H., Takefuji, Y.: Functional-link net computing: theory, system architecture, and functionalities. Computer 25(5), 76–79 (1992)
Schmidt, W.F., Kraaijveld, M.A., Duin, R.P.: Feedforward neural networks with random weights. In: 11th IAPR International Conference on Pattern Recognition, pp. 1–4. IEEE (1992)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: 2004 IEEE International Joint Conference on Neural Networks, pp. 985–990. IEEE (2004)
Huang, G.B., Chen, L., Siew, C.K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17(4), 879–892 (2006)
Zhang, L., Suganthan, P.N.: A comprehensive evaluation of random vector functional link networks. Inf. Sci. 367, 1094–1105 (2016)
Li, M., Wang, D.: Insights into randomized algorithms for neural networks: practical issues and common pitfalls. Inf. Sci. 382, 170–178 (2017)
Wang, D., Li, M.: Robust stochastic configuration networks with kernel density estimation for uncertain data regression. Inf. Sci. 412–413, 210–222 (2017)
Tao, X., Zhou, X., He, Y.L., Ashfaq, R.A.R.: Impact of variances of random weights and biases on extreme learning machine. J. Softw. 11(5), 440–454 (2016)
Balasundaram, S., Gupta, D.: On optimization based extreme learning machine in primal for regression and classification by functional iterative method. Int. J. Mach. Learn. Cybern. 7(5), 707–728 (2016)
Chen, Z.X., Zhu, H.Y., Wang, Y.G.: A modified extreme learning machine with sigmoidal activation functions. Neural Comput. Appl. 22(3–4), 541–550 (2013)
Wang, W., Liu, X.: The selection of input weights of extreme learning machine: a sample structure preserving point of view. Neurocomputing 261, 28–36 (2017)
Lichman, M.: UCI Machine Learning Repository. School of Information and Computer Science, University of California, Irvine (2013). http://archive.ics.uci.edu/ml
Yin, H., Gai, K., Wang, Z.: A classification algorithm based on ensemble feature selections for imbalanced-class dataset. In: The 2nd IEEE International Conference on High Performance and Smart Computing, New York, USA, pp. 245–249 (2016)
Yin, H., Gai, K.: An empirical study on preprocessing high-dimensional class-imbalanced data for classification. In: 2015 IEEE 17th International Conference on High Performance Computing and Communications; The IEEE International Symposium on Big Data Security on Cloud, New York, USA, pp. 1314–1319 (2015)
Acknowledgments
This research is supported by the National Natural Science Foundation of China under Grant nos. 61672358 and the key Project of DEGP nos. 2014GKCG031.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Cao, W., Gao, J., Ming, Z., Cai, S., Zheng, H. (2018). Impact of Probability Distribution Selection on RVFL Performance. In: Qiu, M. (eds) Smart Computing and Communication. SmartCom 2017. Lecture Notes in Computer Science(), vol 10699. Springer, Cham. https://doi.org/10.1007/978-3-319-73830-7_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-73830-7_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73829-1
Online ISBN: 978-3-319-73830-7
eBook Packages: Computer ScienceComputer Science (R0)