An Ensemble of Degraded Neural Networks
In this paper we present a new method to create neural network ensembles. In an ensemble method like bagging one needs to train multiple neural networks to create the ensemble. Here we present a scheme to generate different copies of a network from one trained network, and use those copies to create the ensemble. The copies are produced by adding controlled noise to a trained base network. We provide a preliminary theoretical justification for our method and experimentally validate the method on several standard data sets. Our method can improve the accuracy of a base network and give rise to considerable savings in training time compared to bagging.
KeywordsNeural Network Parameter Vector Training Time Ensemble Method Base Network
- 2.Asuncion, A., Newman, D.J.: UCI machine learning repository (2007)Google Scholar
- 4.Chen, R., Yu, J.: An improved bagging neural network ensemble algorithm and its application. In: Third International Conference on Natural Computation, vol. 5, pp. 730–734 (2007)Google Scholar
- 5.Drucker, H., Schapire, R.E., Simard, P.: Improving performance in neural networks using a boosting algorithm. In: Hanson, S.J., Cowan, J.D., Lee Giles, C. (eds.) NIPS, pp. 42–49. Morgan Kaufmann, San Francisco (1992)Google Scholar
- 7.Gao, H., Huang, D., Liu, W., Yang, Y.: Double rule learning in boosting. International Journal of Innovative Computing, Information and Control 4(6), 1411–1420 (2008)Google Scholar
- 14.Schapire, R.E.: A brief introduction to boosting. In: Dean, T. (ed.) IJCAI, pp. 1401–1406. Morgan Kaufmann, San Francisco (1999)Google Scholar