Repeated Measures Multiple Comparison Procedures Applied to Model Selection in Neural Networks
One of the main research concern in neural networks is to find the appropriate network size in order to minimize the trade-off between overfitting and poor approximation. In this paper the choice among different competing models that fit to the same data set is faced when statistical methods for model comparison are applied. The study has been conducted to find a range of models that can work all the same as the cost of complexity varies. If they do not, then the generalization error estimates should be about the same among the set of models. If they do, then the estimates should be different and our job would consist on analyzing pairwise differences between the least generalization error estimate and each one of the range, in order to bound the set of models which might result in an equal performance. This method is illustrated applied to polynomial regression and RBF neural networks.
Unable to display preview. Download preview PDF.
- 1.Chen, T., Seneta, E.: A stepwise rejective test procedure with strong control of familywise error rate. University of Sidney, School of Mathematics and Statistics, Research Report 99-9, March (1999)Google Scholar
- 4.Efron, B., Tibshirani, R.: Introduction to the Bootstrap, Chapman & Hall, (1993)Google Scholar
- 5.Girden, E.R.: Anova Repeated Measures, Sage Publications (1992)Google Scholar
- 6.Hochberg, Y., Tamhane A.C.: Multiple Comparison Procedures, Wiley (1987)Google Scholar
- 7.Hollander, M., Wolfe, D.A.: Nonparametric Statistical Methods, Wiley (1999)Google Scholar
- 8.Kearns, M., Mansour, Y.: An experimental and theorical comparison of model selection methods. Machine Learning, 27(1), (1997)Google Scholar
- 9.Minke, A.: Conducting Repeated Measures Analyses: Experimental Design Considerations, Annual Meeting of the Southwest Educational Research Association, Austin, (1997)Google Scholar
- 10.Pizarro, J., Guerrero, E., Galindo, P.: A statistical model selection strategy applied to neural networks. Proceedings of the European Symposium on Artificial Neural Networks Vol 1, pp. 55–60, Bruges (2000)Google Scholar
- 11.Vila, J.P., Wagner, V., Neveu, P.:Bayesian nonlinear model selection and neural networks: a conjugate prior approach. IEEE Transactions on neural networks, vol 11,2, march (2000)Google Scholar
- 12.Zar, J.H.: Biostatistical Analysis, Prentice Hall (1996)Google Scholar