Abstract
This paper deals with improving generalisation performances of feed forward neural networks (FFNN) on real world data domains using more complex architectures for modelling. The convention in neural networks is to use as small an architecture as possible to force better generalisation by modelling the underlying distribution and ignoring the details [1]. This practice involves the loss of information from the training data which in real world domains may represent important though poorly represented decision regions. The problem with introducing extra free parameters (more neurons and weights) to a network is that over-fitting can occur causing the network to model the training data too closely and generalise badly on new data from the same domain. This problem is overcome by combining a number of FFNN (with small architectures) that have been trained on the same data, though generalise differently, to produce more complex decision regions and improved generalisation. Committee decision theory is used to produce the combined model and has been shown to give promising results in the past [2][3][4].
A real world medical data set consisting of non discrete attribute values and FFNN trained using Back Propagation (BP) [5] were used to test the validity of the concepts presented.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hertz J., Krogh A. and Palmer R.: Introduction to the Theory of Neural Computation, Sante Fe Institute, Addison Wesley, 1991.
Wolpert D.: Stacked Generalisation, Neural Networks, Vol. 5, p. 241, 1992.
LeBlanc M, Tibshirani R: Combining Estimates in Regression and Classification, Univ. Toronto Statistics Dept., Technical Report. 1993.
Battiti R, Colla A: Democracy in Neural Nets: Voting Schemes for Clasification, Neural Networks, 7, pp. 691–707, 1994.
Rumelhart D., Hinton G., Williams R.: Learning Representations by Back-Propagating Errors, Letters to Nature, vol. 323, pp. 533–535, 1986.
McLean D., Bandar Z., O’Shea J.: Improved Interpolation and Extrapolation from Continuous Training Examples Using a New Neuronal Model with an Adaptive Steepness, 2nd Australian and New Zealand Conference on Intelligent Information Systems, IEEE, pp. 125–129, 1994.
McLean D., Bandar Z., O’Shea J.: An Empirical Comparison of Back Propagation and the RDSE Algorithm on Continuously Valued Real World Data, Neural Networks, vol. 11, pp. 1685–1694, 1998.
Martin G., Pittman J.: Recognizing Hand-Printed Letters and Digits, Advances in Neural Information Processing Systems, II., pp. 405–414, 1990.
Tesauro G., Sejinowski T.J.: A Parallel Network that Learns to Play Backgammon, Artificial Intelligence, No 39, pp. 357–390, 1988.
Morgan N., Bourland H.: Generalization and Parameter Estimation in Feed Forward Nets: Some Experiments, Advances in Neural Information Processing Systems, II., pp. 405–414, 1990.
McLean D., Bandar Z., O’Shea L: A Constructive Decision Boundary Modelling Algorithm, IASTED ‘98, Mexico. 1998.
McLean D., Bandar Z., O’Shea L: The Evolution of a Feed Forward Neural Network Trained under Back Propagation’, ICANNGA′97, Springer-Verlag, 1997.
Michie D., Spiegelhalter D.J., Taylor C.C.: Machine Learning, Neural and Statistical Classification, Ellis Hopwood Series in Artificial Intelligence, Ellis Hopwood, 1994.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Wien
About this paper
Cite this paper
McLean, D., Bandar, Z., O’Shea, J. (1999). Improving Generalisation Using Modular Neural Networks. In: Artificial Neural Nets and Genetic Algorithms. Springer, Vienna. https://doi.org/10.1007/978-3-7091-6384-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-7091-6384-9_7
Publisher Name: Springer, Vienna
Print ISBN: 978-3-211-83364-3
Online ISBN: 978-3-7091-6384-9
eBook Packages: Springer Book Archive