Unlearning in Feed-Forward Multi-Nets
Multi-nets promise an improved performance over monolithic neural networks by virtue of their distributed implementation. Modular neural networks are multi-nets based on an judicious assembly of functionally different parts. This can be viewed as again a monolithic network, but with more complex neurons (the neural modules). Therefore they will share the same learning problems, notably the unlearning effect. In this paper we will look more closely into the reasons for unlearning and discuss how this can be applied to detect novelties.
KeywordsHide Neuron Modular Network Modular Neural Network Neural Module Hide Feature
Unable to display preview. Download preview PDF.
- Caelli, T. Guan, L. Wan, W.: Modularity in Neural Computing, Proc. of the IEEE 87, pp. 1497–1518 (1999).Google Scholar
- Macready, W.G. Siapas, A.G. Kauffman, S.A.: Criticality and parallelism in combinatorial optimization, Science 271, pp. 56–59 (1996).Google Scholar
- Barakova, E.I.: Learning Reliability: a study on indecisiveness in sample selection, Ph.D. thesis (Groningen University, Groningen) 1999.Google Scholar
- Schuermann, B.: Applications and Perspectives of Artificial Neural Networks, VDI Berichte 1526, pp. 1–14 (2000).Google Scholar
- Sprinkhuizen-Kuyper, I.G. Boers, E.J.W.: A local minimum for the 2-3-1 XOR network, IEEE Tr. on Neural Networks 10, pp. 968–971, (1999).Google Scholar
- Venema, R.S. Spaanenburg, L.: Learning feedforward networks, this proceeding.Google Scholar
- Spaanenburg, L. Jansen, W.J. Nijhuis, J.A.G.: Over multiple rule-blocks to modular nets, Proceedings EUROMICRO’97, pp. 698–705 (1997).Google Scholar
- Yamauchi, K. Yamaguchi, Y. Ishii, N.: Incremental learning methods with retrieving of interfered patterns, IEEE Tr. on Neural Networks 10, pp. 1351–1365 (1999).Google Scholar
- TerBrugge, M.H. Nijhuis, J.A.G. Spaanenburg, L.: License-Plate Recognition, pp. 263–296, in: Jain, L.C. Lazzarini, B.: Intelligent Techniques in Character Recognition: Practical Applications, CRC Press 199Google Scholar
- Yu, X.L. Chen, G.K.C. Cheng, S.: Dynamic learning rate optimization of the back-propagation Algorithm, IEEE Tr. on Neural Networks 6, pp. 669–677 (1995).Google Scholar
- VanVeelen, M. Nijhuis, J.A.G. Spaanenburg, L.: Process fault detection through quantitative analysis of learning in neural networks, Proceedings ProRISC’2000, pp. 557–565 (2000).Google Scholar