Unsupervised and Supervised Learning in Radial-Basis-Function Networks
Learning in radial basis function (RBF) networks is the topic of this chapter. Whereas multilayer perceptrons (MLP) are typically trained with backpropagation algorithms, starting the training procedure with a random initalization of the MLP’s parameters, an RBF network may be trained in different ways. We distinguish one-, two-, and three phase learning.
A very common learning scheme for RBF networks is two phase learning. Here, the two layers of an RBF network are trained separately. First the RBF layer is calculated, including the RBF centers and scaling parameters, and then the weights of the output layer are adapted. The RBF centers may be trained through unsupervised or supervised learning procedures utilizing clustering, vector quantization or classification tree algorithms. The output layer of the network is adapted by supervised learning. Numerical experiments of RBF classifiers trained by two phase learning are presented for the classification of 3D visual objects and the recognition of hand-written digits. It can be observed that the performance of RBF classifiers trained with two phase learning can be improved through a third backpropagation-like learning phase of the RBF network, adapting the whole set of parameters (RBF centers, scaling parameters, and output layer weights) simultaneously. This, we call three phase learning in RBF networks. A practical advantage of two and three phase learning in RBF networks is the possibility to use unlabeled training data for the first training phase.
Support vector (SV) learning in RBF networks is a special type of one phase learning, where only the output layer weights of the RBF network are calculated, and the RBF centers are restricted to be a subset of the training data.
Numerical experiments with several classifier schemes including nearest neighbor classifiers, learning vector quantization networks and RBF classifiers trained through two phase, three phase and support vector learning are given. The performance of the RBF classifiers trained through SV learning and three phase learning are superior to the results of two phase learning.
KeywordsRadial Basis Function Supervise Learn Phase Learning Radial Basis Function Network Kernel Width
Unable to display preview. Download preview PDF.
Bibliography on Chapter 10
- 2.C. M. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, Oxford, 1995.Google Scholar
- 6.N. Cristianini and J. Shawe-Taylor. An introduction to support vector machines. Cambridge University Press, 2000.Google Scholar
- 7.C. Darken and J. Moody. Fast adaptive k-means clustering: Some empirical results. Proceedings International Joint Conference on Neural Networks, 1990.Google Scholar
- 10.O. Ghitza. Auditory nerve representation as a basis for speech recognition. In S. Furui and M. Sondhi, editors, Advances in Speech Signal Processing, pages 453–485. Marcel Dekker, NY, 1991.Google Scholar
- 12.J. Hertz, A. Krogh, and R. G. Palmer. Introduction to the Theory of Neural Computation. Addison Wesley, New York, 1991.Google Scholar
- 14.T. Kohonen. Self-Organizing Maps. Springer, 1995.Google Scholar
- 16.W.A. Light. Some aspects of radial basis function approximation. In S.P. Singh, editor, Approximation Theory, Spline Functions and Applications, pages 163–190. Kluwer, 1992.Google Scholar
- 17.T.-J. Lim, W.-Y. Loh, and Y-S-. Shih. A comparision of prediction accuracy, complexity, and training time of thirty-tree old and new classification algorithms. Machine Learning, pages 1–27, 2000.Google Scholar
- 21.D. Man. Vision. Freeman, San Fransisco, 1982.Google Scholar
- 22.D. Man and H.K. Nishihara. Representation and recognition of the spatial organization of three dimensional structure. Proceedings of the Royal Society of London B, 200: 269–294, 1978.Google Scholar
- 25.D. Michie, D.J. Spiegelhalter, and C.C. Taylor. Machine Learning, Neural and Statistical Classification. Ellis Horwood, 1994.Google Scholar
- 33.M. J. D. Powell. The Theory of Radial Basis Function Approximation in 1990. In W. Light, editor, Advances in Numerical Analysis, volume II, pages 105–210. Oxford Science Publications, 1992.Google Scholar
- 34.L. Rabiner and B.-H. Juang. Fundamentals of Speech Recognition. Prentice Hall, 1993.Google Scholar
- 35.B.D. Ripley. Pattern Recognition and Neural Networks. Cambridge University Press, 1996.Google Scholar
- 36.B. Schiele and J. Crowley. Probabilistic object recognition using multidimensional receptive field histograms. In Proc. of the 13th Int. Conf. on Pattern Recognition, pages 50–54. IEEE Computer Press, 1996.Google Scholar
- 37.A. Schölkopf, C. Burges, and A. Smola. Advances in Kernel Methods - Support Vector Learning. MIT Press, 1998.Google Scholar
- 38.F. Schwenker. Hierarchical Support Vector Machines for Multi-Class Pattern Recognition. In R.J. Howlett and L.C. Jain, editors, Knowledge-Based Intelligent Engineering Systems and Aplied Technologies KES 2000, pages 561–565. 2000.Google Scholar
- 39.F. Schwenker and H.A. Kestler. 3-D Visual Object Classification with Hierarchical RBF Networks. In R.J. Howlett and L.C. Jain, editors, Radial Basis Function Neural Networks: Theory and Applications. Physica-Verlag, 2000 (in press).Google Scholar
- 40.F. Schwenker, H.A. Kestler, G. Palm, and M. Höher. Similarities of LVQ and RBF learning. In Proc. IEEE Int. Conf. SMC, pages 646–651, 1994.Google Scholar
- 42.V.N. Vapnik. Statistical Learning Theory. John Wiley and Sons, 1998.Google Scholar