Advertisement

An Overview of Radial Basis Function Networks

  • J. Ghosh
  • A. Nag
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 67)

Abstract

This chapter presents a broad overview of Radial Basis Function Networks (RBFNs), and facilitates an understanding of their properties by using concepts from approximation theory, catastrophy theory and statistical pattern recognition. While this chapter is aimed to provide an adequate theoretical background for the subsequent application oriented chapters in this book, it also covers several aspects with immediate practical implications: alternative ways of training RBFNs, how to obtain an appropriate network size for a given problem, and the impact of the resolution (width) of the radial basis functions on the solution obtained. Some prominent applications of RBFNs are also outlined.

Keywords

Basis Function Radial Basis Function Radial Basis Function Network Hide Unit Ridge Regression 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Ahmad, S. and Tresp, V. (1993), “Some solutions to missing feature problem in vision,” in Cowan, J.D., Hanson, S.J., and Giles, C.L. (Eds.), Advances in Neural Information Processing Systems, vol. 5, pp. 393–400. The MIT Press.Google Scholar
  2. [2]
    Bianchini, M., Frasconi, P., and Gori, M. (1995), “Learning without local minima in radial basis function networks,” IEEE Transactions on Neural Networks, vol. 6, no. 3, pp. 749–756.CrossRefGoogle Scholar
  3. [3]
    Beck, S. and Ghosh, J. (1992), “Noise sensitivity of static neural classifiers,” SPIE Conf. on Applications of Artificial Neural Networks, SPIE Proc. Vol. 1709, pp. 770–779, Orlando, Fl., April.Google Scholar
  4. [4]
    Bishop, C.M. (1995), Neural Networks for Pattern Recognition, Oxford University Press, New York.Google Scholar
  5. [5]
    Broomhead, D.S. and Lowe, D. (1988), “Multivariable functional interpolation and adaptive networks,” Complex Systems, vol. 2, pp. 321–355.MathSciNetMATHGoogle Scholar
  6. [6]
    Bor§, A.G. and Pitas, I. (1996), “Median radial basis function neural network,” IEEE Transactions on Neural Networks, vol. 7, no. 6, pp. 1351–1364, November.Google Scholar
  7. [7]
    Bakshi, B.R. and Stephanopoulos, G. (1993), “Wave-net: a multiresolution, hierarchical neural network with localized learning,” AIChE Journal, vol. 39, no. 1, pp. 57–81, January.Google Scholar
  8. [8]
    Le Cun, Y., Denker, J.S., and Solla, S.A. (1990), “Optimal brain damage,” Advances in Neural Information Processing Systems-2, pp. 598–605.Google Scholar
  9. [9]
    Chakaravathy, S.V. and Ghosh, J. (1996), “Scale based clustering using a radial basis function network,” IEEE Transactions on Neural Networks, vol. 5, no. 2, pp. 1250–61, Sept.Google Scholar
  10. [10]
    Chakaravathy, S.V. and Ghosh, J. (1997), “Function emulation using radial basis function networks,” Neural Networks, vol. 10, pp. 459–478, May.Google Scholar
  11. [11]
    Chakravarthy, S.V. (1996), On the role of singularities in neural networks, PhD thesis, Dept. of Elect. Comp. Eng., Univ of Texas, Austin, May.Google Scholar
  12. [12]
    Friedman, J.H. (1994), “An overview of predictive learning and function approximation,” in Cherkassky, V., Friedman, J.H., and Wechsler, H. (Eds.), From Statistics to Neural Networks, Proc. NATO/ASI Workshop, pp. 1–61. Springer Verlag.Google Scholar
  13. [13]
    Geman, S., Bienenstock, E., and Doursat, R. (1992), “Neural networks and the bias/variance dilemma,” Neural Computation, vol. 4, no. 1, pp. 1–58.CrossRefGoogle Scholar
  14. [14]
    Ghosh, J. and Chakravarthy, S.V. (1994), “The rapid kernel classifier: a link between the self-organizing feature map and the radial basis function network,” Jl. of Intelligent Material Systems and Structures, vol. 5, pp. 211–219, 2.Google Scholar
  15. [15]
    Ghosh, J., Deuser, L., and Beck, S. (1992), “A neural network based hybrid system for detection, characterization and classification of short-duration oceanic signals,” IEEE X. of Ocean Engineering, vol. 17, no. 4, pp. 351–363, October.Google Scholar
  16. [16]
    Gilmore, R. (1981), Catastrophe Theory for Scientists and Engineers, Wiley Interscience, New York.MATHGoogle Scholar
  17. [17]
    Ghahramani, Z. and Jordan, M. (1994), “Supervised learning from incomplete data via an em approach,” in Tesauro, G., Cowan, J.D., and Alspector, J. (Eds.), Advances in Neural Information Processing Systems, vol. 6, pp. 120–127. The MIT Press.Google Scholar
  18. [18]
    Golub, G. and Van Loan, C. (1989), Matrix Computations, John Hopkins University Press, Baltimore, MD.Google Scholar
  19. [19]
    Ghosh, J. and Tumer, K. (1994), “Structural adaptation and generalization in supervised feedforward networks,” fi. of Artificial Neural Networks, vol. 1, no. 4, pp. 431–458.Google Scholar
  20. [20]
    Haykin, S. (1994), Neural Networks: a Comprehensive Foundation, Macmillan, New York.MATHGoogle Scholar
  21. [21]
    Hoskins, J.C., Lee, P., and Chakravarthy, S.V. (1993), “Polynomial modelling behavior in radial basis function networks,” Proc. of World Congress on Neural Networks, pp. 693–699, Portland, OR, July.Google Scholar
  22. [22]
    Jong, J.S.R. and Sun, C.T. (1993), “Functional equivalence between radial basis function networks and fuzzy inference systems,” IEEE Transactions on Neural Networks, vol. 4, no. 1, pp. 156–159, January.Google Scholar
  23. [23]
    Krogh, A. and Hertz, J.A. (1992), “A simple weight decay can improve generalization,” in Hanson, S.J., Moody, J.E., and Lippmann, R.P. (Eds.), Advances in Neural Information Processing Systems-4, pp. 950–957. Morgan Kaufmann, San Mateo, CA.Google Scholar
  24. [24]
    Kowalski, J., Hartman, E., and Keeler, J. (1990), “Layered neural networks with gaussian hidden units as universal approximators,” Neural Computation, vol. 2, pp. 210–215.CrossRefGoogle Scholar
  25. [25]
    Krzyzak, A. and Linder, T. (1997), “Radial basis function networks and complexity regularization in function learning,” in Mozer, M.C., Jordan, M.I., and Petsche, T. (Eds.), Advances in Neural Information Processing Systems, vol. 9, p. 197.Google Scholar
  26. [26]
    Krzyzak, A., Linder, T., and Lugosi, G. (1996), “Nonparametric estimation and classification using radial basis function nets and empirical risk minimization,” IEEE Transactions on Neural Networks, vol. 7, no. 2, pp. 475–487, March.Google Scholar
  27. [27]
    Kadirkamanathan, V. and Niranjan, M. (1993), “A function estimation approach to sequential learning with neural networks,” Neural Computation, vol. 5, pp. 954–975.CrossRefGoogle Scholar
  28. [28]
    Light, W.A. (1992), “Some aspects of radial basis function approximation,” in Singh, S.P. (Ed.), Approximation Theory, Spline Functions and Applications, pp. 163–90. NATO ASI Series Vol. 256, Kluwer Acad., Boston.Google Scholar
  29. [29]
    Lippmann, R.P. (1991), “A critical overview of neural network pattern classifiers,” IEEE Workshop on Neural Networks for Signal Processing.Google Scholar
  30. [30]
    Lee, S. and Kil, R.M. (1988), “Multilayer feedforward potential function network,” Proceedings of the Second International Conference on Neural Networks, pp. 161–171.Google Scholar
  31. [31]
    Lee, S. and Kil, R.M. (1991), “A Gaussian potential function network with hierarchical self-organizing learning,” Neural Networks, vol. 4, pp. 207–224.CrossRefGoogle Scholar
  32. [32]
    Leonard, J.A., Kramer, M.A., and Ungar, L.H. (1992), “Using radial basis functions to approximate a function and its error bounds,” IEEE Transactions on Neural Networks, vol. 3, no. 4, pp. 624–627, July.Google Scholar
  33. [33]
    Mackay, D.J.C. (1995), “Probable networks and plausible predictions–a review of practical Bayesian methods for supervised neural networks,” Network: Computation in Neural Systems, vol. 6, no. 3, pp. 469–505.MATHCrossRefGoogle Scholar
  34. [34]
    Moody, J. and Darken, C.J. (1989), “Fast learning in networks of locally-tuned processing units,” Neural Computation, vol. 1, no. 2, pp. 281–294.CrossRefGoogle Scholar
  35. [35]
    Megdassy, P. (1961), Decomposition of superposition of distributed functions, Hungarian Academy of Sciences, Budapest.Google Scholar
  36. [36]
    Micchelli, C.A. (1986), “Interpolation of scattered data: distance matrices and conditionally positive definite functions,” Constructive Approximation, vol. 2, pp. 11–22.MathSciNetMATHCrossRefGoogle Scholar
  37. [37]
    Molina, C. and Niranjan, M. (1996), “Pruning with replacement on limited resource allocating networks by F-projections,” Neural Computation, vol. 8, pp. 855–868.CrossRefGoogle Scholar
  38. [38]
    Moody, J.E. (1994), “Prediction risk and architecture selection for neural networks,” in Cherkassky, V., Friedman, J.H., and Wechsler, H. (Eds.), From Statistics to Neural Networks, Proc. NATO/ASI Workshop, pp. 143–156. Springer Verlag.Google Scholar
  39. [39]
    Mulgrew, B. (1996), “Applying radial basis functions,” IEEE Signal Processing Magazine, pp. 50–65, March.Google Scholar
  40. [40]
    Nadaraya, E.A. (1964), “On estimating regression,” Theory of Probability and its Applications, vol. 9, no. 1, pp. 141–142,.Google Scholar
  41. [41]
    Niyogi, P. and Girosi, F. (1996), “On the relationship between generalization error, hypothesis complexity and sample complexity for radial basis functions,” Neural Computation, vol. 8, pp. 819–842.CrossRefGoogle Scholar
  42. [42]
    Osman, H. and Fahmy, M.M. (1994), “Probabilistic winner-takeall learning algorithm for radial-basis-function neural classifiers,” Neural Computation, vol. 6, no. 5, pp. 927–943.MATHCrossRefGoogle Scholar
  43. [43]
    Orr, M.J.L. (1995), “Regularization in the selection of radial basis function centers,” Neural Computation, vol. 7, pp. 606–623.CrossRefGoogle Scholar
  44. [44]
    Orr, M.J.L. (1996), “Introduction to radial basis function networks,” Technical Report April, Center for Cognitive Science, Univ. of Edinburgh.Google Scholar
  45. [45]
    Poggio, T. and Girosi, F. (1990), “Networks for approximation and learning,” Proc. IEEE, vol. 78, no. 9, pp. 1481–97, Sept.Google Scholar
  46. [46]
    Platt, J.C. (1991), “A resource allocation network for function interpolation,” Neural Computation, vol. 3, no. 2, pp. 213–225.MathSciNetCrossRefGoogle Scholar
  47. [47]
    Pal, S.K. and Mitra, S. (1992), “Multilayer perceptron, fuzzy sets, and classification,” IEEE Transactions on Neural Networks, vol. 3, no. 5, pp. 683–697, September.Google Scholar
  48. [48]
    Powell, M.J.D. (1985), “Radial basis functions for multivariable interpolation: a review,” IMA Conf on Algorithms for the approximation of functions and data, pp. 143–167.Google Scholar
  49. [49]
    Park, J. and Sandberg, I.W. (1991), “Universal approximation using radial basis function networks,” Neural Computation, vol. 3, no. 2, pp. 246–257, Summer.Google Scholar
  50. [50]
    Park, J. and Sandberg, I.W. (1993), “Universal approximation and radial basis function networks,” Neural Computation, vol. 5, no. ??, pp. 305–316.Google Scholar
  51. [51]
    Reed, R. (1993), “Pruning algorithms — a survey,” IEEE Transactions on Neural Networks, vol. 4, no. 5, pp. 740–747, September.Google Scholar
  52. [52]
    Rosenblum, M., Yacoob, Y., and Davis, L.S. (1996), “Human expression recognition from motion using a radial basis function network architecture,” IEEE Transactions on Neural Networks, vol. 7, no. 5, pp. 1121–1138, September.Google Scholar
  53. [53]
    Shim, C. and Cheung, J.Y. (1995), “Pattern classification using RBF based fuzzy neural network,” Intelligent Engineering Systems Through Artificial Neural Networks, vol. 5, pp. 485–90. ASME Press, November.Google Scholar
  54. [54]
    Schioler, H. and Hartmann, U. (1992), “Mapping neural network derived from the parzen window estimator,” Neural Networks, vol. 5, pp. 903–909.CrossRefGoogle Scholar
  55. [55]
    Specht, J. (1990), “Probabilistic neural networks,” Neural Networks, vol. 3, pp. 45–74.CrossRefGoogle Scholar
  56. [56]
    Tresp, R., Neuneier, V., and Ahmad, S. (1995), “Efficient methods for dealing with missing data in supervised learning,” in Touretzky, D.S., Tesauro, G., and Leen, T.K. (Eds.), Advances in Neural Information Processing Systems, vol. 7, pp. 687–696. The MIT Press.Google Scholar
  57. [57]
    Taha, I. and Ghosh, J. (1997), “Hybrid intelligent architecture and its application to water reservoir control,” International Journal of Smart Engineering Systems, vol. 1, pp. 59–75, 1.Google Scholar
  58. [58]
    Thom, R. (1975), Structural Stability and Morphogenesis, Reading: Benjamin.MATHGoogle Scholar
  59. [59]
    Tresp, S., Ahmad, V., and Neuneier, R. (1994), “Training neural networks with deficient data,” in Tesauro, G., Cowan, J.D., and Alspector, J. (Eds.), Advances in Neural Information Processing Systems, vol. 6, pp. 128–135. The MIT Press.Google Scholar
  60. [60]
    Wahba, G. (1990), Spline Models for Observational Data, vol. 59. Society for Industrial and Applied Mathematics, Philadephia.MATHCrossRefGoogle Scholar
  61. [61]
    Watson, G.S. (1964), “Smooth regression analysis,” Sankhya: the Indian Journal of Statistics, Series A 26, pp. 359–372.Google Scholar
  62. [62]
    Webb, A.R. (1996), “An approach to non-linear principal component analysis using radially symmetric kernel functions,” Statistics and Computing, vol. 9, pp. 159–68.CrossRefGoogle Scholar
  63. [63]
    Witkin, A.P. (1983), “Scale-space filtering,” 8th Int. Joint Conf. Art. Intell., pp. 1019–1022, Karlsruhe, Germany, August.Google Scholar
  64. [64]
    Wong, Y.F. (1993), “Clustering data by melting,” Neural Computation, vol. 5, no. 1, pp. 89–104.CrossRefGoogle Scholar
  65. [65]
    Webb, A.R. and Shannon, S.M. (1996), “Spherically-symmetric basis functions for discrimination and regression: determining the nonlinearity,” Technical Report 651/2/JP95/22, DRA/CIS, Malvern, England.Google Scholar
  66. [66]
    Xu, L., Krzyzak, A., and Yuille, A. (1994), “On radial basis function nets and kernel regression: statistical consistency, convergence rates and receptive field size,” Neural Networks, vol. 7, no. 4, pp. 609–628.MATHCrossRefGoogle Scholar
  67. [67]
    Yingwei, L., Sunderararajan, N., and Saratchandran, P. (1997), “A sequential learning scheme for function approximation using minimal radial basis function networks,” Neural Computation, vol. 9, pp. 461–78.MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • J. Ghosh
  • A. Nag

There are no affiliations available

Personalised recommendations