Advertisement

Three Conjectures on Neural Network Implementations of Volterra Models (Mappings)

  • Vasilis Z. Marmarelis

Abstract

Three conjectures are presented that form the methodological link between Volterra models (mappings) and a popular class of artificial neural networks (multi-layer perceptrons). The first conjecture elucidates the equivalence between these two types of nonlinear mapping and shows how we can achieve a network implementation of a Volterra model by employing proper linear input transformations and polynomial activation functions in the hidden units; while the output unit(s) may be simple adder(s). The second conjecture outlines the trade-offs between general polynomial and fixed sigmoidal activation functions traditionally used in multilayer perceptrons. The former are more flexible in defining nonlinear mappings; while thelatter, being far more restrictive, lead to increased numbers of hidden units and heavier computational burden during training via back-propagation. In general, an infinite number of sigmoidal hidden units is required to represent exactly a Volterra model (mapping) or a network with (finite) polynomial hidden units. The third conjecture extends the results for continous-output models to binary-(or spike-)output models/mappings, often encountered in neural networks. These conjectures collectively point to the potential versatility and efficiency of a class of networks that utilize polynomial activation functions in the hidden units and linear output unit(s) with fixed weights. Practical procedures for optimal use of these networks are currently developed.

Keywords

Input Vector Hide Unit Volterra Model Trigger Zone Hard Threshold 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Blum, E.K. and Li, L.K. (1991) Approximation theory and feedforward networks. Neural Networks, 4:511–515.CrossRefGoogle Scholar
  2. 2.
    Cybenko, G. (1989) Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals & Systems, 2:303–314.MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    de Figueiredo, R.J.P. (1980) Implications and applications of Kolmogorov’s superposition theorem. IEEE Trans. Autom. Contr., 25:1227–1231.zbMATHCrossRefGoogle Scholar
  4. 4.
    Funahashi, K.-I. (1989) On the approximate realization of continuous mappings by neural networks. Neural Networks, 2:183–192.CrossRefGoogle Scholar
  5. 5.
    Hornik, K., Stinchcombe, M. and White, H. (1989) Multi-layer feedforward networks are universal approximators. Neural Networks, 2:359–366.CrossRefGoogle Scholar
  6. 6.
    Ito, (1991) Neural Networks, 4:385–394.CrossRefGoogle Scholar
  7. 7.
    Kolmogorov, A.N. (1957) On the representation of continuous functions of several variables by superposition of continuous functions of one variable and addition. Doklady Akademii Nauk. SSSR, 114:953–956; AMS Transi. 2:55–59, (1963).MathSciNetzbMATHGoogle Scholar
  8. 8.
    Marmarelis, V.Z. (1989) Signal transformation and coding in neural systems. IEEE Trans. Biomed. Eng., 36:15–24.CrossRefGoogle Scholar
  9. 9.
    Marmarelis, VIZ. and Orme, M.E. (1993) Modeling of neuronal systems by use of neuronal modes. IEEE Trans. Biomed. Eng., 40:1149–1158.CrossRefGoogle Scholar
  10. 10.
    Rosenblatt, F. (1962) Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books, Washington, D.C.zbMATHGoogle Scholar
  11. 11.
    Rumelhart, D.E. and McClelland, J.L. (1986) (Eds.) Parallel Distributed Processing, Vol. I. and II., MIT Press, Cambridge, Massachussetts.Google Scholar
  12. 12.
    Sprecher, D.A. (1972) An improvement in the superposition theorem of Kolmogorov. /. Math. Anal. Amol, 38:208–213.MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Widrow, B. and Lehr, M.A. (1990) 30 years of adaptive neural networks: Perceptron, madaline and back propagation. Proc. IEEE, 78:1415–1442.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1994

Authors and Affiliations

  • Vasilis Z. Marmarelis
    • 1
  1. 1.Departments of Biomedical and Electrical EngineeringUniversity of Southern CaliforniaUSA

Personalised recommendations