What Can a Neural Network Do without Scaling Activation Function?
It has been proved that a three layered feedforward neural network having a sigmoid function as its activation function can 1) realize any mapping of arbitrary n points in R d into R, 2) approximate any continuous function defined on any compact subset of R d, and 3) approximate any continuous function defined on the compactification of R d , without scaling the activation function ,,,. In this paper, these results are extended doubly in the sense that the activation function defined on R is not restricted to sigmoid functions and the concept of activation function is extended to functions defined on higher dimensional spaces R c (1 ≤ c ≤ d). In this way sigmoid functions and radial basis functions are treated on a common basis. The conditions on the activation function for 1) and 2) are exactly the same, but more restricted for 3).
KeywordsRadial Basis Function Activation Function Discriminatory Function Sigmoid Function Layer Unit
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Cybenko, G., Approximation by superpositions of sigmoidal function. Math. Control Signals Systems, 2, 303–314. (1989)MathSciNetCrossRefMATHGoogle Scholar
Hornik, K., Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366. (1989)CrossRefGoogle Scholar
Ito, Y., Representation of functions by superpositions of a step or sigmoid function and their applications to neural network theory. Neural Networks 4, 385–394. (1991a)CrossRefGoogle Scholar
Ito, Y., Approximation of functions on a compact set by finite sums of a sigmoid function without sealing. Neural Networks 4, 817–826. (1991b)CrossRefGoogle Scholar
Ito, Y., Finite mapping by neural networks and truth functions. Math. Scien. 17, 69–77. (1992a)MATHGoogle Scholar
Ito, Y., Approximation of continuous functions on Rd
by linear combinations of shifted rotations of a sigmoid function with and without scaling. Neural Networks 5, 105–115. (1992b)CrossRefGoogle Scholar
Ito, Y., Nonlinearity creates linear independence. Adv. Compt. Math. 5, 189–203. (1996)CrossRefMATHGoogle Scholar
Ito, Y., Independenee of unscaled basis functions and finite mappings by neural networks, to appear in Math. Scien. 26 (2001).Google Scholar
Ito, Y., Approximations with unscaled basis functions on compact sets, to appear.Google Scholar
Ito, Y., Approximations with unscaled basis functions on Rd
, to appear.Google Scholar
Ito, Y. K. Saito, Superposition of linearly independent functions and finite mapping by neural networks. Math. Scientists 21, 27–33. (1996)MathSciNetMATHGoogle Scholar
Kůrková, V., P. Savický K. Hlaváčková, Representations and rates of approximation of realvalued Boolean functions by neural networks, Neural Networks 11, 651–659, (1998).CrossRefGoogle Scholar
Rudin, W., Real and complex analysis. New York: McGraw-Hill Book Company.(1966)MATHGoogle Scholar
Rudin, W., Functional analysis. New York: McGraw-Hill Book Company.(1973)MATHGoogle Scholar
Sussmann, H.J.,. Uniqueness of the weights for minimal feedforward nets with a given input-output map.Neural Networks 5, 0589–593. (1992)CrossRefGoogle Scholar
© Springer-Verlag Wien 2001