What Can a Neural Network Do without Scaling Activation Function?

  • Yoshifusa Ito
Conference paper


It has been proved that a three layered feedforward neural network having a sigmoid function as its activation function can 1) realize any mapping of arbitrary n points in R d into R, 2) approximate any continuous function defined on any compact subset of R d, and 3) approximate any continuous function defined on the compactification of R d , without scaling the activation function [3],[4],[6],[11]. In this paper, these results are extended doubly in the sense that the activation function defined on R is not restricted to sigmoid functions and the concept of activation function is extended to functions defined on higher dimensional spaces R c (1 ≤ cd). In this way sigmoid functions and radial basis functions are treated on a common basis. The conditions on the activation function for 1) and 2) are exactly the same, but more restricted for 3).


Radial Basis Function Activation Function Discriminatory Function Sigmoid Function Layer Unit 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Cybenko, G., Approximation by superpositions of sigmoidal function. Math. Control Signals Systems, 2, 303–314. (1989)MathSciNetCrossRefMATHGoogle Scholar
  2. [2]
    Hornik, K., Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366. (1989)CrossRefGoogle Scholar
  3. [3]
    Ito, Y., Representation of functions by superpositions of a step or sigmoid function and their applications to neural network theory. Neural Networks 4, 385–394. (1991a)CrossRefGoogle Scholar
  4. [4]
    Ito, Y., Approximation of functions on a compact set by finite sums of a sigmoid function without sealing. Neural Networks 4, 817–826. (1991b)CrossRefGoogle Scholar
  5. [5]
    Ito, Y., Finite mapping by neural networks and truth functions. Math. Scien. 17, 69–77. (1992a)MATHGoogle Scholar
  6. [6]
    Ito, Y., Approximation of continuous functions on Rd by linear combinations of shifted rotations of a sigmoid function with and without scaling. Neural Networks 5, 105–115. (1992b)CrossRefGoogle Scholar
  7. [7]
    Ito, Y., Nonlinearity creates linear independence. Adv. Compt. Math. 5, 189–203. (1996)CrossRefMATHGoogle Scholar
  8. [8]
    Ito, Y., Independenee of unscaled basis functions and finite mappings by neural networks, to appear in Math. Scien. 26 (2001).Google Scholar
  9. [9]
    Ito, Y., Approximations with unscaled basis functions on compact sets, to appear.Google Scholar
  10. [10]
    Ito, Y., Approximations with unscaled basis functions on Rd, to appear.Google Scholar
  11. [11]
    Ito, Y. K. Saito, Superposition of linearly independent functions and finite mapping by neural networks. Math. Scientists 21, 27–33. (1996)MathSciNetMATHGoogle Scholar
  12. [12]
    Kůrková, V., P. Savický K. Hlaváčková, Representations and rates of approximation of realvalued Boolean functions by neural networks, Neural Networks 11, 651–659, (1998).CrossRefGoogle Scholar
  13. [13]
    Rudin, W., Real and complex analysis. New York: McGraw-Hill Book Company.(1966)MATHGoogle Scholar
  14. [14]
    Rudin, W., Functional analysis. New York: McGraw-Hill Book Company.(1973)MATHGoogle Scholar
  15. [15]
    Sussmann, H.J.,. Uniqueness of the weights for minimal feedforward nets with a given input-output map.Neural Networks 5, 0589–593. (1992)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Wien 2001

Authors and Affiliations

  • Yoshifusa Ito
    • 1
  1. 1.Aichi-Gakuin UniversityNisshin-shi, Aichi-kenJapan

Personalised recommendations