Advertisement

Tightness of Upper Bounds on Rates of Neural-Network Approximation

  • Věra Kůrková
  • Marcello Sanguineti

Abstract

Tightness of upper bounds on neural network approximation is investigated in the framework of variable-basis approximation. Conditions are given on a variable basis that do not allow a possibility of improving such bounds beyond \(O\left( {{n^{ - \left( {\frac{1}{2} + \frac{1}{d}} \right)}}} \right)\) where d is the number of variables of the functions to be approximated. Such conditions are satisfied by Lipschitz sigmoidal perceptrons.

Keywords

Neural Network Hide Unit Normed Linear Space Orthogonal Element Multivariable Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Barron, A.R.: Neural net approximation. Proc. 7th Yale Work. on Adaptive and Learning Systems (K. Narendra, Ed.), pp. 69–72. Yale Univ. Press, 1992.Google Scholar
  2. [2]
    Barron, A.R.: Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. on Information Th. 39, pp. 930–945, 1993.MathSciNetCrossRefMATHGoogle Scholar
  3. [3]
    Darken, C., Donahue, M., Gurvits, L., Sontag, E.: Rate of approximation results motivated by robust neural network learning. Proc. Sixth Annual ACM Conf. on Computational Learning Th.. The Association for Computing Machinery, New York, N.Y., pp. 303–309, 1993.Google Scholar
  4. [4]
    DeVore, R.A., Temlyakov, V.N.: Nonlinear approximation by trigonometric sums. The J. of Fourier Analysis and Appl. 2, pp. 29–48, 1995.MathSciNetCrossRefMATHGoogle Scholar
  5. [5]
    Girosi, F.:Approximation error bounds that use VC-bounds. Proc. Int. Conf. on Artificial Neural Networks ICA NN’ 95. Paris: EC2 & Cie, pp. 295–302, 1995.Google Scholar
  6. [6]
    Gurvits, L., Koiran, P.: Approximation and learning of convex superpositions. J. of Computer and System Sciences55, pp. 161–170, 1997.MathSciNetCrossRefMATHGoogle Scholar
  7. [7]
    Jones, L.K.: A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Annals of Statistics 20, pp. 608–613, 1992.MathSciNetCrossRefMATHGoogle Scholar
  8. [8]
    Kainen, P.C., Kůrková, V.: Quasiorthogonal dimension of Euclidean spaces. Appl. Math. Lett. 6, pp. 7–10, 1993.CrossRefMATHGoogle Scholar
  9. [9]
    KůFrková, V.: Dimension-independent rates of approximation by neural networks. In Computer-Intensive Methods in Contral and Signal Processing. The Curse of Dimensionality (K. Warwick M. Kárny, Eds.). Birkhauser, Boston, pp. 261–270, 19Google Scholar
  10. [10]KůFrková, V., Sanguineti, M.: Tools for comparing neural network and linear approximation. Submitted to IEEE Trans. on Information Th.Google Scholar
  11. [11]KůFrková, V., Sanguineti, M.: Bounds on rates of variable-basis and neural-network approximation. To appear in IEEE 7rans. on Information Th.Google Scholar
  12. [12]
    KůFrková, V., Sanguineti, M.: Covering numbers and rates of neural-network approximation. Research Report ICS-00-830, 2000.Google Scholar
  13. [13]
    KůFrková, V., Savický, P., Hlaváčková, K.: Representations and rates of approximation of realvalued Boolean functions by neural networks. Neural Networks 11, pp. 651–659, 1998.CrossRefGoogle Scholar
  14. [14]
    Makovoz, Y.: Random approximants and neural networks. J. of Approx. Th. 85, pp. 98–109, 1996.MathSciNetCrossRefMATHGoogle Scholar
  15. [15]
    Makovoz, Y.: Uniform approximation by neural networks. J. of Approx. Th. 95, pp. 215–228, 1998.MathSciNetCrossRefMATHGoogle Scholar
  16. 16]
    Mhaskar, H.N. Micchelli, C.A.: Dimension independent bounds on the degree of approximation by neural networks. IBM J. of Research and Development 38, n. 3, pp. 277–283, 1994CrossRefMATHGoogle Scholar
  17. [17]
    Pisier, G.: Remarques sur un resultat non publié de B. Maurey. Seminaire d’Analyse Fonctionelle, vol. I, no. 12. École Polytechnique, Centre de Mathématiques, Palaiseau, 1980-81.Google Scholar

Copyright information

© Springer-Verlag Wien 2001

Authors and Affiliations

  • Věra Kůrková
    • 1
  • Marcello Sanguineti
    • 2
  1. 1.Institute of Computer ScienceAcademy of Sciences of the Czech RepublicPrague 8Czech Republic
  2. 2.DISTUniversity of GenoaGenoaItaly

Personalised recommendations