Hybrid Learning of Regularization Neural Networks

  • Petra Vidnerová
  • Roman Neruda
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6114)


Regularization theory presents a sound framework to solving supervised learning problems. However, the regularization networks have a large size corresponding to the size of training data. In this work we study a relationship between network complexity, i.e. number of hidden units, and approximation and generalization ability. We propose an incremental hybrid learning algorithm that produces smaller networks with performance similar to original regularization networks.


Hybrid Algorithm Generalization Ability Hide Unit Small Network Reproduce Kernel Hilbert Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tikhonov, A., Arsenin, V.: Solutions of Ill-posed Problems. W.H. Winston, Washington (1977)zbMATHGoogle Scholar
  2. 2.
    Poggio, T., Girosi, F.: A theory of networks for approximation and learning. Technical report, Cambridge, MA, USA (1989); A. I. Memo No. 1140, C.B.I.P. Paper No. 31Google Scholar
  3. 3.
    Poggio, T., Smale, S.: The mathematics of learning: Dealing with data. Notices of the AMS 50, 536–544 (2003)MathSciNetGoogle Scholar
  4. 4.
    Xu, L., Krzyżak, A., Yuille, A.: On radial basis function nets and kernel regression: statistical consistency, convergence rates, and receptive field size. Neural Netw. 7(4), 609–628 (1994)zbMATHCrossRefGoogle Scholar
  5. 5.
    Corradi, V., White, H.: Regularized neural networks: some convergence rate results. Neural Computation 7, 1225–1244 (1995)CrossRefGoogle Scholar
  6. 6.
    Kůrková, V., Sanguineti, M.: Learning with generalization capability by kernel methods of bounded complexity. J. Complex 21(3), 350–367 (2005)zbMATHCrossRefGoogle Scholar
  7. 7.
    Vidnerová, P., Neruda, R.: Testing error estimates for regularization and radial function networks. In: Sun, F., Zhang, J., Tan, Y., Cao, J., Yu, W. (eds.) ISNN 2008, Part I. LNCS, vol. 5263, pp. 549–554. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Prechelt, L.: PROBEN1 – a set of benchmarks and benchmarking rules for neural network training algorithms. Technical Report 21/94, Universitaet Karlsruhe (November 1994)Google Scholar
  9. 9.
    Yao, X.: Evolving artificial neural networks. Proceedings of the IEEE 9(87), 1423–1447 (1999)Google Scholar
  10. 10.
    Stanley, K.O., D’Ambrosio, D., Gauci, J.: A hypercube-based indirect encoding for evolving large-scale neural networks. Artificial Life 15(2) (2009)Google Scholar
  11. 11.
    Neruda, R., Kudová, P.: Learning methods for radial basis functions networks. Future Generation Computer Systems 21, 1131–1142 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Petra Vidnerová
    • 1
  • Roman Neruda
    • 1
  1. 1.Institute of Computer ScienceAcademy of Sciences of the Czech RepublicPrague 8Czech Republic

Personalised recommendations