Skip to main content
SpringerLink
Log in

Product Unit Neural Networks with Constant Depth and Superlinear VC Dimension

  • Conference paper
  • First Online:
Book cover Artificial Neural Networks — ICANN 2001 (ICANN 2001)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2130))

Included in the following conference series:

  • 3131 Accesses

Abstract

It has remained an open question whether there exist product unit networks with constant depth that have superlinear VC dimension. In this paper we give an answer by constructing two-hidden-layer networks with this property. We further show that the pseudo dimension of a single product unit is linear. These results bear witness to the cooperative effects on the computational capabilities of product unit networks as they are used in practice.

Work supported by the ESPRIT Working Group in Neural and Computational Learning II, NeuroCOLT2, No. 27150. A longer (9-page) version of this paper is available from http://www.ruhr-uni-bochum.de/lmi/mschmitt/. Complete proofs also appear in [9]

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 189.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. M. Anthony and P. L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge, 1999.

    MATH  Google Scholar 

  2. R. Durbin and D. Rumelhart. Product units: A computationally powerful and biologically plausible extension to backpropagation networks. Neural Computation, 1:133–142, 1989.

    Article  Google Scholar 

  3. A. Ismail and A. P. Engelbrecht. Global optimization algorithms for training product unit neural networks. In International Joint Conference on Neural Networks IJCNN’2000, vol. I, pp. 132–137, IEEE Computer Society, Los Alamitos, CA, 2000.

    Google Scholar 

  4. P. Koiran and E. D. Sontag. Neural networks with quadratic VC dimension. Journal of Computer and System Sciences, 54:190–198, 1997.

    Article  MATH  MathSciNet  Google Scholar 

  5. L. R. Leerink, C. L. Giles, B. G. Horne, and M. A. Jabri. Learning with product units. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems 7, pp. 537–544, MIT Press, Cambridge, MA, 1995.

    Google Scholar 

  6. W. Maass. Neural nets with super-linear VC-dimension. Neural Computation, 6:877–884, 1994.

    Article  MATH  Google Scholar 

  7. W. Maass and G. Turán. Lower bound methods and separation results for on-line learning models. Machine Learning, 9:107–145, 1992.

    MATH  Google Scholar 

  8. A. Sakurai. Tighter bounds of the VC-dimension of three layer networks. In Proceedings of the World Congress on Neural Networks, vol. 3, pp. 540–543. Erlbaum, Hillsdale, NJ, 1993.

    Google Scholar 

  9. M. Schmitt. On the complexity of computing and learning with multiplicative neural networks. Neural Computation, to appear.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Lehrstuhl Mathematik und Informatik, Fakultät für Mathematik, Ruhr-Universität Bochum, D-44780, Bochum, Germany

    Michael Schmitt

Authors
  1. Michael Schmitt
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dept. of Mecidal Cybernetics and Artificial Intelligence, University of Vienna, Freyung 6/2, 1010, Vienna, Austria

    Georg Dorffner

  2. Institute for Computer Aided Automation Pattern Recognition and Image Processing Group, Technical University of Vienna, Favoritenstr. 9/1832, 1040, Vienna, Austria

    Horst Bischof

  3. Institut für Statistik, Wirtschaftsuniversität Wien, Augasse 2-6, 1090, Wien, Austria

    Kurt Hornik

Rights and permissions

Reprints and permissions

Copyright information

© 2001 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Schmitt, M. (2001). Product Unit Neural Networks with Constant Depth and Superlinear VC Dimension. In: Dorffner, G., Bischof, H., Hornik, K. (eds) Artificial Neural Networks — ICANN 2001. ICANN 2001. Lecture Notes in Computer Science, vol 2130. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44668-0_36

Download citation

  • DOI: https://doi.org/10.1007/3-540-44668-0_36

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-42486-4

  • Online ISBN: 978-3-540-44668-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation