Skip to main content
SpringerLink
Log in

Neural Networks: Architectures, Learning and Performance

  • Conference paper
Information Systems and Data Analysis

Summary

We review recent work on some principles of a theory of neural networks. Contributions about network architectures, performance criteria, learning objectives and learning strategies from statistical mechanics, computer sciences, statistics and approximation theory are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • AMARI, S., FUJITA, N., and SHINOMOTO, S. (1992): Four Types of Learning Curves. Neural Computation, 4, 605–618.

    Article  Google Scholar 

  • ABU-MOSTAFA, Y.S. (1989): The Vapnik-Chervonenkis Dimension: Information versus Complexity in Learning. Neural Computation, 1, 312–317.

    Article  Google Scholar 

  • BAUM, E.B., and HAUSSLER, D. (1988): What Size Net Gives Valid Generalization? Neural Computation, 1, 151–160.

    Article  Google Scholar 

  • BAXT, W.G. (1992): Improving the Accuracy of an Artificial Neural Network Using Multiple Differently Trained Networks. Neural Computation, 4, 772–780

    Article  Google Scholar 

  • BARLOW, H.B. (1989): Unsupervised Learning Neural Computation, 1, 295–311

    Article  Google Scholar 

  • BILBRO, G.L., and VAN DEN BOUT, D.E. (1992): Maximum Entropy and Learning Theory. Neural Computation, 4, 839–853.

    Article  Google Scholar 

  • BLUM, A., and RIVEST, R.L. (1988): Training a 3-node neural network is NP-complete InProc. Workshop on Computational Learning Theory, 9–18.

    Google Scholar 

  • CARNEVALI, P., and PATARNELLO, S. (1987): Exhaustive Thermodynamical Analysis of Boolean Learning Networks. Europhys. Lett. 4, 1199–1204.

    Article  Google Scholar 

  • CYBENKO, G. (1988): Continuous Valued Neural Networks with Two Hidden Layers are Sufficient. Technical Report, Department of Computer Science, Tufts Univ., Medford, MA.

    Google Scholar 

  • DERRIDA, B., GARDNER, E., and ZIPPELIUS, A. (1987): An Exactly Solvable Model of an Asymmetric Neural Network Europhys. Lett., 4, 167.

    Article  Google Scholar 

  • DOMANY, E., VAN HEMMEN, J.L., SCHULTEN, K. (ed.) (1992): Models of Neural Networks, Springer, Berlin.

    Google Scholar 

  • DURBIN, R., and WIILSHAW, D. (1987): An Analogue Approach to the Travelling Salesman Problem Using an Elastic Net Method. Nature, 326, 689–691.

    Article  Google Scholar 

  • GYÖRGYI, G. (1990): Inference of a Rule by a Neural Network with Thermal Noise. Phys. Rev. Lett, 64, 2951–2960.

    Article  Google Scholar 

  • HERTZ, J., KROGH, A., and PALMER, R.G. (1991):Introduction to the Theory of Neural Computation. Addison-Wesley, Redwood City.

    Google Scholar 

  • HINTON, G.E., and SEJNOWSKI, T.J. (1986): Learning and Relearning in Boltzmann machines. In: RUMELHART, MCCLELLAND (ed.) Parallel Distributed Processing., Vol.1, Ch. 7.

    Google Scholar 

  • HOPFIELD, J.J. (1982): Neural Networks and Physical Systems with Emergent Collective Computational Properties. Proc. Nat. Acad. Sc. USA, 81, 3088–3092

    Article  Google Scholar 

  • HORNER, H. (1992):Dynamics of Learning in a Binary Perceptron. Zeitschrift f. Phys. B, 86, 291–308

    Article  Google Scholar 

  • HORNIK, K., STINCHCOMB, M., and WHITE, H. (1989): Multilayer Feedforward Networks are Universal Approximators. Neural Networks, 2, 359–368

    Article  Google Scholar 

  • JACOBS, R.A., JORDAN, M.I., NOWLAN, S.J., and HINTON, G.E. (1991): Adaptive Mixtures of Local Experts Neural Computation, 3, 79–87.

    Article  Google Scholar 

  • JUDD, S. (1988): On the Complexity of Loading Shallow Neural Networks. Journal of Complexity, 4.

    Google Scholar 

  • KOHONEN, T. (1989): Self-Organization and Associative Memory. Springer, Berlin.

    Google Scholar 

  • KREE, R. and ZIPPELIUS, A. (1987): Continuous-Time Dynamics of Asymmetrically Diluted Networks. Phys. Rev. A, 36, 4421–4427

    Article  Google Scholar 

  • KREE, R., and ZIPPELIUS, A. (1988): Recognition of Topological Features of Graphs and Images in Neural Networks. Journ. Phys. A, 21, L813-L818

    Article  Google Scholar 

  • KREE, R., and MÜLLER, A. (1991): Classification Properties of Communicating Neural Networks. In: Proceedings of the 16th Annual Meeting of the GfKl, Dortmund.

    Google Scholar 

  • KREE, R., and MÜLLER, A. (1993): Multi-Agent Neural Network Models In: Proc.III. European Congress of Psychology, Tampere, Finland

    Google Scholar 

  • KULLBACK, S., and LEIBLER, R.A. (1951): On Information and Sufficiency. Ann. Math. Stat, 22, 79–86

    Article  Google Scholar 

  • LEVIN, E., TISHBY, N., and SOLLA, S. (1990): A Statistical Approach to Learning and Generalization in Layered Neural Networks. Proc. IEEE, 78, 1568–1574.

    Article  Google Scholar 

  • LINSKER, R. (1988): Self-Organization in a Perceptual Network. Computer, 105–117

    Google Scholar 

  • LIPPMANN, R.P.(1989): Review of Neural Networks for Speech Recognition. Neural Computation, 1, 1–38.

    Article  Google Scholar 

  • LITTLE, W.A. (1974): The Existence of Persistent States in the Brain. Math. Biosc., 19, 101–120.

    Article  Google Scholar 

  • MARCHAND, M., GOLEA, M., and RUJAN, P. (1990): A Convergence Theorem for Sequential Learning in Two-Layer Perceptrons. Europhys. Lett, .11, 487–492.

    Article  Google Scholar 

  • MCCULLOCH, W.S., PITTS, W. (1943): A Logical Calculus of Ideas Immanent in Nervous Activity. Bull. Math. Biophys., 5, 115–133.

    Article  Google Scholar 

  • MEZARD, M., PARISI, G., and VIRASORO, M.A. (ed.) (1986): Spin Glass Theory and Beyond, World Scientific, Singapure

    Google Scholar 

  • MINSKY, M.L., and PAPERT, S.A. (1969): Perceptrons. MIT Press, Cambridge, MA.

    Google Scholar 

  • NERRAND, O., ROUSSEL-RAGOT, P., PERSONNAZ, L., DREYFUS, G., and MARCOS, S. (1993): Neural Networks and Non-Linear Adaptive Filtering:Unifying Concepts and New Algorithms, preprint

    Google Scholar 

  • OPPER, M., and HAUSSLER, D. (1991): Generalization Performance of Bayes Optimal Classification Algorithm for Learning a Perceptron. Phys. Rev. Lett, 66, 2677–2680

    Article  Google Scholar 

  • PARISI, G. (1992): On the Classification of Learning Machines. Network, 3, 259–265.

    Article  Google Scholar 

  • PETERSON, C. (1990): Parallel Distributed Approaches to Combinatorial Optimization: Benchmark Studies on Travelling Salesman Problem. Neural Computation, 3, 261–269.

    Article  Google Scholar 

  • RITTER, H., MARTINETZ, T., and SCHULTEN, K (1991): Neural Computation and Self-Organizing Maps. Addison-Wesley, Reading, MA.

    Google Scholar 

  • ROSENBLATT, F. (1962): Principles of Neurodynamics. Spartan, New York

    Google Scholar 

  • SEUNG, H.S., OPPER, M., and SOMPOLINSKY, H. (1992). In: Proc. Annual ACM Workshop on Computation and Learning Theory, 287–294.

    Chapter  Google Scholar 

  • SEUNG, H.S., SOMPOLINSKY, H., and TISHBY, N. (1992): Statistical Mechanics of Learning from Examples. Phys. Rev. A, 45, 6065–6091

    Article  Google Scholar 

  • SHERRINGTON, D. (1993): Neural Networks: The Spin Glass Approach, preprint

    Google Scholar 

  • TANK, D.W., and HOPFIELD, J.J. (1986): Simple Neural Optimization Networks IEEE Trans. Circuits and Systems, 33, 533–541.

    Article  Google Scholar 

  • TISHBY, N., LEVIN, E., and SOLLA, S. (1989): Consistent Inference of Probabilities in Layered Networks. IJCNN, II, 403–410

    Google Scholar 

  • TSIRUKIS, G., REKLAITIS, V., and TENORIO, M.F. (1989): Nonlinear Optimization Using Generalized Hopfield Networks. Neural Computation, 1, 511–521.

    Article  Google Scholar 

  • VALIANT, L.G. (1984): A Theory of the Learnable. Commun. ACM., 27, 1134–1142.

    Article  Google Scholar 

  • VAPNIK, V.N., and Chervonenkis, A. (1971): On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities. Theory Prob. Appl., 16, 264–280.

    Article  Google Scholar 

  • WATKIN, T.L.H., (1992): Optimal Learning with a Neural Network preprint

    Google Scholar 

  • WATKIN, T.L.H., RAU, A., and BIEHL, M. (1992): The Statistical Mechanics of Learning a Rule (to be published in Rev. Mod. Phys.)

    Google Scholar 

  • WHITE, H. (1989): Learning in Artificial Neural Networks: A Statistical Perspective. Neural Computation, 1, 425–464.

    Article  Google Scholar 

  • WONG, K.Y.M., SHERRINGTON, D. (1988): Storage Properties of Randomly Connected Boolean Neural Networks for Associative Memory. Europhys. Lett, 7, 197–201.

    Article  Google Scholar 

  • WULFF, N.H. (1992): Learning Dynamics with Recurrent Networks. NORDITA preprint

    Google Scholar 

  • ZAK, M. (1988): Terminal Attractors for Addressable Memory in Neural Networks. Phys. Lett 133A, 18–22.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut für Theoretische Physik, Bunsenstrasse 9, W-3400, Göttingen, Germany

    Reiner Kree

Authors
  1. Reiner Kree
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut für Statistik, Rheinisch-Westfälische Technische Hochschule Aachen (RWTH), Wüllnerstr. 3, D-52056, Aachen, Germany

    Hans-Hermann Bock

  2. Forschungsstelle “Mathematische Logik” der Heidelberger Akademie der Wissenschaften Fachbereich Informatik, Universität Kaiserslautern, Erwin-Schrödinger-Str. 57, D-67653, Kaiserslautern, Germany

    Wolfgang Lenski

  3. Fachbereich Informatik, Universität Kaiserslautern, Erwin-Schrödinger-Str. 57, D-67653, Kaiserslautern, Germany

    Michael M. Richter

  4. Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI), Erwin-Schrödinger-Str. 57, D-67653, Kaiserslautern, Germany

    Michael M. Richter

Rights and permissions

Reprints and permissions

Copyright information

© 1994 Springer-Verlag Berlin · Heidelberg

About this paper

Cite this paper

Kree, R. (1994). Neural Networks: Architectures, Learning and Performance. In: Bock, HH., Lenski, W., Richter, M.M. (eds) Information Systems and Data Analysis. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-46808-7_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-46808-7_14

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-58057-7

  • Online ISBN: 978-3-642-46808-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation