A Hierarchical Classification of First-Order Recurrent Neural Networks

  • Jérémie Cabessa
  • Alessandro E. P. Villa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6031)


We provide a refined hierarchical classification of first-order recurrent neural networks made up of McCulloch and Pitts cells. The classification is achieved by first proving the equivalence between the expressive powers of such neural networks and Muller automata, and then translating the Wadge classification theory from the automata-theoretic to the neural network context. The obtained hierarchical classification of neural networks consists of a decidable pre-well ordering of width 2 and height ω ω , and a decidability procedure of this hierarchy is provided. Notably, this classification is shown to be intimately related to the attractive properties of the networks, and hence provides a new refined measurement of the computational power of these networks in terms of their attractive behaviours.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Duparc, J.: A hierarchy of deterministic context-free ω-languages. Theor. Comput. Sci. 290(3), 1253–1300 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Finkel, O.: An effective extension of the Wagner hierarchy to blind counter automata. In: Fribourg, L. (ed.) CSL 2001 and EACSL 2001. LNCS, vol. 2142, pp. 369–383. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  3. 3.
    Hopfield, J.J., Feinstein, D.I., Palmer, R.G.: ‘unlearning’ has a stabilizing effect in collective memories. Nature 304, 158–159 (1983)CrossRefGoogle Scholar
  4. 4.
    Kaneko, K., Tsuda, I.: Chaotic itinerancy. Chaos 13(3), 926–936 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Kleene, S.C.: Representation of events in nerve nets and finite automata. In: Automata Studies. Annals of Mathematics Studies, vol. 34, pp. 3–42. Princeton University Press, Princeton (1956)Google Scholar
  6. 6.
    Kremer, S.C.: On the computational power of elman-style recurrent networks. IEEE Transactions on Neural Networks 6(4), 1000–1004 (1995)CrossRefGoogle Scholar
  7. 7.
    McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysic 5, 115–133 (1943)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Minsky, M.L.: Computation: finite and infinite machines. Prentice-Hall, Inc., Upper Saddle River (1967)zbMATHGoogle Scholar
  9. 9.
    Selivanov, V.: Wadge degrees of ω-languages of deterministic Turing machines. Theor. Inform. Appl. 37(1), 67–83 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Siegelmann, H.T.: Computation beyond the Turing limit. Science 268(5210), 545–548 (1995)CrossRefGoogle Scholar
  11. 11.
    Siegelmann, H.T.: Neural and super-Turing computing. Minds Mach. 13(1), 103–114 (2003)zbMATHCrossRefGoogle Scholar
  12. 12.
    Siegelmann, H.T., Sontag, E.D.: Turing computability with neural nets. Applied Mathematics Letters 4(6), 77–80 (1991)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Siegelmann, H.T., Sontag, E.D.: Analog computation via neural networks. Theor. Comput. Sci. 131(2), 331–360 (1994)zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Siegelmann, H.T., Sontag, E.D.: On the computational power of neural nets. J. Comput. Syst. Sci. 50(1), 132–150 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Sperduti, A.: On the computational power of recurrent neural networks for structures. Neural Netw. 10(3), 395–400 (1997)CrossRefGoogle Scholar
  16. 16.
    Tsuda, I.: Chaotic itinerancy as a dynamical basis of hermeneutics of brain and mind. World Futures 32, 167–184 (1991)CrossRefGoogle Scholar
  17. 17.
    Tsuda, I., Koerner, E., Shimizu, H.: Memory dynamics in asynchronous neural networks. Prog. Th. Phys. 78(1), 51–71 (1987)CrossRefGoogle Scholar
  18. 18.
    Wadge, W.W.: Reducibility and determinateness on the Baire space. PhD thesis, University of California, Berkeley (1983)Google Scholar
  19. 19.
    Wagner, K.: On ω-regular sets. Inform. and Control 43(2), 123–177 (1979)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Jérémie Cabessa
    • 1
  • Alessandro E. P. Villa
    • 1
    • 2
  1. 1.GIN Inserm UMRS 836University Joseph FourierGrenoble
  2. 2.Faculty of Business and EconomicsUniversity of LausanneLausanne

Personalised recommendations