Abstract
Temporal synchrony of activation spikes has been proposed as the representational code by which the brain segments perceptual patterns into multiple visual objects or multiple auditory sources. In this chapter we look at the implications of this neuroscientific proposal for learning and computation in artificial neural networks. Previous work has defined an artificial neural network model which uses temporal syn- chrony to represent and learn about multiple entities (Simple Synchrony Networks). These networks can store arbitrary amounts of information in their internal state by segmenting their representation of state into arbitrarily many entities. They can also generalize what they learn to larger internal states by learning generalizations about individual enti- ties. These claims are empirical demonstrated through results on training a Simple Synchrony Network to do syntactic parsing of real natural lan- guage sentences.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., Reitboeck, H.J.: Coherent oscillations: a mechanism of feature linking in the visual cortex? Biological Cybernetics 60 (1988) 121–130
Elman, J.L.: Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning 7 (1991) 195–225
Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: A critical analysis. Cognition 28 (1988) 3–71
Gray, C.M., Koenig, P., Engel, A.K., Singer, W.: Oscillatory responses in cat visual cortex exhibit intercolumnar synchronization which reflects global stimulus properties. Nature 338 (1989) 334–337
Hadley, R.F.: Systematicity in connectionist language learning. Mind and Language 9(3) (1994) 247–272
Henderson, J.: A connectionist architecture with inherent systematicity. In Proceedings of the Eighteenth Conference of the Cognitive Science Society, La Jolla, CA (1996) 574–579
Henderson, J.: Constituency, context, and connectionism in syntactic parsing. In Matthew Crocker, Martin Pickering, and Charles Clifton, editors, Architectures and Mechanisms for Language Processing, Cambridge University Press, Cambridge UK (2000) 189–209
Henderson, J.: A neural network parser that handles sparse data. In Proceedings of the 6th International Workshop on Parsing Technologies, Trento, Italy (2000) 123–134
Henderson, J., Lane, P.: A connectionist architecture for learning to parse. In Proceedings of COLING-ACL, Montreal, Quebec, Canada (1998) 531–537
Ho, E.K.S., Chan, L.W.: How to design a connectionist holistic parser. Neural Computation 11(8) (1999) 1995–2016
Lane, P., Henderson, J.: Incremental syntactic parsing of natural language corpora with simple synchrony networks. IEEE Transactions on Knowledge and Data Engineering (in press)
Lane, P, Henderson, J.: Simple synchrony networks: Learning to parse natural language with temporal synchrony variable binding. In Proceedings of the International Conference on Artificial Neural Networks, Skovde, Sweden (1998) 615–620
Shastri, L., Wendelken, C.: Seeking coherent explanations-a fusion of structured connectionism, temporal synchrony, and evidential reasoning. In Proceedings of Cognitive Science 2000, Philadelphia, PA (2000)
Shastri, L.: Advances in shruti-a neurally motivated model of relational knowledge representation and rapid inference using temporal synchrony. Applied Intelligence 11 (1999) 79–108
Shastri, L., Ajjanagadde, V.: From simple associations to systematic reasoning: A connectionist representation of rules, variables, and dynamic bindings using temporal synchrony. Behavioral and Brain Sciences 16 (1993) 417–451
Singer, W., Gray, C.M.: Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience 18 (1995) 555–586
von der Malsburg, C.: The correlation theory of brain function. Technical Report 81-2, Max-Planck-Institute for Biophysical Chemistry, Gottingen (1981)
von der Malsburg, C., Buhmann, J.: Sensory segmentation with coupled neural oscillators. Biological Cybernetics 67 (1992) 233–242
Wang, D.L.: Primitive auditory segregation based on oscillatory correlation. Cognitive Science 20 (1996) 409–456
Wang, D.L., Terman, D.: Image segmentation based on oscillatory correlation. Neural Computation 9 (1997) 805–836 (For errata see Neural Computation 9 (1997) (1623-1626)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Henderson, J. (2001). Segmenting State into Entities and Its Implication for Learning. In: Wermter, S., Austin, J., Willshaw, D. (eds) Emergent Neural Computational Architectures Based on Neuroscience. Lecture Notes in Computer Science(), vol 2036. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44597-8_17
Download citation
DOI: https://doi.org/10.1007/3-540-44597-8_17
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42363-8
Online ISBN: 978-3-540-44597-5
eBook Packages: Springer Book Archive