An Incremental Approach to Language Acquisition: Thematic Role Assignment with Echo State Networks
In previous research a model for thematic role assignment (θRARes) was proposed, using the Reservoir Computing paradigm. This language comprehension model consisted of a recurrent neural network (RNN) with fixed random connections which models distributed processing in the prefrontal cortex, and an output layer which models the striatum. In contrast to this previous batch learning method, in this paper we explored a more biological learning mechanism. A new version of the model (i-θRARes) was developed that permitted incremental learning, at each time step. Learning was based on a stochastic gradient descent method. We report here results showing that this incremental version was successfully able to learn a corpus of complex grammatical constructions, reinforcing the neurocognitive plausibility of the model from a language acquisition perspective.
Keywordsreservoir computing recurrent neural network language acquisition incremental learning anytime processing grammar acquisition
Unable to display preview. Download preview PDF.
- 2.Dominey, P.F., Voegtlin, T.: Learning word meaning and grammatical constructions from narrated video events. In: Proc HLT-NAACL (2003)Google Scholar
- 8.Hinaut, X., Dominey, P.F.: Real-Time Parallel Processing of Grammatical Structure in the Fronto-Striatal System: A Recurrent Network Simulation Study Using Reservoir Computing. PloS ONE 8(2), e52946 (2013)Google Scholar
- 10.Hoerzer, G.M., Legenstein, R., Maass, W.: Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. Cereb Cortex, Advance online publication (November 11, 2012) (retrieved)Google Scholar