Abstract
Previous work in the literature has shown that, using a local representation of the alphabet, simple recurrent neural networks were able to estimate the probability distribution corresponding to strings which belong to a stochastic regular language. This paper carries on with the empirical works in the matter by including input time delays in simple recurrent networks. This technique could sometimes avoid the use of fully-recurrent architectures (with high computational requirements) to learn certain grammars. Therefore, we could avoid the problems of memory that arise using networks with simple recurrences.
Partially supported by the Spanish Fundación Bancaja, project P1A99-10.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
M. A. Castaño, F. Casacuberta, E. Vidal. Simulation of Stochastic Regular Grammars through Simple Recurrent Networks. In “New Trends in Neural Computation”. Lecture Notes in Computer Science, vol. 686, pp. 210–215. Eds. J. Mira, J. Cabestany, A. Prieto, Springer-Verlag. 1993.
M. A. Castaño, Redes Neuronales Recurrentes para Inferencia Gramatical y Traducción Automaática. Ph.D. dissertation. Univesidad Politécnica de Valencia. 1998.
J. L. Elman. Finding Structure in Time. Cognitive Science 2, no. 4, pp. 279–311. 1990
4. D.E. Rumelhart, G. Hinton, R. Williams. Learning Sequential Structure in SImple Recurrent Networks. In “Parallel distributed processing: Experiments in the microstructure of cognition”, vol. 1. Eds. D.E. Rumelhart, J.L. McClelland and the PDP Research Group, MIT Press. 1986.
A. W Smith, D. Zipser. Learning Sequential Structure with the Real-Time Recurrent Learning Algorithm. International Journal of Neural Systems, vol. 1, no. 2, pp. 125–131. 1991.
D. Servan-Schreiber, A. Cleeremans, J.L. McClelland. Graded State Machines: The Representation of Temporal Contingencies in Simple Recurrent Networks. Machine Learning, no. 7, pp. 161–193. 1991.
R.J. Williams, D. Zipser. Experimental Analysis of the Real-Time Recurrent Learning Algorithm. Connection Science, vol. 1, no. 1, pp. 87—111. 1989.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Casañ, G.A., Asunción Castaño, M. (2001). Inference of Stochastic Regular Languages through Simple Recurrent Networks with Time Delays. In: Mira, J., Prieto, A. (eds) Bio-Inspired Applications of Connectionism. IWANN 2001. Lecture Notes in Computer Science, vol 2085. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45723-2_81
Download citation
DOI: https://doi.org/10.1007/3-540-45723-2_81
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42237-2
Online ISBN: 978-3-540-45723-7
eBook Packages: Springer Book Archive