Learning a deterministic finite automaton with a recurrent neural network
- 140 Downloads
We consider the problem of learning a finite automaton with recurrent neural networks from positive evidence. We train an Elman recurrent neural network with a set of sentences in a language and extract a finite automaton by clustering the states of the trained network. We observe that the generalizations beyond the training set, in the language recognized by the extracted automaton, are due to the training regime: the network performs a “loose” minimization of the prefix DFA of the training set, the automaton that has a state for each prefix of the sentences in the set.
KeywordsNetwork State Recurrent Neural Network Hide Unit Finite Automaton Input Unit
Unable to display preview. Download preview PDF.
- 1.M. Casey. The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Neural Computation, 8:1135–1178, 1996.Google Scholar
- 2.A. Cleeremans, D. Servan-Schreiber, and J.L. McClelland. Finite state automata and simple recurrent networks. Neural Computation, 1:372–381, 1989.Google Scholar
- 3.P. R. Cohen. Empirical Methods for Artificial Intelligence. The MIT Press, 1995.Google Scholar
- 5.J. L. Elman. Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 1992.Google Scholar
- 6.C. L. Giles, C. B. Miller, D. Chen, G. Z. Sun, H. H. Chen, and Y. C. Lee. Extracting and learning an unknown grammar with recurrent neural networks. In Advances in Neural Information Processing Systems 4. 1992.Google Scholar
- 8.John F. Kolen. Fool's gold: Extracting finite state machines from recurrent network dynamics. In Advances in Neural Information Processing Systems 6, 1994.Google Scholar
- 9.E. Makinen. Inferring regular languages by merging nonterminals. Technical Report A-1997-6, Department of Computer Science, University of Tampere, 1997.Google Scholar
- 12.D. E. Rumelhart, R. Durbin, R. Golden, and Y. Chauvin. Backpropagation: The basic theory. In Backpropagation: Theory, architectures, and applications. Erlbaum, 1993.Google Scholar
- 13.H. T Siegelmann. Theoretical Foundations of Recurrent Neural Networks. PhD thesis, Rutgers, 1992.Google Scholar
- 14.P. N. Werbos. The roots of backpropagation. John Wiley & Sons, Inc., 1994.Google Scholar