Abstract
Connectionist learning models have had considerable empirical success, but it is hard to characterize exactly what they learn. The learning of finite-state languages (FSL) from example strings is a domain which has been extensively studied and might provide an opportunity to help understand connectionist learning. A major problem is that traditional FSL learning assumes the storage of all examples and thus violates connectionist principles. This paper presents a provably correct algorithm for inferring any minimum-state deterministic finite-state automata (FSA) from a complete ordered sample using limited total storage and without storing example strings. The algorithm is an iterative strategy that uses at each stage a current encoding of the data considered so far, and one single sample string. One of the crucial advantages of our algorithm is that the total amount of space used in the course of learning for encoding any finite prefix of the sample is polynomial in the size of the inferred minimum state deterministic FSA. The algorithm is also relatively efficient in time and has been implemented. More importantly, there is a connectionist version of the algorithm that preserves these properties. The connectionist version requires much more structure than the usual models and has been implemented using the Rochester Connectionist Simulator. We also show that no machine with finite working storage can iteratively identify the FSL from arbitrary presentations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Angluin, D. (1987). Learning regular sets from queries and counterexamples. Information and Computation, 75, 87–106.
Angluin, D. (1981). A note on the number of queries needed to identify regular languages.Information and Control, 51, 76–87.
Angluin, D. (1978). On the complexity of minimum inference of regular sets. Information and Control, 39, 337–350.
Angluin, D. (1976). An application of the theory of computational complexity to the study of inductive inference. Ph.D. dissertation, Department of Electrical Engineering & Computer Science, Univ. California, Berkeley.
Angluin, D., & Smith, C.H. (1983). Inductive inference: Theory and methods. Computing Surveys, 15, 237–269.
Biermann, A.W., &Feldman, J.A. (1978). On the synthesis of finite-state machines from samples of their behavior. IEEE Trans. on Computers, C-21, 592–597
Brooks, R.A. (1987). Intelligence without representation. Proceedings of the Conf. on Foundations of AI. Cambridge, MA: MIT.
Feldman, J.A., & Ballard, D.H. (1982). Connectionist models and their properties.Cognitive Science, 6, 205–254.
Gold, E.M. (1978). Complexity of automaton identification from given data. Information and Control, 37, 302–320.
Gold, E.M. (1972). System identification via state characterization. Automatica, 8, 621–636.
Gold, E.M. (1967). Language identification in the limit. Information and Control, 10, 447–474.
Hinton, G.E. (1987). Connectionist learning procedures (TR CMU-CS-87-115). Pittsburgh, PA: Carnegie Mellon University, Computer Science Department.
Hopcroft, J.E., & Ullman, J.D. (1979). Introduction to automata and formal languages. Reading, MA: Addison-Wesley.
Horning, J.K. (1969). A study of grammatical inference. Ph.D. thesis, Stanford University.
Ibarra, O.H., & Jiang, T. (1988). Learning regular languages from counterexamples.Proceedings of the 1988 Workshop on Computational Learning Theory (pp. 371–385). Boston, MA.
Jantke, K.P., &Beick, H-R. (1981). Combining postulates of naturalness in inductive inference. Journal of Information Processing and Cybernetics, 17, 465–484.
Kearns, M., Li, J., Pitt, L., & Valiant, L. (1987). On the learnability of boolean formulae. Proceedings of the 9th Annual ACM Symp. on Theory of Computing (pp. 284–295). New York, NY.
Natarajan, B.K. (1987). On learning boolean functions. Proceedings of the 9th Annual ACM Symp. on Theory of Computing (pp. 296–394). New York, NY.
Osherson, D.N., Stob, M., & Weinstein, S. (1986). Systems that learn: An introduction to learning theory for conginitive and computer scientists. Cambridge, MA: MIT Press.
Rivest, R.L., & Schapire, R.E. (1987). A new approach to unsupervised learning in deterministic environments. Proceedings of the 4th International Workshop on Machine Learning. Irvine, CA.
Rivest, R.L., & Schapire, R.E. (1987). Diversity-based inference of finite automata. Proceedings of the 28th Annual Symp. on Foundations of Computer Science. Los Angeles, CA.
Rumelhart, D.E., & McClelland, J.L. (Eds.). (1986). Parallel distributed processing, explorations in the microstructure of cognition. Cambridge, MA: Bradford Books/MIT Press.
Sharon, M. (1990). Learning automata. M.Sc. Thesis in Computer Science, Technion, Haifa, Israel, (in Hebrew).
Trakhtenbrot, B.A., & Barzdin, Ya.M. (1973). Finite automata. Amsterdam: North-Holland.
Valiant, L.G. (1985). Learning disjunctions of conjunctions.Proceedings of the 9th IJCAI (pp. 560–566). Los Angeles, CA.
Valiant, L.G. (1984). A theory of the learnable.CACM, 27, 1134–1142.
Waltz, D., & Feldman, J.A. (Eds.). (1987). Connectionist models and their implications. Ablex Publishing Corp.
Weisberg, Y. (1990).Iterative learning finite automata—Application by neural net. M.Sc. thesis in Electrical Engineering, Technion, Haifa, Israel, (in Hebrew).
Wiehagen, R. (1976). Limeserkennung rekursiver funktionen durch spezielle Strategien. Elektronische Informationsverarbeitung und Kybernetik, 12, 93–99.
Williams, R.J. (1987). Reinforcement-learning connectionist systems (TR NU-CCS-87-3).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1991 Springer Science+Business Media New York
About this chapter
Cite this chapter
Porat, S., Feldman, J.A. (1991). Learning Automata from Ordered Examples. In: Touretzky, D. (eds) Connectionist Approaches to Language Learning. The Springer International Series in Engineering and Computer Science, vol 154. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-4008-3_2
Download citation
DOI: https://doi.org/10.1007/978-1-4615-4008-3_2
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4613-6792-5
Online ISBN: 978-1-4615-4008-3
eBook Packages: Springer Book Archive