Abstract
Estimating the parameters of stochastic context-free grammars (SCFGs) from data is an important, well-studied problem. Almost without exception, existing approaches make repeated passes over the training data. The memory requirements of such algorithms are ill-suited for embedded agents exposed to large amounts of training data over long periods of time. We present a novel algorithm, called HOLA, for estimating the parameters of SCFGs that computes summary statistics for each string as it is observed and then discards the string. The memory used by HOLA is bounded by the size of the grammar, not by the amount of training data. Empirical results show that HOLA performs as well as the Inside-Outside algorithm on a variety of standard problems, despite the fact that it has access to much less information.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sakakibara, Y., Brown, M., Highey, R., Mian, I.S., Sjolander, K., Haussler, D.: Stochastic context-free grammars for tRNA modeling. Nucleic Acids Research 22 (1994) 5112–5120
Jurafsky, D., Wooters, C., Segal, J., Stolcke, A., Fosler, E., Tajchman, G., Morgan, N.: Using a stochastic context-free grammar as a language model for speech recognition. In: Proceedings of ICASSP. (1995) 189–192
Schabes, Y., Roth, M., Osborne, R.: Parsing the Wall Street Journal with the inside-outside algorithm. In: Proceedings of the 6th Conference of the European Chapter of the Association for Computational Linguistics. (1993) 341–346
Hopcroft, J.E., Ullman, J.D.: Introductin to Automata Theory, Languages, and Computation. Addison Wesley (1979)
Lari, K., Young, S.J.: The estimation of stochastic context-free grammars using the inside-outiside algorithm. Computer Speech and Language 4 (1990) 35–56
Lari, K., Young, S.J.: Applications of stochastic context-free grammars using the inside-outside algorithm. Computer Speech and Language 5 (1991) 237–257
Dempster, N.M., Laird, A.P., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B 39 (1977) 185–197
Neal, R.M., Hinton, G.E.: A view of the EM algorithm that justifies incremental, sparse, and other variants. In Jordan, M.I., ed.: Learning in Graphical Models, Kluwer Academic (1998)
Boyen, X., Koller, D.: Approximate learning of dynamic models. In: Neural Information Processing Systems. (1998)
Cook C.M., Rosenfeld, A., Aronson, A.: Grammatical inference by hill climbing. Informational Sciences 10 (1976) 59–80
Stolcke, A., Omohundro, S.: Inducing probabilistic grammars by bayesian model merging. In Carrasco, R.C., Oncina, J., eds.: Grammatical Inference and Applications, Berlin, Heidelberg, Springer (1994) 106–118
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Oates, T., Heeringa, B. (2002). Estimating Grammar Parameters Using Bounded Memory. In: Adriaans, P., Fernau, H., van Zaanen, M. (eds) Grammatical Inference: Algorithms and Applications. ICGI 2002. Lecture Notes in Computer Science(), vol 2484. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45790-9_15
Download citation
DOI: https://doi.org/10.1007/3-540-45790-9_15
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44239-4
Online ISBN: 978-3-540-45790-9
eBook Packages: Springer Book Archive