Grammatical inference: An old and new paradigm

  • Yasubumi Sakakibara
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 997)


In this paper, we provide a survey of recent advances in the field “grammatical inference” with a particular emphasis on the results concerning the learnability of target classes represented by deterministic finite automata, context-free grammars, hidden Markov models, stochastic context-free grammars, simple recurrent neural networks, and casebased representations.


Regular Language Finite Automaton Inference Algorithm Tree Automaton Membership Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    N. Abe and H. Mamitsuka. A new method for predicting protein secondary structures based on stochastic tree grammars. In Proceedings of 11th International Conference on Machine Learning, 1994.Google Scholar
  2. 2.
    N. Abe and M. K. Warmuth. On the computational complexity of approximating distributions by probabilistic automata. Machine Learning, 9:205–260, 1992.Google Scholar
  3. 3.
    D. W. Aha, D. Kibler, and M. K. Albert. Instance-based learning algorithms. Machine Learning, 6:37–66, 1991.Google Scholar
  4. 4.
    A. V. Aho and J. D. Ullman. The Theory of Parsing, Translation and Compiling, Vol. I: Parsing. Prentice Hall, Englewood Cliffs, N.J., 1972.Google Scholar
  5. 5.
    D. Angluin. Finding patterns common to a set of strings. Journal of Computer and System Sciences, 21:46–62, 1980.CrossRefGoogle Scholar
  6. 6.
    D. Angluin. Inductive inference of formal languages from positive data. Information and Control, 45:117–135, 1980.CrossRefGoogle Scholar
  7. 7.
    D. Angluin. A note on the number of queries needed to identify regular languages. Information and Control, 51:76–87, 1981.CrossRefGoogle Scholar
  8. 8.
    D. Angluin. Inference of reversible languages. Journal of the ACM, 29:741–765, 1982.CrossRefGoogle Scholar
  9. 9.
    D. Angluin. Learning regular sets from queries and counter-examples. Information and Computation, 75:87–106, 1987.CrossRefGoogle Scholar
  10. 10.
    D. Angluin. Queries and concept learning. Machine Learning, 2:319–342, 1988.Google Scholar
  11. 11.
    D. Angluin. Negative results for equivalence queries. Machine Learning, 5:121–150, 1990.Google Scholar
  12. 12.
    D. Angluin. Computational learning theory: survey and selected bibliography. In Proceedings of 24th Annual ACM Symposium on Theory of Computing, pages 351–369. ACM Press, 1992.Google Scholar
  13. 13.
    D. Angluin and M. Kharitonov. When won't membership queries help? In Proceedings of 23rd Annual ACM Symposium on Theory of Computing, pages 444–454. ACM Press, 1991.Google Scholar
  14. 14.
    D. Angluin and C. H. Smith. Inductive inference: Theory and methods. ACM Computing Surveys, 15(3):237–269, 1983.CrossRefGoogle Scholar
  15. 15.
    S. Arikawa, T. Shinohara, and A. Yamamoto. Elementary formal systems as a unifying framework for language learning. In Proceedings of 2nd Workshop on Computational Learning Theory, pages 312–327. Morgan Kaufmann, 1989.Google Scholar
  16. 16.
    J. K. Baker. Trainable grammars for speech recognition. Speech Communication Papers for the 97th Meeting of the Acoustical Society of America, pages 547–550, 1979.Google Scholar
  17. 17.
    A. Brāzma and K. Cerāns. Efficient learning of regular expressions from good examples. In Proceedings of 4th International Workshop on Analogical and Inductive Inference (AII'94), Lecture Notes in Artificial Intelligence 872, pages 76–90. Springer-Verlag, 1994.Google Scholar
  18. 18.
    A. Burago. Learning structurally reversible context-free grammars from queries and counterexamples in polynomial time. In Proceedings of 7th Workshop on Computational Learning Theory (COLT'93), pages 140–146. ACM Press, 1994.Google Scholar
  19. 19.
    R. C. Carrasco and J. Oncina, editors. Proceedings of Second International Colloquium on Grammatical Inference (ICGI-94), Lecture Notes in Artificial Intelligence 862. Springer-Verlag, 1994.Google Scholar
  20. 20.
    S. Crespi-Reghizzi. An effective model for grammar inference. In B. Gilchrist, editor, Information Processing 71, pages 524–529. Elsevier North-Holland, 1972.Google Scholar
  21. 21.
    P. Dupon. Regular grammatical inference from positive and negative samples by genetic search: the GIG method. In Proceedings of Second International Colloquium on Grammatical Inference (ICGI-94), Lecture Notes in Artificial Intelligence 862, pages 236–245. Springer-Verlag, 1994.Google Scholar
  22. 22.
    J. L. Elman. Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7:195–225, 1991.Google Scholar
  23. 23.
    A. F. Fahmy and A. W. Biermann. Synthesis of real time acceptors. Journal of Symbolic Computation, 15:807–842, 1993.Google Scholar
  24. 24.
    C. L. Giles, G. Z. Sun, H. H. Chen, Y. C. Lee, and D. Chen. Higher order recurrent networks & grammatical inference. In Advances in Neural Information Processing Systems 2, pages 380–387. Morgan Kaufmann, 1990.Google Scholar
  25. 25.
    E. M. Gold. Language identification in the limit. Information and Control, 10:447–474, 1967.CrossRefGoogle Scholar
  26. 26.
    E. M. Gold. Complexity of automaton identification from given data. Information and Control, 37:302–320, 1978.CrossRefGoogle Scholar
  27. 27.
    M. Golea, M. Matsuoka, and Y. Sakakibara. Unsupervised learning of time-varying probability distributions using two-layer recurrent networks. Unpublished manuscript, 1995.Google Scholar
  28. 28.
    J. Hertz, A. Krogh, and R. G. Palmer. Introduction to the Theory of Neural Computation. Addison-Wesley, 1991.Google Scholar
  29. 29.
    L. Hunter. Artificial Intelligence and Molecular Biology. AAAI Press/MIT Press, 1993.Google Scholar
  30. 30.
    H. Ishizaka. Polynomial time learnability of simple deterministic languages. Machine Learning, 5:151–164, 1990.Google Scholar
  31. 31.
    K. P. Jantke and S. Lange. Case-based representation and learning of pattern languages. In Proceedings of 4th Workshop on Algorithmic Learning Theory (ALT'93), Lecture Notes in Artificial Intelligence 744, pages 87–100. Springer-Verlag, 1993.Google Scholar
  32. 32.
    T. Koshiba. Typed pattern languages and their learnability. In Proceedings of 2nd European Conference on Computational Learning Theory (EuroCOLT'95), Lecture Notes in Artificial Intelligence 904, pages 367–379. Springer-Verlag, 1995.Google Scholar
  33. 33.
    A. Krogh, M. Brown, I. S. Mian, K. Sjölander, and D. Haussler. Hidden Markov models in computational biology. Applications to protein modeling. Journal of Molecular Biology, 235:1501–1531, Feb. 1994.PubMedGoogle Scholar
  34. 34.
    P. D. Laird. A survey of computational learning theory. In R. B. Banerji, editor, Formal Techniques in Artificial Intelligence — A Sourcebook, pages 173–215. Elsevier Science Publishers, 1990.Google Scholar
  35. 35.
    K. Lari and S. J. Young. The estimation of stochastic context-free grammars using the inside-outside algorithm. Computer Speech and Language, 4:35–56, 1990.CrossRefGoogle Scholar
  36. 36.
    L. S. Levy and A. K. Joshi. Skeletal structural descriptions. Information and Control, 39:192–211, 1978.Google Scholar
  37. 37.
    E. Mäkinen. On the structural grammatical inference problem for some classes of context-free grammars. Information Processing Letters, 42:193–199, 1992.Google Scholar
  38. 38.
    L. Miclet. Grammatical inference. In H. Bunke and A. Sanfeliu, editors, Syntactic and Structural Pattern Recognition — Theory and Applications, pages 237–290. World Scientific, 1986.Google Scholar
  39. 39.
    S. Miyano, A. Shinohara, and T. Shinohara. Which classes of elementary formal systems are polynomial-time learnable? In Proceedings of 2nd Workshop on Algorithmic Learning Theory (ALT'91), pages 139–150. Japanese Society for Artificial Intelligence, Ohmsha, Ltd, 1991.Google Scholar
  40. 40.
    J. Oncina, P. Garcia, and E. Vidal. Learning subsequential transducers for pattern recognition interpretation tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15:448–458, 1993.CrossRefGoogle Scholar
  41. 41.
    F. Pereira and Y. Schabes. Inside-outside reestimation for partially bracketed corpora. In Proceedings of 30th Annual Meeting of the Association for Computational Linguistics, pages 128–135, 1992.Google Scholar
  42. 42.
    L. Pitt. Inductive inference, DFAs, and computational complexity. In Proceedings of AII-89 Workshop on Analogical and Inductive Inference (Lecture Notes in Computer Science, 397), pages 18–44. Springer-Verlag, 1989.Google Scholar
  43. 43.
    L. Pitt and M. K. Warmuth. The minimum consistent DFA problem cannot be approximated within any polynomial. In Proceedings of 21st Annual ACM Symposium on Theory of Computing. ACM Press, 1989.Google Scholar
  44. 44.
    L. R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE, 77(2):257–286, 1989.Google Scholar
  45. 45.
    R. L. Rivest. Learning decision lists. Machine Learning, 2:229–246, 1987.Google Scholar
  46. 46.
    D. Ron and R. Rubinfeld. Learning fallible deterministic finite automata. Machine Learning, 18:149–185, 1995.Google Scholar
  47. 47.
    Y. Sakakibara. Learning context-free grammars from structural data in polynomial time. Theoretical Computer Science, 76:223–242, 1990.Google Scholar
  48. 48.
    Y. Sakakibara. On learning from queries and counterexamples in the presence of noise. Information Processing Letters, 37:279–284, 1991.Google Scholar
  49. 49.
    Y. Sakakibara. Efficient learning of context-free grammars from positive structural examples. Information and Computation, 97:23–60, 1992.Google Scholar
  50. 50.
    Y. Sakakibara, M. Brown, R. Hughey, I. S. Mian, K. Sjolander, R. C. Underwood, and D. Haussler. Stochastic context-free grammars for tRNA modeling. Nucleic Acids Research, 22:5112–5120, 1994.PubMedGoogle Scholar
  51. 51.
    Y. Sakakibara and M. Golea. Simple recurrent networks as generalized hidden markov models with distributed representations. Unpublished manuscript, 1995.Google Scholar
  52. 52.
    Y. Sakakibara, K. P. Jantke, and S. Lange. Learning languages by collecting cases and tuning parameters. In Proceedings of 5th International Workshop on Algorithmic Learning Theory (ALT'94), Lecture Notes in Artificial Intelligence 872, pages 532–546. Springer-Verlag, 1994.Google Scholar
  53. 53.
    Y. Sakakibara and R. Siromoney. A noise model on learning sets of strings. In Proceedings of 5th Workshop on Computational Learning Theory (COLT'92), pages 295–302. ACM Press, 1992.Google Scholar
  54. 54.
    D. B. Searls. The linguistics of DNA. American Scientist, 80:579–591, Nov.–Dec. 1992.Google Scholar
  55. 55.
    T. Shinohara. Inductive inference from positive data is powerful. In Proceedings of 3rd Workshop on Computational Learning Theory, pages 97–110. Morgan Kaufmann, 1990.Google Scholar
  56. 56.
    A. Stolcke and S. Omohundro. Inducing probabilistic grammars by bayesian model merging. In Proceedings of Second International Colloquium on Grammatical Inference (ICGI-94), Lecture Notes in Artificial Intelligence 862, pages 106–118. Springer-Verlag, 1994.Google Scholar
  57. 57.
    Y. Takada. Grammatical inference for even linear languages based on control sets. Information Processing Letters, 28:193–199, 1988.CrossRefGoogle Scholar
  58. 58.
    Y. Takada. A hierarchy of language families learnable by regular language learners. In Proceedings of Second International Colloquium on Grammatical Inference (ICGI-94), Lecture Notes in Artificial Intelligence 862, pages 16–24. Springer-Verlag, 1994.Google Scholar
  59. 59.
    L. G. Valiant. A theory of the learnable. Communications of the ACM, 27:1134–1142, 1984.CrossRefGoogle Scholar
  60. 60.
    T. Yokomori. Polynomial-time learning of very simple grammars from positive data. In Proceedings of 4th Workshop on Computational Learning Theory (COLT'91), pages 213–227. Morgan Kaufmann, 1991.Google Scholar
  61. 61.
    T. Yokomori. Learning nondeterministic finite automata from queries and counterexamples. In Furukawa, Michie, and Muggleton, editors, Machine Intelligence 13, pages 169–189. Oxford Univ. Press, 1994.Google Scholar
  62. 62.
    T. Yokomori. On polynomial-time learnability in the limit of strictly deterministic automata. To appear in Machine Learning, 1995.Google Scholar
  63. 63.
    T. Zeugmann and S. Lange. A guided tour across the boundaries of learning recursive languages. GOSLER-Report 26, TH Leipzig, FB Mathematik und Informatik, 1994.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Yasubumi Sakakibara
    • 1
  1. 1.Fujitsu Laboratories Ltd.Institute for Social Information ScienceShizuokaJapan

Personalised recommendations