Abstract
Grammatical inference is known as one of the most attractive paradigms of scientific learning that is nowadays a classical but still novel discipline. The problem of grammatical inference is roughly to infer (discover) a grammar that generates a given set of sample sentences in some manner that is supposed to be realized by some algorithmic device, usually called inference algorithm. Therefore, grammatical inference can be taken as one of the typical formulations for a broader word “learning”, and provides a good theoretical framework for investigating a learning process. The goal of this chapter is to present standard but important results in the area of grammatical inference as learning, including not only theoretical fruits but also interesting applications of learning algorithms.
The author is deeply indebted to Satoshi Kobaashi for his valuable comments on the draft of this article.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abe, N., Mamitsuka, H.: Predicting protein secondary structures based on stochastic tree grammars. Machine Learning, 29 (1997), 275–301.
Aho, A.V., Ullman, J.D.: The Theory of Parsing, Translation and Compiling, Vol. I: Parsing (Prentice Hall, 1972).
Angluin, D.: On the complexity of minimum inference of regular sets. Information and Control, 39 (1978), 337–350.
Angluin, D.: Finding common patterns to a set of strings. Journal of Computer and System Science, 21 (1980), 46–62.
Angluin, D.: Inductive inference of formal languages from positive data. Information and Control, 45 (1980), 117–135.
Angluin, D.: A note on the number of queries needed to identify regular languages. Information and Control, 51 (1982), 76–87.
Angluin, D.: Inference of reversible languages. Journal of the Association for Computing Machinery, 29 (1982), 741–765.
Angluin, D.: Learning regular sets from queries and counter-examples. Information and Computation, 75 (1987), 87–106.
Angluin, D.: Learning k-bounded context-free grammars. Research Report YALEU/DCS/RR-557, Department of Computer Science, Yale University, 1987.
Angluin, D.: Learning k-term DNF Formulas using Queries and Counterexamples. Tech. Rep. YALEU/DCS/RR-559“, Department of Computer Science, Yale University, 1987.
Angluin, D.: Queries and concept learning. Machine Learning,2 (1988), 319342.
Angluin, D.: Negative results for equivalence queries. Machine Learning, 5 (1990), 121–150.
Angluin, D., Laird, P.: Learning from noisy examples. Machine Learning, 2 (1988), 343–370.
Angluin, D., Kharitonov, M.: When won’t membership queries help ?. Journal of Computer and System Sciences, 50 (1995), 336–355.
Arikawa, S., Shinohara, T., Yamamoto, A.: Learning elementary formal system. Theoretical Computer Science, 95 (1992), 97–113.
Arimura, H., Shinohara, T., Otsuki, S.: Finding minimal generalizations for unions of pattern languages and its application to inductive inference from positive data. In Proc. of 11th Annual Symposium on Theoretical Aspects of Computer Science, LNCS 775 (1994), pp 649–660.
Baker, J. K.: Trainable grammars for speech recognition. Speech Communication Papers for the 97th Meeting of the Acoustical Society of America, (1979), pp 547–550.
Baum, E.B., Haussler, D.: What size net gives valid generalization?. Neural Computation, 1 (1989), 151–160.
Biermann, A.W.: An interactive finite-state language learner. In Proc. of 1st USA-JAPAN Computer Conference, (1972), pp 13–20.
Biermann, A., Feldman, J.: On the synthesis of finite-state machines from samples of their behavior. IEEE Trans. Comput., C-21 (1972), 592–597.
Berman, P., Roos, R.: Learning one-counter languages in polynomial time. In Proc. of 28th IEEE Symp. on FOCS, (1987), pp 61–67.
Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K.: Occam’s razor. Information Processing Letters, 24 (1987), 377–380.
Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K.: Learnability and the Vapnik-Chervonenkis dimension. Journal of the Association for Computing Machinery, 36 (1989), 929–965.
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines (Cambridge University Press, Cambridge, United Kingdom, 2000 ).
Durbin, R., Eddy, S., Krogh, A., Mitchison, G.: Biological sequence analysis (Cambridge University Press, 1998 ).
Ehrenfeucht, A., Haussier, D.: Learning decision trees from random examples. Information and Computation, 82 (1989), 231–246.
Ehrenfeucht, A., Haussier, D., Kearns, M., Valiant, L. G.: A general lower bound on the number of examples needed for learning. Information and Computation, 82 (1989), 247–261.
Erlebach, T., Rossmanith, P., Stadtherr, H., Steger, A., Zeugmann, T.: Learning one-variable pattern languages very efficiently on average, in parallel, and by asking queries. Theoretical Computer Science, 261 (2001), 119–156.
Freund, Y.: Boosting a weak learning algorithm by majority. Information and Computation, 121 (1995), 256–285.
Freund, Y., Schapire, R. E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55 (1997), 119–139.
Garcia, P., Vidal, E., Oncina, J.: Learning locally testable languages in the strict sense. In Proc. of 1st Int. Workshop on Algorithmic Learning Theory, (1980), pp 325–338.
Garcia, P., Cano.A., Ruiz, J.: A comparative study of two algorithms for automata identification. In Proc. 5th International Colloquium on Grammatical Inference (ICGI 2000), Lecture Notes in Artificial Intelligence, 1891 (Springer, 2000 ), 115–126.
Gold, E. M.: Language identification in the limit. Information and Control, 10 (1967), 447–474.
Gold, E. M.: Complexity of automaton identification from given data. Information and Control, 37 (1978), 302–320.
Goldman, S.A., Kwek, S.S.: On learning unions of pattern languages and tree patterns. In Proc. of 10th Int. Conference on Algorithmic Learning Theory, LNAI 1720 (1999), pp 347–363.
Haussier, D.: Decision theoretic generalizations of the PAC model for neural net and other learning applications. Information and Computation, 100 (1992), 78–150.
Head, T.: Formal language theory and DNA: An analysis of the generative capacity of specific recombinant behaviors. Bulletin of Mathematical Biology, 49 (1987), 737–759.
Helmbold, D., Warmuth, M. K.: On weak learning. Journal of Computer and System Sciences, 50 (1995), 551–573.
Hoperoft, J., Ullman, J.: Introduction to Automata Theory, Languages and Computation (Addison-Wesley, Readings, Mass., U.S.A., 1979 ).
Ibarra, O.H., Jiang, T.: Learning regular languages from counterexamples. In Proc. of 1st Workshop on Computational Learning Theory, (1988), pp 371–385.
Ishizaka, H.: Inductive inference of regular languages based on model inference. International Journal of Computer Mathematics, 27 (1989), 67–83.
Ishizaka, H.: Polynomial time learnability of simple deterministic languages. Machine Learning, 5 (1990), 151–164.
Jain, S., Osherson, D., Royer, J.S., Sharma, A.: Systems That Learn (MIT Press, 1999 ).
Jiang T., Salomaa, A. Salomaa, K., Yu, S.: Inclusion is undecidable for pattern languages. In Proc. of International Colloquium on Automata, Languages and Programming, 20nd ICALP ‘83, LNCS 700 (1993), pp 301–312.
Kearns, M.: The Computational Complexity of Machine Learning (MIT Press, 1990).
Kearns, M.: Efficient noise-tolerant learning from statistical queries. In Proc. of 25th Annual ACM Symposium on Theory of Computing, (ACM Press, 1993 ), pp 392–401.
Kearns, M., Li, M.: Learning in the presence of malicious errors. SIAM Journal on Computing, 22 (1993), 807–837.
Kearns, M., Li, M., Pitt, L., Valiant, L. G.: On the learnability of Boolean formulae. In Proc. of 19th Annual ACM Symposium on Theory of Computing, (ACM Press, 1987 ), pp 285–294.
Kearns, M., Li, M., Pitt, L., Valiant, L. G.: Weakly learning DNF and characterizing statistical query learning using Fourier analysis. In Proc. of 26th Annual ACM Symposium on Theory of Computing (ACM Press, 1994 ), pp 253–262.
Kearns, M., Pitt, L.: A polynomial-time algorithm for learning k-variable pattern languages from examples. In Proc. of 2nd Annu. Workshop on Computational Learning Theory (Morgan Kaufmann, 1989 ), pp 57–71.
Kearns, M., Schapire, R. E.: Efficient Distribution-free learning of probabilistic concepts. In Proc. of 31st IEEE Symposium on Foundations of Computer Science (IEEE Computer Society Press, 1990 ), pp 382–391.
Kearns, M., Valiant, L.G.: Cryptographic limitations on learning Boolean formulae and finite automata. Journal of the Association for Computing Machinery, 41 (1994), 67–95.
Kearns, M., Vazirani, U.: An Introduction to Computational Learning Theory (MIT Press, Cambridge, Massachusetts, 1994 ).
Knobe, B., Knobe, K.: A Method for inferring context-free grammars. Information and Control, 31 (1976), 129–146.
Kobayashi, S., Yokomori, T.: On approximately identifying concept classes in the limit. In Proc. of 6th Int. Workshop on Algorithmic Learning Theory, LNAI 997 (1995), pp 298–312.
Kobayashi, S., Yokomori, T.: Learning approximately regular languages with reversible languages. Theoretical Computer Science, 174 (1997), 251–257.
Kobayashi, S.: Iterated transductions and efficient learning from positive data: A unifying view. In Proc. of 5th International Colloquium on Grammatical Inference, LNAI 1891 (2000), 157–170.
Laird, P.: Learning from Good and Bad Data (Kluwer Academic, 1988).
Lari, K., Young, S. J.: The estimation of stochastic context-free grammars using the inside-outside algorithm. Computer Speech and Language, 4 (1990), 35–56.
Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and Computation, 108 (1994), 212–261.
Mäkinen, E.: The grammatical inference problem for the Szilard languages of linear grammars Information Processing Letters, 36 (1990), 203–206.
Matsumoto, S., Shinohara, A.: Learning pattern languages using queries. In Proc. of Computational Learning Theory: EuroColt ‘87, LNAI 1208 (1997), pp 185–197.
Mitchell, A. R.: Learnability of a subclass of extended pattern languages. In Proc. of 11th Annu. Conference on Computational Learning Theory (ACM Press, 1998 ), pp 64–71.
Miyano, S. et al.: Knowledge acquisition from amino acid sequences by decision trees and indexing. In Proc. of Genome Informatics Workshop III, (1992), pp 69–72.
Moore, E.F.: Gedanken-experiments on sequential machines. In C.E. Shannon, J. McCarthy (Eds.) Automata Studies (Princeton Univ. Press, Princeton, N.J., 1956 ), pp 129–153.
Motoki, T., Shinohara, T., Wright, K.: The correct definition of finite elasticity: corrigendum to Identification of unions. In Proc. of 4th Annu. Workshop on Computational Learning Theory (Morgan Kaufmann, 1991 ), pp 375.
Mukouchi, Y.: Induction inference of an approximate concept from positive data. In Proc. of 5th Int. Workshop on Algorithmic Learning Theory, LNAI 872 (1994), pp 484–499.
Mukouchi, Y., Arikawa, S.: Towards a mathematical theory of machine discovery from facts. Theoretical Computer Science, 137 (1995), 53–84.
Natarajan, B. K.: Machine Learning: A Theoretical Approach (Morgan Kaufmann, 1991 ).
Nessel, O., Lange, S.: Learning erasing pattern languages with queries. In Proc. of 11 th Int. Conference on Algorithmic Learning Theory. LNAI 1968 (2000), pp 86–100.
Oncina, J., Garcia, P.: Inferring regular languages in polynomial update time. In Pattern Recognition and Image Analysis, Series in Machine Perception and Artificial Intelligence, 1 (World Scientific, Singapore, 1992 ), pp 49–61.
Pitt, L.: Inductive inference, DFAs, and computational complexity. In K.P.Jantke (Ed.), Proc. of Intern. Workshop on Analogical and Inductive Inference, LNCS 397 (1989), pp 18–44.
Pitt, L., Valiant, L. G.: Computational limitations on learning from examples. Journal of the Association for Computing Machinery, 35 (1988), 965–984.
Pitt, L., Warmuth, M. K.: Prediction preserving reducibility. Journal of Computer and System Sciences, 41 (1990), 430–467.
Pitt, L., Warmuth, M. K.: The minimum consistent DFA problem cannot be approximated within any polynomial. Journal of the Association for Computing Machinery, 40 (1993), 95–142.
Quinlan, J. R.: Induction of decision trees. Machine Learning, 1 (1986), 81–106.
Quinlan, J. R.: C4.5: Programs for machine learning ( Morgan Kaufmann, San Mateo, CA, 1993 ).
Rivest, R. L.: Learning decision lists. Machine Learning, 2 (1987), 229–246.
Ron, D., Rubinfeld, R.: Learning fallible deterministic finite automata. Machine Learning, 18 (1995), 149–185.
Sakakibara, Y.: Learning context-free grammars from structural data in polynomial time. Theoretical Compututer Science, 76 (1990), 223–242.
Sakakibara, Y.: Inductive inference of logic programs based on algebraic semantics. New Generation Computing, 7 (1990), 365–380.
Sakakibara, Y.: On learning Smullyan’s elementary formal systems: Towards an efficient learning method for context-sensitive languages. Advances in Software Science and Technology, 2 (1990), 79–101.
Sakakibara, Y.: Efficient learning of context-free grammars from positive structural examples. Information and Computation, 97 (1992), 23–60.
Sakakibara, Y: Noise-tolerant Occam algorithms and their applications to learning decision trees. Machine Learning, 11 (1993), 37–62.
Sakakibara, Y: Recent advances of grammatical inference, Theoretical Computer Science, 185 (1997), 15–45.
Sakakibara, Y., Brown, M. P., Hughey, R., Mian, I. S., Sjölander, K., Underwood, R., Haussier, D.: Stochastic context-free grammars for tRNA modeling. Nucleic Acids Research, 22 (1994), 5112–5120.
Sakakibara, Y., Kondo, M.: GA-based learning of context-free grammars using tabular representations. Proc. of 16th International Conference on Machine Learning (ICML-99), Bled, Slovenia, (1999), pp 354–360.
Sakakibara, Y., Misue, K., Koshiba, T.: A machine learning approach to knowledge acquisitions from text databases. International Journal of Human Computer Interaction, 8 (1996), 309–324.
Sakakibara, Y., Siromoney, R.: A noise model on learning sets of strings. In Proc. of 5th Ann. Workshop on Computational Learning Theory, (ACM Press, 1992 ), pp 295–302.
Schapire, R. E.: The strength of weak learnability. Machine Learning, 5 (1990), 197–227.
Schapire, R. E.: The Design and Analysis of Efficient Learning Algorithms (MIT Press, 1992 ).
Shapiro, E.Y.: Inductive inference of theories from facts. Research Report YALEU/DCS/RR-192, Department of Computer Science, Yale University, 1981.
Shapiro, E.Y.: Algorithmic Program Debugging (MIT Press, 1983).
Shinohara, T.: Polynomial time inference of pattern languages and its application. In Proc. of 7th IBM Symp. Math. Found. of Comput. Sci., (1982), pp 191–209.
Shinohara, T.: Polynomial time inference of extended regular pattern languages. In Proc. of RIMS Symposium on Software Science and Engineering, LNCS 147 (1982), pp 115–127.
Shinohara, T.: Inferring unions of two pattern languages. Bull. Inform. Cybern., 22 (1986), 9–18.
Shinohara, T.: Rich classes inferable from positive data: Length-bounded elementary formal systems. Information and Control, 108 (1994), 175–186.
Shinohara, T.: Inductive inference of monotonic formal systems from positive data. New Generation Computing, 8 (1991), 371–384.
Shinohara, T., Arimura, H.: Inductive inference of unbounded unions of pattern languages from positive data. Theoretical Computer Science,241 (2000), 191209.
Shirakawa, H. Yokomori, T.: Polynomial-time MAT learning of C-deterministic context-free languages. Transactions of Information Processing Society of Japan,34 (1993), 380–390.
Tajima, Y., Tornita, E.: A polynomial time learning algorithm of simple deterministic languages via memebership queries and a representative sample. Proc. of 5th International Colloqium on Grammatical Inference (ICGI2000), LNAI 1891 (2000), pp 284–297.
Takada, Y.: Inferring parenthesis linear grammars based on control sets. Journal of Information Processing, 12 (1988), 27–33.
Takada, Y.: Grammatical inference for even linear languages based on control sets. Information Processing Letters, 28 4 (1988), 193–199.
Takada, Y.: Learning equal matrix grammars and mutlitape automata with structural information. In Proc. of 1st Int. Workshop on Algorithmic Learning Theory, (1990), pp 302–313.
Takada, Y.: Interactive systhesis of process flowcharts. Research Report IIASRR-92–4E, IIAS-SIS, Fujitsu Laboratories Ltd., March, 1992.
Tanida, N., Yokomori, T.: Polynomical-time identification of strictly regular languages in the limit. IEICE Transactions on Information and Systems, E75–D (1992), 125–132.
Uemura, Y., Hasegawa, A., Kobayashi, S., Yokomori, T.: Tree adjoining grammars for RNA structures prediction. Theoretical Computer Science, 210 (1999), 277–303.
Valiant, L.G.: A theory of the learnable. Communications of the Association for Computing Machinery, 27 (1984), 1134–1142.
Yamanishi, K.: A learning criterion for stochastic rules. Machine Learning, 9 (1992), 165–203.
Watanabe, O.: A formal study of learning via queries. In Proc. of ICALP’90, LNCS 443 (1990), pp 139–152.
Wright, K.: Identification of unions of languages drawn from an identifiable class. In Proc. of 2nd Annu. Workshop on Computational Learning Theory (Morgan Kaufmann, 1989 ), pp 328–333.
Yokomori, T.: Inductive inference of contex-free languages based on context-free expressions. International Journal of Computer Mathematics, 24 (1988), 115–140.
Yokomori, T.: Learning Context-free Languages Efficiently, In K.P.Jantke (Ed.) Proc. of Intern. Workshop on Analogical and Inductive Inference, LNCS 397 (1989), pp 104–123.
Yokomori, T.: Learning non-deterministic finite automata from queries and counterexamples. Machine Intelligence 13. Machine Intelligence and Inductive Learning ( Oxford Univ. Press, 1994 ) pp 169–189.
Yokomori, T.: Inductive inference of OL languages. In G. Rozenberg, A. Salomaa (Eds.) Lindenmayer Systems (Springer-Verlag, 1992 ), pp 115–132.
Yokomori, T.: On polynomial-time learnability in the limit of strictly deterministic automata. Machine Learning, 19 (1995), 153–179.
Yokomori, T.: Learning two-tape automata from queries and counterexamples. Mathematical Systems Theorey, 29 (1996), 259–270. Corrigendum. Theory of Computing Systems, 35, 465–466, 2002.
Yokomori, T.: Polynomial-time identification of very simple grammars from positive data. To appear in Theoretical Computer Science.
Yokomori, T., Kobayashi, S.: Learning local languages and their application to DNA sequence analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence, 20 (1998), 1067–1079.
Zeugmann, T., Lange, S.: A guided tour across the boundaries of learning recursive languages. In Algorithmic Learning for Knowledge-Based Systems. LNAI 961 (1995), pp 190–258.
Home of Computational Learning Theory: http://www.learningtheory.org/
COLTBIB HOMEPAGE: http://www.i.kyushu-u.ac.jp/thomas/COLTBIB/ coltbib. jhtml
Homepage of the Grammatical Inference Community: http://www.cs.iastate.edu/-honavar/gi/gi.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Yokomori, T. (2004). Grammatical Inference and Learning. In: Martín-Vide, C., Mitrana, V., Păun, G. (eds) Formal Languages and Applications. Studies in Fuzziness and Soft Computing, vol 148. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39886-8_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-39886-8_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-53554-3
Online ISBN: 978-3-540-39886-8
eBook Packages: Springer Book Archive