Abstract
The historically second elementary grammar formalism was published in 1936 by the American logician Emil Post (1897–1957). Known as rewrite systems or Post production systems, they originated in the mathematical context of recursion theory and are closely related to automata theory and computational complexity theory.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Apparently, Chomsky forgot to mention Post’s contribution. This was commented on by Bar-Hillel (1964) as follows:
This approach [i.e., rewriting systems] is the standard one for the combinatorial systems conceived much earlier by Post [1936], as a result of his penetrating researches into the structure of formal calculi, though Chomsky seems to have become aware of the proximity of his ideas to those of Post only at a later stage of his work.
Bar-Hillel (1964), p. 103
This is remarkable insofar as Chomsky’s thesis advisor Harris and Bar-Hillel had been in close scientific contact since 1947. Moreover, Bar-Hillel and Chomsky discussed “linguistics, logic, and methodology in endless talks” beginning in 1951 (Bar-Hillel 1964 , p. 16).
- 2.
\(\mathrm{V}^{\mathsf{+}}\) is the positive closure and \(\mathrm{V}^{\mathsf{*}}\) is the Kleene closure of V (Sect. 7.2).
- 3.
Context-free grammar sometimes uses so-called ‘epsilon rules’ of the form A→ε, where ε is the empty string. However, epsilon rules can always be eliminated (Hopcroft and Ullman 1979, p. 90, Theorem 4.3). We specify the right-hand side of type 2 rules as a nonempty sequence in order to formally maintain the context-free rules as a special form of the context-sensitive rules (subset relation).
- 4.
This is the definition of right linear PS Grammars. PS Grammars in which the order of the terminal and the nonterminal symbols on the right-hand side of the rule is inverted are called left linear. Left and right linear grammars are equivalent (Hopcroft and Ullman 1979 , p. 219, Theorem 9.2).
- 5.
For example, the generative capacity of the PS Grammar 7.1.3 for the artificial language a k b k is higher than that of a regular PS Grammar 8.3.2 for the free monoid over {a,b} (7.1.2). The free monoid contains all the expressions of a k b k, but its regular PS Grammar is unable to exclude the expressions which do not belong to the language a k b k.
- 6.
Earley (1970) characterizes a primitive operation as “in some sense the most complex operation performed by the algorithm whose complexity is independent of the size of the grammar and the input string.” The exact nature of the primitive operation varies from one grammar formalism to the next.
For example, Earley chose the operation of adding a state to a state set as the primitive operation of his famous algorithm for context-free grammars (Sect. 9.3). In LA Grammar, on the other hand, the subclass of C LAGs uses a rule application as its primitive operation (Sect. 11.4).
- 7.
We are referring here to time complexity.
- 8.
As explained in Sect. 15.3, the Limas corpus was built in analogy to the Brown and the LOB corpora, and contains 500 texts of 2000 running word forms each. The texts were selected at random from roughly the same 15 genres as those of the Brown and LOB corpora in order to come as close as possible to the desideratum of a balanced corpus which is representative of the German language in the year 1973.
- 9.
These data were provided by Schulze at CLUE (Computational Linguistics at the University of Erlangen Nürnberg).
- 10.
The class of regular languages is not part of the hierarchy of LA Grammar, though it may be reconstructed in it (CoL, Theorem 3, p. 138). Instead the LA Grammar hierarchy provides the alternative linear class of C1 languages. As shown in Sects. 11.5 ff., the class of C1 languages contains all regular languages, all deterministic context-free languages which are recognized by an epsilon-free DPDA, as well as many context-sensitive languages.
- 11.
Hopcroft and Ullman (1979) , pp. 55f.
- 12.
- 13.
Each context-free language is homomorphic with the intersection of a regular set and a semi-Dyck set (Chomsky-Schützenberger Theorem). See Harrison (1978) , pp. 317f.
- 14.
The superscript R in WW R stands mnemonically for reverse.
- 15.
The class of context-free languages is not part of the hierarchy of LA Grammar, though it may be reconstructed in it (CoL, Theorem 4, pp. 138). See also Sect. 12.3, footnote 10. Instead the LA Grammar hierarchy provides the alternative polynomial class of C2-languages. As shown in Sect. 12.4, the class of C2-languages contains most, though not all, context-free languages, as well as many context-sensitive languages.
- 16.
Hopcroft and Ullman (1979) , pp. 125f.
- 17.
One of the earliest and simplest examples of diagonalization is the Ackermann Function (Ackermann 1928).
- 18.
Hopcroft and Ullman (1979), p. 228, Theorem 9.8.
- 19.
In the hierarchy of LA Grammar, in contrast, the class of recursive languages is formally defined as the class of A-languages, generated by unrestricted LA Grammars (11.2.2). Furthermore, the class of context-sensitive languages is formally defined as the class of B languages, generated by bounded LA Grammars.
- 20.
Hopcroft and Ullman (1979) , p. 175, 7.4.
- 21.
The class of recursively enumerable languages is not part of the LA Grammar hierarchy, though it may be reconstructed in it (footnote 10 at the end of Sect. 11.2).
- 22.
Even if phrase structures of the form A–B–C…A are excluded, the number of different phrase structure trees still grows exponentially with the length of the input. From a formal as well as an empirical point of view, however, structures like A–B–C…A are legitimate in phrase structure grammar.
- 23.
CoL, p. 24.
- 24.
Bar-Hillel wrote in 1960 (Bar-Hillel 1964, p. 102) that he had abandoned his 1953 work on C Grammar because of analogous difficulties with the discontinuous elements in sentences like He gave it up.
- 25.
In active consultation with Chomsky (personal communication by Bob Ritchie, Stanford, 1983).
- 26.
Cf. Barton et al. (1987) . If context-free rule loops like A→B→…→A are forbidden, the complexity of LFG is exponential. However, because such loops are formally legal within context-free PS Grammar, this restriction is not really legitimate from the viewpoint of complexity theory. Besides, even an exponential complexity is much too high for computational applications.
References
Ackermann, W. (1928) “Zum Hilbertschen Aufbau der reellen Zahlen,” Mathematische Annalen 99:118–133
Bar-Hillel, Y. (1964) Language and Information. Selected Essays on Their Theory and Application, Reading: Addison-Wesley
Barton, G., R.C. Berwick, and E.S. Ristad (1987) Computational Complexity and Natural Language, Cambridge: MIT Press
Bloomfield, L. (1933) Language, New York: Holt, Rinehart, and Winston
Chomsky, N. (1957) Syntactic Structures, The Hague: Mouton
Chomsky, N. (1965) Aspects of the Theory of Syntax, Cambridge: MIT Press
Earley, J. (1970) “An Efficient Context-Free Parsing Algorithm,” Communications of the ACM 2:94, reprinted in B. Grosz, K. Sparck Jones, and B.L. Webber (eds.), 1986
Garey, M.R., and D.S. Johnson (1979) Computers and Intractability: A Guide to the Theory of NP-Completeness, San Francisco: W.H. Freeman
Harris, Z. (1951) Method in Structural Linguistics, Chicago: University of Chicago Press
Harrison, M. (1978) Introduction to Formal Language Theory, Reading: Addison-Wesley
Hopcroft, J.E., and J.D. Ullman (1979) Introduction to Automata Theory, Languages, and Computation, Reading: Addison-Wesley
Peters, S., and R. Ritchie (1973) “On the Generative Power of Transformational Grammar,” Information and Control 18:483–501
Author information
Authors and Affiliations
Exercises
Exercises
Section 8.1
-
1.
State an algebraic definition of PS Grammar.
-
2.
What is the difference between terminal and nonterminal symbols in PS Grammar?
-
3.
By whom and when was PS Grammar invented, under what name, and for what purpose?
-
4.
By whom and when was PS Grammar first used for describing natural language?
-
5.
Describe the standard restrictions on the rules of PS Grammar.
-
6.
Explain the term generative capacity.
Section 8.2
-
1.
Explain the relation between special kinds of PS Grammar, formal language classes, and different degrees of complexity.
-
2.
Name the main classes of complexity. Why are they independent of specific formalisms of generative grammar?
-
3.
What is the complexity of language classes in the PS hierarchy?
-
4.
What is the average sentence length in the Limas corpus?
-
5.
What is the maximal sentence length in the Limas corpus?
-
6.
How much time would an exponential algorithm require in the worst case to parse the Limas corpus?
-
7.
Explain the PS Grammar hierarchy of formal languages.
-
8.
Which language classes in the PS Grammar hierarchy are of a complexity still practical for computational linguistics?
Section 8.3
-
1.
Define a PS Grammar which generates the free monoid over {a, b, c}. Classify this language, called \(\mathsf{\{a, b, c\}}^{\mathsf{+}}\), within the PS Grammar hierarchy. Compare the generative capacity of the grammar for \(\mathsf{\{a ,b, c\}}^{\mathsf{+}}\) with that for a k b k c k. Which is higher and why?
-
2.
Where does the term context-free come from in PS Grammar?
-
3.
What kinds of structures can be generated by context-free PS Grammars?
-
4.
Name two artificial languages which are not context-free. Explain why they exceed the generative power of context-free PS Grammars.
-
5.
Define a PS Grammar for a k b 2k. Explain why this language fulfills the context-free schema pairwise inverse.
-
6.
Define a PS Grammar for ca m dyb n. What are examples of well-formed expressions of this artificial language? Explain why it is a regular language.
-
7.
Why would 8.3.6 violate the definition of context-sensitive PS Grammar rules if β were zero?
-
8.
What is a pumping lemma?
-
9.
Why is there no pumping lemma for the context-sensitive languages?
-
10.
Are the recursively enumerable languages recursive?
-
11.
Name a recursive language which is not context-sensitive.
Section 8.4
-
1.
State the definition of constituent structure.
-
2.
Explain the relation between context-free PS Grammars and phrase structure trees.
-
3.
Describe how the notion of constituent structure developed historically.
-
4.
Name two kinds of distribution tests and explain their role for finding correct constituent structures.
-
5.
Why was it important to American structuralists to segment sentences correctly?
Section 8.5
-
1.
Describe the notion of a discontinuous element in natural language and explain why discontinuous elements cause the constituent structure paradox.
-
2.
How does transformational grammar try to solve the problem caused by discontinuous elements?
-
3.
Compare the goal of transformational grammar with the goal of computational linguistics.
-
4.
What is the generative capacity of transformational grammar?
-
5.
Explain the structure of a Bach-Peters sentence in relation to the recoverability of deletions condition. Which mathematical property was shown with this sentence? Is it a property of natural language or of transformational grammar?
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Hausser, R. (2014). Language Hierarchies and Complexity. In: Foundations of Computational Linguistics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41431-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-41431-2_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41430-5
Online ISBN: 978-3-642-41431-2
eBook Packages: Computer ScienceComputer Science (R0)