Adjective-Noun Combinations and the Generative Lexicon

Abstract

The present paper reports two experimental studies on cognitive processing of adjective-noun combinations in which lexical sematic representations and processes are modeled according to the Generative Lexicon theory (Pustejovsky J. The generative lexicon. MIT Press, Cambridge, MA, 1995). The focus of these studies is on investigating the effects of the factors adjectival formal type, and compatibility of concepts in combinations on computational complexity and on the content of semantic interpretation of adjective-noun combinations. Three types of adjective-noun combinations were distinguished namely, intersective (e.g., yellow car), subsective compatible (e.g., interesting car), and subsective incompatible (e.g., fast car). In Experiment 1, the hypothesis is tested that semantic interpretation of the three types of combinations varies in terms of computational complexity with intersective combinations being the simplest and the two subsective types being progressively more complex. The analysis of the results obtained in Experiment 1 showed a significant effect of the factor adjectival formal type in predicted direction. Intersective combinations were processed significantly faster (M_(I) = 794 ms) than the two subsective types (M_(SC) = 851 ms, and M_(SI) = 855 ms, respectively). In Experiment 2, written paraphrases for the three types of combinations were compared. The analysis of the results showed that the proportion of responses congruent with the combination type was highest for the intersective and the subsective incompatible combinations (approximately 75%), and lowest for the subsective compatible ones (39%). The low congruence in this category of combinations is possibly due to a high level of adjectival semantic underspecification. Generally, the findings obtained in the two experiments support the proposed model of semantic interpretation of adjective-noun combinations in which generative, type-driven meaning computation processes are emphasized.

Appendices

Appendices

8.1.1 Appendix A

8.1.1.1 A.1 Materials Used in Experiments 1 and 2

Table 8.1 List of Test Combinations Used in Experiments 1 and 2

8.1.1.1.1 Filler Stimuli Used in Experiment 1

• Filler type 1: Additional intersective combinations. (1) metalen lepel (metal spoon), (2) groot hotel (big hotel), (3) gestolen jas (stolen jacket), (4) rijpe appel (ripe apple), (5) hete soep (hot soup).

• Filler type 2: Highly familiar (specialized) combinations. (1) lekke band (flat tire), (2) eerste hulp (first aid), (3) gouden medaille (golden medal), (4) tamme kastanjes (tame maroon), (5) witte haai (white shark).

• Filler type 3: Meaningless combinations. (1) wrede deur (savage door), (2) spontaan gebit (spontaneous denture), (3) tochtig bier (draughty beer), (4) machtige spons (mighty sponge), (5) zachte vliegtuig (soft airplane), (6) dwaze drop (silly licorice), (7) pezig riool (tendony sewage), (8) brave folder (‘nice behaving’ folder), (9) dreigende veter (threatening bootlace), (10) brutale steen (brutal stone), (11) roerige bril (restless spectacles), (12) duizelige klok (dizzy watch), (13) blauwe klacht (blue complaint), 14. stille kam (quiet comb), 15. sluwe cadeau (sly present), 16. zoete mouw (sweet sleeve), 17. boze reis (angry journey), 18. slanke storm (slender storm), 19. lenige pap (lithe porridge), 20. zwoele sprong (sultry jump), 21. gespannen zon (tense sun), 22. blonde receptie (blonde reception), 23. rauw hemd (row shirt), 24. serieuze schaar (serious scissors), 25. luchtig stoplicht (airy stoplight).

8.1.2 Appendix B

8.1.2.1 B.1 Analysis of Variance (ANOVA)

8.1.2.1.1 General Comments

Although ANOVA is an extension of the two group comparison embodied in the t-test, understanding ANOVA requires some shift in logic. In the t-test, if we wanted to know if there was a significant difference between two groups we merely subtracted the two means from each other and divided by the measure of random error (standard error). But when it comes to comparing three or more means, it is not clear which means we should subtract from which other means.

Table 8.2 Example 5 means

For example, with five means, we could compare Mean 1 against Mean 2, or against Mean 3, or against Mean 4, or against Mean 5. We could also compare Mean 2 against Mean 3 or against Mean 4, or against Mean 5. We could also compare Mean 3 against Mean 4, or against Mean 5. Finally, we could compare Mean 4 against Mean 5. This gives a total of 10 possible two-group comparisons. Obviously, the logic used for the t-test cannot immediately be transferred to ANOVA. Instead, ANOVA uses some simple logic of comparing variances (hence the name ‘Analysis of Variance’). If the variance amongst the five means is significantly greater than our measure of random error variance, then our means must be more spread out than we would expect due to chance alone.

$$ F = \frac{\text{variance\ among\ sample\ means}}{{\text{variance\ expected\ from\ sampling\ error}}} $$

If the variance amongst our sample means is the same as the error variance, then you would expect an F = 1.00. If the variance amongst our sample means is greater than the error variance, you would get F > 1.00. What we need therefore is a way of deciding when the variance amongst our sample means is significantly greater than 1.00. (An F < 1.00 indicates that error-term variance is higher than the variance among sample means; it is always non-significant). This is achieved by means of the distribution of the F-ratio. F distributions depend on the degrees of freedom associated with the numerator in the ratio and the degrees of freedom associated with the denominator.

(From: http://www.une.edu.au/WebStat/unit_materials/c7_anova/).

8.1.2.1.2 Brief Explanation of Statistical Terms

8.1.2.1.2.1 F-Ratio

The statistic calculated by Analysis of Variance, which reveals the significance of the hypothesis that Y depends on X. It comprises the ratio of two mean-squares: MS[X]/MS[e]. The mean-square, MS, is the average sum of squares, in other words the sum of squared deviations from the mean X or e (as defined above) divided by the appropriate degrees of freedom. This is why the F-ratio is always presented with two degrees of freedom, one used to create the numerator MS[X], and one the denominator, MS[e]. The F-ratio tells us precisely how much more of the variation in Y is explained by X (MS[X]) than is due to random, unexplained, variation (MS[e]). A large proportion indicates a significant effect of X. In fact, the observed F-ratio is connected by a very complicated equation to the exact probability of a true null hypothesis, i.e. that the ratio equals unity, but you can use standard tables to find out whether the observed F-ratio indicates a significant relationship.

8.1.2.1.2.2 Significance

This is the probability of mistakenly rejecting a null hypothesis that is actually true. In the biological sciences a critical value P = 0.05 is generally taken as marking an acceptable boundary of significance. A large F-ratio signifies a small probability that the null hypothesis is true. Thus finding a significant nationality effect: F(3,23) = 3.10, P < 0.05 means that the variation in weight between the samples from four nations is 3.10 times greater than the variation within samples, and that tables of the F-distribution tell us we can have greater than 95% (i.e. >[1–0.05] × 100) confidence in an effect of nationality on weight (i.e. less than 5% confidence in the null hypothesis of no effect).

(From: http://www.soton.ac.uk/~cpd/term.html).

8.1.2.1.3 Experiment 1

Participants. 45 students of the Nijmegen University participated in this experiment.

Materials and Design. The set of stimuli consisted of 45 adjective-noun combinations (see Appendix A). The combinations were formed by pairing 15 nouns with three adjectives each, thus representing the three experimental conditions as presented in Table 8.3 , below.

Table 8.3 Examples of the tree types of adjective-noun combinations used in the present study.

The stimuli in the three conditions were assumed to differ with respect to the level of computational complexity in their semantic interpretation. A within-items design was used. The noun was kept constant, while different conditions were formed by replacing adjectives (yellow car, interesting car, fast car). In order to be able to ascribe possible effects to the manipulated variable Complexity and not the other variables which may also produce effects in the same direction, adjective-noun combinations were matched for length and (written) word frequency of the adjectives (nouns were the same). The mean lengths of the adjectives in the Intersective, Subsective Compatible and Subsective Incompatible conditions are 6.9, 7.4, and 7.5 letters respectively [F < 1; no significant differences], and mean log-frequencies (based on the Celex corpus of 42 million tokens (Baayen et al. 1993) are 3.4, 3.5, and 3.5 respectively [F < 1; no significant differences]. In addition, two rating studies were conducted in order to match the stimuli in the three conditions on the variables salience of the adjectival property in the semantic representation of the noun, and typicality of the combination referent for the category of entities denoted by the noun (e.g., typicality of red apple for the category apple is higher than the typicality of brown apple). This kind of matching is important because salience and typicality may produce effects in the same direction as the factor Complexity manipulated in our experiment (see, e.g., Hampton 1997a; Murphy 1988, 1990). Both rating studies were performed in the same way. The 45 combinations were divided into three lists containing 15 combinations each. On each list, each condition was represented by five combinations. In addition filler combinations of high and low salience/typicality (15 and 10, respectively) were added to the lists. Five practice items were added to each list. In the salience rating study, noun -dimension² pairs (e.g., leaf – green) were printed together with a 7-point rating scales. In the typicality rating study, adjective-noun combinations (e.g., brown soil) were printed together with a 7-point rating scale. Participants (typicality: N = 15, salience: N = 15) were instructed to rate the stimuli for their salience/typicality. In both rating studies the mean scores in the three experimental conditions did not differ significantly. Mean scores for salience (on a 7-point scale in the intersective, subsective compatible and subsective incompatible condition are 4.0, 3.9, 3.7 respectively (all F < 1). Mean scores for Typicality (on a 7-point scale in the same three conditions are 4.4, 4.3, 3.8 respectively (all F < 1). In addition to typicality and salience, familiarity with the combinations is another possible covariate. As an indirect measure of familiarity, the co-occurrence frequency of the constituents of the combinations was used. To that aim we have used corpus data from a (written) corpus based on the Dutch daily newspaper ‘Trouw’, editions from 1993/1994; approximately 163000 tokens. Two out of 45 test combinations appeared in the corpus. The combination dik boek (thick book) appeared 6 times (of which three times in plural form, and 1 time as dik boekwerk where the noun boekwerk is a close synonym of thick book). The combination Nederlandse acteur (Dutch actor) appeared once. This low co-occurrence frequency implies low familiarity of all test combinations.

The argument validity test. In order to differentiate between the intersective and subsective types of combinations, the argument validity test for subjectivity was used (see Table 8.4). For all 45 adjective-noun combinations, arguments with two premises and a conclusion were formed. In this test, valid conclusions indicate that the combination in the first premise is intersective, while invalid conclusions indicate that the combination in the first premise is subsective. Although this test does not differentiate between the subsective compatible and subsective incompatible combinations, it is important to establish that both are indeed subsective. Adjectives in the subsective incompatible condition were selected from the Celex list of adjectives with adverbial usage (e.g., slow - slowly) which renders them event modifiers. The 45 items (arguments) containing our experimental combinations (see Table 8.4) were divided in three lists according to a Latin-square design.

Table 8.4 Examples of usage of an argument validity test as a diagnostic tool to discriminate between de intersective and the subsective combinations

Each list contained 20 items (arguments): five arguments formed with intersective combinations, ten arguments with subsective combinations (five compatible, and five incompatible), and five additional intersective combinations which were added to each list in order to balance the proportion of intersective and subsective combinations. Nine judges were presented booklets containing an instruction and a list of 20 arguments. They were naive with respect to the relation between the argument validity and adjectival type. Their task was to decide, for each argument, whether the conclusion was valid i.e., whether the conclusion followed necessarily from the premises. The judges fulfilled the task individually, at their own pace. A ‘yes’ response classifies the conclusions as valid, indicating that the combination in the first premise is intersective, whereas a ‘no’ response classifies the conclusion as ‘invalid’, indicating that the combination is subsective. The percentage of agreement amongst judges was calculated for each combination. Combinations with minimally 67% agreement were entered into the experimental stimulus set. The combinations with less than 67% agreement were replaced by new ones which were also subjected to the argument test and for which the criteria for inclusion in the experimental set were the same as for the initial set. In this way, 15 triplets of adjective-noun combinations were selected and were used in the two experiments reported below.

Semantic classification experiment. Fifteen participants were randomly assigned to each list. Each participant was presented with 50 adjective-noun combinations: 15 experimental combinations (five in each condition), five intersective filler combinations, five specialized filler combinations (e.g., gold medal, expected to yield fast yes-responses because of high familiarity). Twenty-five meaningless filler combinations (e.g., sensitive folder) were added in order to yield no-responses in the Semantic Classification task. There was no adjective or noun repetition on any of the three lists. The three sets of five adjective-noun combinations on each list were matched for length and log-frequency of adjectives. There were no significant main effects of list or condition [length: all F < 1, frequency: all F < 1], and no interaction effect [length: F < 1, frequency: F < 1].

Procedure. Participants were tested individually, in noise-attenuating booths. Stimuli were presented on a CRT connected to an 80486DX2/66 personal computer which controlled the presentation of the stimuli and the registration of responses. Stimuli (adjective-noun combinations) were presented at the center of the computer screen. Each trial started with the presentation of the fixation mark (*) for 800 ms. After a blank screen for 150 ms, adjective-noun combinations, printed in lower-case letters, were presented for 650 ms. Time-out was set to 1,750 ms after target-offset. Inter-trial interval was 1,500 ms. Participants were instructed to read carefully the adjective-noun combinations appearing on the screen, and to decide as quickly and as accurately as possible whether the combinations were meaningful or meaningless. They were instructed to push the yes-button if they found a combination meaningful; otherwise they had to push the no-button. Both right- and left-handed participants gave yes-responses using their dominant hand. When an error was made on a trial immediately preceding an experimental combination, a dummy item was inserted in between the two in order to attenuate the effects of erroneous responding on the subsequent processing of an experimental item. A set of 28 practice items was presented prior to the experimental session, 4 of which were buffer items at the beginning of the experimental series. The set of practice items had characteristics similar to the experimental set. The whole session lasted about 15 min.

Results

Two items were excluded from the analysis of Reaction times (RTs) in all conditions, because the results of Experiment 2 reported below clearly showed that one of the combinations, vlotte pen (facile pen), involved an idiomatic reading (talented writer); the other combination elicited more than 70% responses in a different category in two conditions. Latencies for the no-responses (M = 18.8%; based on the remaining 13 items) were excluded from the analysis of reaction times (rts). Outliers were determined on the basis of items (per list, condition) and participants (per list, condition) statistics (2SD). No outliers were found. Analysis of rts were conducted with complexity as a within-participants and within-items factor. Overall, the effect of complexity was significant [F₁(2,88) = 6.09, Mse = 8,534, p < .005, F₂(2,24) = 3.41, MSe = 6,501, p = .05]. Planned comparisons confirmed our prediction regarding differences in latencies between the intersective and both subsective combinations (see Table 8.5). Latencies for the intersective combinations are significantly shorter than those for either the subsective compatible [F₁(1,44) = 14.60, MSe = 5,016, p < .001, F₂(1,12) = 7.38, MSe = 2,374, p < .05], or the subsective incompatible combinations [F₁(1,44) = 6.67, MSe = 12,368, p < .05, F₂(1,12) = 5.02, MSe = 8,610, p = .05]. However, latencies in the latter two conditions did not differ significantly [F₁ < 1, F₂ < 1]. The finding of significant differences between the intersective and both subsective conditions supports the hypothesis of lower computational complexity for the former than for the latter two types of combinations. The hypothesis that subsective incompatible combinations are the most complex is not supported in the analysis of rts.

Table 8.5 Mean latencies (in milliseconds), and percentages of ‘No’ responses (in parentheses) obtained in Experiment 1

The analysis of percentages of no-responses was conducted with all items (N = 15). (The removal of the same two items as in the analysis of rts did not affect the outcomes of the analyses). Mean percentages of ‘no’ responses per condition are presented in Table 8.5. The three conditions differed from each other only in the analysis by participants: intersective vs. subsective compatible – [F₁(1,44) = 4.60, MSe = 189.29, p < .05, F₂ < 1]; intersective vs. subsective incompatible – [F₁(1,44) = 28.54, MSe = 262.02, p < .001, F₂(1,14) = 4.16, MSe = 598.31, p > .05]; subsective compatible vs. subsective incompatible [F₁(1,44) = 18.37, MSe = 176.36, p < .001, F₂(1,14) = 2.43, MSe = 445.08, p > .10].

8.1.2.1.4 Experiment 2

Method

Participants. The same 45 participants as in the Experiment 1 took part in the present experiment. All were paid for their participation.

Materials and Design. In this experiment, the same materials were used as in the Experiment 1, with the exception of the ‘meaningless’ filler combinations used only in Experiment 1. Forty-five experimental combinations were divided in three lists, so that each list contained 15 combinations: five in each of the three conditions. In addition, each list was supplemented with five filler intersective combinations (in order to counterbalance the number of intersective and subsective combinations), and five practice combinations. For each list, three different randomizations were made. The lists were counterbalanced across the two experiments. This way, participants responded to different sets of stimuli in each part of the study.

Procedure. The participants were tested individually. They received a booklet containing an instruction to perform a paraphrase task, and a list of 25 combinations, 5 of which were practice combinations at the beginning of each list. They were instructed to write down paraphrases for the combinations, reflecting as precisely as possible how they interpreted them. They were told that the combinations may vary with respect to how easily they can be interpretated. After reading the instruction, they performed the task at their own pace. The whole session lasted approximately 10 min. Participants performed this task after taking part in Experiment 1. They had a short break between the two experiments.

Criteria for the Classification of the Paraphrase Task Responses

• 1. Intersective. Responses are simple paraphrases of the combinations. No additional noun-related concepts are present. Adjectives and nouns may be substituted by their synonyms. In Example 1 below, the response is a simple paraphrase with no additional noun-related concepts inserted. In Example 2, there is a substitution such that the synonymous more than 70 years old is substituted for the adjective elderly.

  1.	groene gesp: Een gesp die groen is.
  	green clasp: A clasp that green is.
  	(A clasp which is green.)
  2.	bejaarde tandarts: Tandarts van meer dan 70 jaar oud.
  	elderly dentist: Dentist of more than 70 years old.
  	(A dentist who is more than 70 years old.)

• 2. Subsective compatible. Paraphrases contain one or more simple (non-event) noun properties which define a nominal subset. In Example 3 below, strong poison is interpreted as very concentrated poison. In Example 4 interesting novel} is interpreted as a novel with an interesting plot. In both cases, the interpretations involve knowledge related to the nouns and not the adjectives, This is suggested by the fact that changing the noun (or at least the noun class) automatically results in a different insertion (e.g., a strong horse would not be a very concentrated horse, similarly an interesting car would not be a car with an interesting plot).

  3.	sterk gif: Gif dat zeer geconcentreerd is.
  	strong poison: Poison that very concentrated is.
  	(A very concentrated poison.)
  4.	interessante roman: Een roman die een interessant verhaal heeft.
  	interesting novel: A novel that an interesting plot has.
  	(A novel with an interesting plot.)

• 3. Subsective incompatible (event mapping). Paraphrases of the event-mapping combinations contain one or more noun-related events. In Example 5 below, slow dentist is interpreted as a dentist which works slowly, that is, the event to work associated with the noun dentist is modified. In Example 6, urgent letter is interpreted as a letter which has to be delivered urgently. In both cases, adjectival modification became adverbial modification (or manner pps), modifying the events of working and of delivering, respectively.

  5.	trage tandarts: Een tandarts die langzaam werkt.
  	slow dentist: A dentist who slowly works.
  	(A dentist who works slowly.)
  6.	urgente brief: Een brief die met spoed moet worden bezorgd.
  	urgent letter: A letter that with urgency must be delivered.
  	(A letter that must be delivered urgently.)

• 4. Idiosyncratic. Either it is not clear from the paraphrase what the meaning of the combination should be, or no agreement amongst the judges can be reached regarding the classification of a response (e.g., for the combination versleten machine (worn-out machine) the paraphrase classified as idiosyncratic was a machine which should be replaced). This is an inference based on knowledge of the world, rather than a representation of the content of the semantic interpretation of the combination.

Results

On the basis of the criteria outlined above, the responses were scored by two judges (experimenters), independently of each other, as indicating one of the three types of semantic interpretation, namely intersective, subsective compatible, or subsective incompatible (event mapping). The final scoring involved reaching consensus amongst judges. Responses for which no consensus could be obtained were placed in the category idiosyncratic, together with the responses that were idiosyncratic by consensus. In each condition responses were classified in four categories, namely intersective, subsective property mapping, subsective event mapping, and idiosyncratic. For each condition, one of the response types is congruent with the combination type while the others are incongruent. For instance, in the condition intersective, a response classified as indicating an intersective kind of interpretation is congruent while all other responses are incongruent.

In general, the results are convergent with those obtained in Experiment 1 (see Fig. 8.1 below). The results were analyzed using the non-parametric Friedman test (Friedman anova) and involving factor response type. We looked at differences between the conditions in percentages of congruent responses. Overall, the percentage of idiosyncratic responses was very low (M = 2.1%) with 2.22% in the intersective condition, 1.33% in the subsective compatible condition, and 2.67% in the subsective incompatible condition. The three types of combinations did not differ significantly with respect to percentages of idiosyncratic responses [$\chi_{{(2)}}^2$ < 1, p = .92]. The highest percentage of responses congruent with the combination type was obtained in the conditions intersective (71%), and subsective incompatible (76%). The lowest percentage of congruent responses was obtained in the subsective compatible condition (39%). However, in this condition half (39%) of the subsective kind of responses involved event mappings. Although these interpretations are also subsective, contrary to our expectation, they involved noun related events. In addition, the three conditions differed significantly in percentages of each of the three response types (except the idiosyncratic). The differences were in the expected directions. Intersective – [$\chi_{{(2)}}^2$ = 65.34, p < .001]; subsective property mapping – [$\chi_{{(2)}}^2$ = 32.08, p < .001]; and subsective event mapping – [$\chi_{{(2)}}^2$ = 63.33, p < .001]. These findings are being discussed in the main text, above.

Fig. 8.1

Mean response type percentages per combination type in experiment 2