Distributed Subsymbolic Representations for Natural Language: How many Features Do You Need?
In a Natural Language Understanding system, be it connectionist or otherwise, it is often desirable for representations to be as compact as possible. In this paper we present a simple algorithm for thinning down an existing set of distributed concept representations which form the lexicon in a prototype story paraphrase system which exploits both conventional and connectionist approaches to Artificial Intelligence (AI). We also present some performance measures for evaluating a lexicon’s performance. The main result is that the algorithm appears to work well — we can use it to balance the level of detail in a lexicon against the amount of space it requires. There are also interesting ramifications concerning meaning in natural language.
KeywordsBark Alan Glean
Unable to display preview. Download preview PDF.
- 3.Fillmore C. The case for Case. In Bach E and Harms RT (eds) Universals of Linguistic Theory. Holt, Rinehart and Winston New York, 1968, pp 1–88Google Scholar
- 4.Wilks Y. An Artificial Intelligence Approach to Machine Translation. In: Schank RC and Colby KM (eds) Computer Models of Thought and Language. WH Freeman San Fransciso, pp 114–151Google Scholar
- 5.Schank RC, Abelson RP. Scripts, Plans and Knowledge. In Proceedings of the 4th UCAI, Tbilisi, USSR, 1975Google Scholar
- 6.Dyer MG. In-Depth Understanding. A computer model of integrated processing for narrative comprehension. Research Report Number 219, Department of Computer Science, Yale University, 1982Google Scholar
- 8.Katz JJ, Fodor JA. The structure of a semantic theory. Language 1963; 39(2): pp 170–210. Also in Fodor JA, Katz JJ (eds) The structure of Language. Prentice Hall Englewood Cliffs.NJ 1964Google Scholar
- 9.Hinton GE. Implementing semantic networks in parallel hardware. In Hinton GE. and Anderson JA. (eds) Parallel models of associative memory. Lawrence Erlbaum Associates Hillsdale NJ, 1981, pp 161–187Google Scholar
- 11.Hinton GE, Sejnowsi TJ. Learning Semantic Features. In: Proceedings of the 6th Annual Conference of the Cognitive Science Society, 1984, pp 63–70Google Scholar
- 12.Hinton GE. Learning distributed representations of concepts. In: Proceedings of the 8th Annual Conference of the Cognitive Science Society, 1986, pp 1–12Google Scholar
- 13.Miikkulainen R, Dyer MG. Building distributed representations without microfeatures. Technical Report UCLA-AI-87–17, AI Laboratory, Computer Science Department, University of California at Los Angeles, CA, 1987Google Scholar
- 14.Miikkulainen R, Dyer MG. A Modular Neural Network Architecture for Sequential Paraphrasing of Script-Based Stories. Technical Report UCLA-AI89–02, AI Lab, Computer Science Department, UCLA, LA, LA 90024, 1989Google Scholar
- 15.Elman JL. Finding structure in time. TR 8801, Center for Research in Language, University of California, San Diego, CA. April, 1988Google Scholar
- 16.Chomsky N. Aspects of the theory of syntax. MIT Press Cambridge MA, 1965Google Scholar
- 19.Rosch E. Human categorization. In: Warren N (ed) Advances in cross-cultural psychology ( Vol I ). Academic Press London UK, 1977Google Scholar
- 21.Brachman RJ. What IS-A is and isn’t: An Analysis of Taxonomic Links in Semantic Networks. IEEE Computer 16(10, 1983, pp 30–36Google Scholar
- 22.Sutcliffe RFE. A Parallel Distributed Processing Approach to the Representation of Knowledge for Natural Language Understanding. Unpublished doctoral thesis, University of Essex, UK, 1988Google Scholar
- 23.Rumelhart DE, Hinton GE, Williams RJ. Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL (eds) Parallel Distributed Processing: Explorations in the microstructure of cognition. Volume I: foundations. MIT Press Cambridge MA, 1986, pp 318–362Google Scholar
- 24.Minsky L, Papert, S. Perceptions. MIT Press Cambridge MA, 1988 (1969)Google Scholar
- 25.Sutcliffe RFE. Representing Meaning using Microfeatures. In: Reilly R, Sharkey NE (eds) Connectionist Approaches to Natural Language Processing. Erlbaum Hillsdale NJ, 1991Google Scholar
- 26.Salton G (ed). The SMART Retrieval System - Experiments in Automatic Document Processing. Prentice-Hall Englewood Cliffs NJ, 1971.Google Scholar
- 28.Everitt B. Cluster Analysis. Heinemann Halstead John Wiley London 1974Google Scholar