Advertisement

Connectionist Representations for Natural Language: Old and New

  • Noel E. Sharkey
Part of the Informatik-Fachberichte book series (INFORMATIK, volume 252)

Abstract

Connectionist natural language processing research has been in the literature for less than a decade and yet it is already claimed that it has established a whole new way of looking at representation. This article presents a survey of the main representational techniques employed in connectionist research on natural language processing and assesses claims as to their novelty value i.e. whether or not they add anything new to Classical representation schemes.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Chalmers, D.J. (in press) Syntactic Transformations on Distributed Representations. Connection Science, 2.1. Google Scholar
  2. Cottrell, G.W. (1985) A Connectionist Approach to Word Sense Disambiguation. PhD Thesis, TR154, Department of Computer Science, University of Rochester, NY.Google Scholar
  3. Cottrell, G.W., Munro, P. & Zipser, D. (1989) Image Compression by Back Propagation: An Example of Extensional Programming. In N.E. Sharkey (Ed) Models of Cognition: A Review of Cognitive Science. Norwood, N.J.: Ablex.Google Scholar
  4. Dolan, C.P. & Smolensky, P. (1989) Tensor Production System: a Modular Architecture and Representation. Connection Science 1(1), 53–68.CrossRefGoogle Scholar
  5. Elman, J.L (1988) Finding the structure in time. TR 880, CRL, University of California, San Diego.Google Scholar
  6. Elman, J.L. (1989) Representation and structure in connectionist models. IR 8903, CRL, University of California, San Diego.Google Scholar
  7. Feldman, J.A. (1989) Neural Representation of Conceptual Knowledge. In N.E. Sharkey (Ed) Models of Cognition: A Review of Cognitive Science. Norwood, N.J.: Ablex.Google Scholar
  8. Fillmore, C.J. (1968) The Case for Case. In E. Bach and R. Harris (Eds) Universalsin Linguistic Theory. Holt, Rhinehart & Winston.Google Scholar
  9. Fodor, J.A. & McLaughlin, B. (1990) Connectionism and the Problems of Systemacity I:Why Smolensky’s Solution Doesn’t Work. Cognition, 35, 183–204.CrossRefGoogle Scholar
  10. Fodor, J.A, & Pylyshyn, Z.W. (1988). Connectionism and Cognitive Architecture: A Critical Analysis. Cognition, 28, 2–71.CrossRefGoogle Scholar
  11. Hinton, G.E. (1981) Implementing Semantic Networks in Parallel Hardware. In G.E. Hinton & J.A. Anderson (Eds) Parallel Models of Associative Memory. Hillsdale, N.J.: Lawrence Erlbaum.Google Scholar
  12. Hinton, G.E. (1986) Learning Distributed Representations of Concepts. Proceedings of the Eighth Annual Conference of the Cognitive Science Society. Google Scholar
  13. Hinton, G.E. (1989) Connectionist Learning Procedures. Artificial Intelligence, 40, 184–235.CrossRefGoogle Scholar
  14. Hinton, G.E., McClelland, J.L. & Rumelhart, D.E. (1986) Distributed Representations. In D.E. Rumelhart & J.L. McClelland (Eds) Parallel Distributed Processing, Volume 1. Cambridge, MA: MIT.Google Scholar
  15. Hopfield, J.J. (1982) Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences, USA, 79, 2554–2558.MathSciNetCrossRefGoogle Scholar
  16. Hornik, K. Stinchcombe, M. & White, H. (1989) Multilayer Feedfoewrad Network are Univesal Approximators. Discussion paper 88–45R. Department of Economics, UCSD.Google Scholar
  17. Jordan, M.I. (1986) Attractor Dynamics and parallelism in a Connectionist Sequential Machine. Proceedings of the 8th Annual Conference of the Cognitive Science Society, Amherst, MA. 531–545.Google Scholar
  18. Kohonen, T. (1982) Clustering, Taxonomy, and Topological Maps of Patterns. In M. Lang (Ed) Proceedings of the Sixth International Conference on Pattern Recognition. Silver Spring, MD: IEEE Computer Society Press.Google Scholar
  19. Kolen, J.F. & Pollack, J.B. (1990) Back Propagation is Sensitive to Initial Conditions. Technical Report 90-JK-BPSIC. Ohio Satat University.Google Scholar
  20. McClelland, J.L. & Kawamoto A.H. (1986) Mechanisms of Sentence Processing:Assigning Roles to Constituents. In J.L. McClelland & D.E. Rumelhart (Eds) Parallel Distributed Processing Volume 2. Cambridge, MA: MIT.Google Scholar
  21. McClelland, J.L. & Rumelhart, D.E. (1981) An Interactive Activation Model of Effects in Letter Perception: Part I. An Account of Basic Findings. Psychological Review, 88, 375–407.CrossRefGoogle Scholar
  22. Miikkulainen, R. & Dyer, M. G. (1987) Building Distributed Representations without Microfeatures. Technical Report UCLA-AI-87–17, AI Laboratory, Computer Science Department, University of California at Los Angeles, CA.Google Scholar
  23. Minsky, M. (1975) A Framework for Representing Knowledge. In P.H. Winston (Ed) The Psychology of Computer Vision. New York: McGraw-Hill.Google Scholar
  24. Minsky, M. (1975) A Framework for Representing Knowledge. In P.H. Winston (Ed) The Psychology of Computer Vision. New York: McGraw-Hill.Google Scholar
  25. Robbins, A. (1989) The Distributed Representation of Type and Category. Connection Science1. 4.CrossRefGoogle Scholar
  26. Schank, R.C. (1972) Conceptual dependency: a theory of natural language understanding. Cognitive Psychology, 3, 552–631.CrossRefGoogle Scholar
  27. Sejnowski, T.J. & Rosenberg, C.R. (1986) A Parallel Network that Learns to Read Out Loud. Technical Report JHU/SECS-86/01. John Hopkins University.Google Scholar
  28. Servan-Schreiber, D., Cleermans, A., & McClelland, J.L. (1988) Encoding sequential structure in simple recurrent nets. TR CMU-CS-88–183. Computer Science department, Carnegie-Mellon University.Google Scholar
  29. Sharkey, N.E. (1989a) The Lexical Distance Model and Word Priming. Proceedings of the Eleventh Cognitive Science Society. Google Scholar
  30. Sharkey, N.E. (1989b) Lexical Representations for Prepositional Attachment. Report to British Telecom CONNEX Project. Google Scholar
  31. Sharkey, N.E. (1989c) A PDP Learning Approach to Natural Language Understanding. In I. Aleksander (Ed) Neural Computing Architectures. London: North Oxford Academic.Google Scholar
  32. Sharkey, N.E. (1989c) A PDP Learning Approach to Natural Language Understanding. In I. Aleksander (Ed) Neural Computing Architectures. London: North Oxford Academic.Google Scholar
  33. Shastri L. & Feldman J.A. (1986) Neural Nets, Routines and Semantic Networks. In N.E. Sharkey (Ed) Advances in Cognitive Science. Chichester: Ellis Horwood.Google Scholar
  34. Shastri L. & Feldman J.A. (1986) Neural Nets, Routines and Semantic Networks. In N.E. Sharkey (Ed) Advances in Cognitive Science. Chichester: Ellis Horwood.Google Scholar
  35. Smolensky, P. (1988) On the Proper Treatment of Connectionism. Behavioraland Brain Sciences, 11, 1–74.CrossRefGoogle Scholar
  36. Smolensky, P. (1988) On the Proper Treatment of Connectionism. Behavioraland Brain Sciences, 11, 1–74.CrossRefGoogle Scholar
  37. Smolensky, P. (1988) On the Proper Treatment of Connectionism. Behavioraland Brain Sciences, 11, 1–74.CrossRefGoogle Scholar
  38. Touretzky, D.S. & Geva, S. (1987) A Distributed Connectionist Representation for Concept Structures. Proceedings of the Ninth Annual Conference of the Cognitive Science Society, 155–164.Google Scholar
  39. Touretzky, D.S. & Hinton, G.E. (1988) A Distributed Connectionist Production System. Cognitive Science 12(3), 423 - 466.CrossRefGoogle Scholar
  40. Van Gelder, T. (1990) Compositionality: a Connectionist Variation on a Classical Theme. Cognitive Science, 14.Google Scholar
  41. Wickelgren, W.W.A.(1969) Context-sensitive coding, associative memory, and serial order in (speech) behavior. Psychological Review, 76, 1–15.CrossRefGoogle Scholar
  42. Willshaw, D.J. & von der Malsburg, C. (1979) A Marker Induction Mechanism for the Establishment of Ordered Neural Mapping: Its Application to the Retino-tectal Connections. In Philos, Trans, Roy, Soc. Lond. B 287, 203 - 243.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1990

Authors and Affiliations

  • Noel E. Sharkey
    • 1
  1. 1.Department of Computer ScienceUniversity of ExeterUK

Personalised recommendations