Why Distributed Representation is Inherently Non-Symbolic
There are many conflicting views concerning the nature of distributed representation, its compatibility or otherwise with symbolic representation, and its importance in characterizing the nature of connectionist models and their relationship to more traditional symbolic approaches to understanding cognition. Many have simply assumed that distribution is merely an implementation issue, and that symbolic mechanisms can be designed to take advantage of the virtues of distribution if so desired. Others, meanwhile, see the use of distributed representation as marking a fundamental difference between the two approaches. One reason for this diversity of opinion is the fact that the relevant notions — especially that of distribution — are rarely adequately characterized before addressing the issues. At this level of generality, an adequate characterization is one that is sufficiently abstract to subsume most paradigm cases of representation of a given type, yet also sufficiently precise to give real theoretical bite when addressing questions such as those raised above. This paper advances a definition of distributed representation and shows that, understood this way, distribution is in fact incompatible with the core notion of symbolic representation found in the cognitive science literature. For this reason, genuinely distributed connectionist models cannot be, or implement, physical symbol systems (Newell & Simon 1976) or “classical” symbolic models (Fodor and Pylyshyn 1988). Thus, I am endorsing the view that distributed connectionist models do indeed present a radical new approach to modeling cognitive processes.
Unable to display preview. Download preview PDF.
- Chalmers D.J. (1990) Syntactic transformations on distributed representations. forthcoming in Connection Science.Google Scholar
- Fodor J. & McLaughlin B. (forthcoming) What is wrong with tensor product connectionism? in Horgan T. & Tienson J. (eds) Connectionism and the Philosophy of Mind.Google Scholar
- Elman J. L. (1989) Representation and structure in connectionist models. CRL Technical Report 8903, Center for Research in Language, University of California San Diego La Jolla CA 92093.Google Scholar
- Haugeland J. (1981) Analog and analog. Philosophical Topics; 12: 213–225.Google Scholar
- Pollack J. (1988) Recursive auto-associative memory: Devising compositional distributed representations. Proceedings of the Tenth Annual Conference of the Cognitive Science Society. Montreal, Quebec, Canada.Google Scholar
- Pylyshyn Z. (1984) Computation and Cognition: Toward a Foundation for Cognitive Science. Cambridge: MIT/Bradford.Google Scholar
- Rumelhart and McClelland (1986): On learning the past tenses of English verbs. in McClelland J.L., Rumelhart D.E. and the PDP Research Group Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge MA: Bradford/MIT Press; 216–271.Google Scholar
- Touretzky D. S. (1989) BoltzCONS: Dynamic Symbol Structures in a Connectionist Network. Technical Report CMU-CS-89–182, Department of Computer Science, Carnegie Mellon. (To appear in a special issue of Artificial Intelligence on connectionist symbol processing.)Google Scholar
- van Gelder T.J. (1990a) Compositionality: A Connectionist Variation on a Classical Theme. Cognitive Science (forthcoming).Google Scholar
- van Gelder T.J. (1990b) What is the ‘D’ in ’PDP’? An Overview of the Concept of Distribution. forthcoming in Stich S., Rumelhart D. & Ramsey W. (eds) Philosophy and Connectionist Theory Hillsdale N.J.: Lawrence Erlbaum Associates 1990.Google Scholar